A user on the Hugging Face forums is developing a custom logits processor that enforces verbatim generation of input text alongside inline classification tags. While this approach works for models without chain-of-thought reasoning, the user now aims to implement a two-phase generation process where the model reasons freely before producing the classified output.
- The user previously implemented the processor for direct-output models but needs to handle separate thinking and responding phases.
- Initial ideas involved detecting an end-of-thinking token, but the user notes that APIs like Ollama and vLLM already structure this data internally.
- Access to logits during generation is required, a feature not supported by Ollama and only recently added to vLLM for Hugging Face compatibility.
- The core question is whether there is a way to divide these two phases using only the Hugging Face library without relying on external API structures.