User struggles with local LLM mapping PDF to JSON

A Reddit user reports that after three months of attempting to convert PDFs to JSON, they have settled on Docling for parsing but face persistent issues with the subsequent data mapping step. The user employs a local Qwen 3.5-9B model to transform Docling's markdown output into a specific JSON format.

• The primary failure mode involves incorrect labeling of counts and amounts, as well as the replacement of values that should remain unchanged.
• Prompt engineering attempts have failed because adjustments create new problems rather than providing a generalized solution.
• The user suspects the issue stems from inherent limitations in local LLM capabilities rather than flaws in the workflow itself.

The post highlights the difficulty of achieving reliable, structured data extraction from PDFs using only local models without relying on external APIs.