Schema-Guided Extraction

Schema-guided extraction is the practice of passing a target schema, typically JSON Schema or a Pydantic model, to a language model alongside page content, so the output arrives typed and validated instead of as prose you then have to parse. It replaces per-site CSS selectors with a single description of the shape you want, which is why one extraction definition can run against page layouts it has never encountered. This is the mechanism underneath the whole agentic extraction category.

Two separate layers do the work, and production systems need both. Generation-time constraint, usually called constrained decoding or strict mode, restricts token sampling so the model can only emit tokens that keep the output schema-valid; by 2026 OpenAI, Anthropic, and Google all ship a native structured-output mode of this kind. Post-hoc validation parses the returned JSON against the schema in your own code and rejects what fails. They solve different problems: the first stops mechanical breakage such as truncated objects and misspelled keys, the second decides what your downstream systems are allowed to trust. Neither is an accuracy guarantee. Constrained decoding ensures a price field will be a number. It says nothing about whether it is the right number, or whether the model read the strikethrough list price instead of the sale price. Schema failures trigger a retry; wrong values pass silently and are caught only by evaluation.

The cost model differs sharply from selector-based scraping. Every page costs a model invocation, the schema itself occupies input tokens on every request, and nested schemas cost more of them. Vendors expose this as a multiplier over a plain fetch rather than as a separate product. Firecrawl's published credit table bills a standard page scrape at one credit and adds several more per page for JSON extraction, so a workflow with structured output enabled runs at a multiple of the base rate before any retries. Retry loops multiply it again, and constrained decoding adds latency overhead on top.

Deterministic selectors still win in a specific and common case: high volume from a small number of stable sources. If you pull two hundred thousand pages a day from one site that redesigns twice a year, a CSS selector costs microseconds and effectively no money, and the maintenance burden is a few hours annually. Schema-guided extraction inverts that arithmetic on the long tail, where you need fifty fields from four hundred low-volume sources and the dominant cost is a human authoring and repairing selectors. The common arrangement is to run both: deterministic parsers on the top sources, model-driven extraction on everything else, and a shared validation layer so both paths fail the same way.

Vendor approaches differ in what you are allowed to specify. Firecrawl accepts a JSON Schema on its scrape endpoint. ScrapeGraphAI takes a natural-language prompt plus a Pydantic model. AgentQL asks you to express the target as a query over the page rather than as a schema. Diffbot goes the opposite direction, applying its own fixed ontology so callers receive a consistent entity shape without supplying a schema at all, which is a good fit when your fields match its ontology and a poor one when they do not.