Extractions

An extraction is one document solved against one schema. Tickmarks doesn’t store extractions at all. You get the extraction back in the response, and if you want it later, you store it yourself.

That matters as soon as a person reviews the result. When a reviewer spots a value the model missed, or wants to settle a conflict, you send the stored extraction to the edit endpoint together with one change, and get a new extraction back without the model running again. Everything Tickmarks needs for that is inside the extraction object, so if you might edit an extraction later, store the whole object, not just response.

Endpoint	Scope
`POST /v1/extractions`	`extractions:create`
`POST /v1/extractions/edit`	`extractions:edit`

The extraction object

Both endpoints return this object. The one exception is a reconciliation preview, which returns a preview instead.

{
  "object": "extraction",
  "reference": "batch-7/invoice-42",
  "language": "en",
  "document": "Invoice INV-004211. Net EUR 100.00, VAT EUR 21.00, total EUR 121.00.",
  "schema": { "hydra_schema": "1.0", "name": "invoice", "fields": ["…"], "relations": ["…"] },
  "options": { "on_conflict": "withhold", "fallback": "off" },
  "response": {
    "hydra_response": "1.0",
    "schema_hash": "sha256:5b1f…",
    "data": {
      "number": "INV-004211",
      "net": { "amount": "100.00", "currency": "EUR" },
      "vat": { "amount": "21.00", "currency": "EUR" },
      "total": { "amount": "121.00", "currency": "EUR" }
    },
    "fields": { "…": "…" }
  },
  "active_bundles": [0],
  "revision": 0,
  "session": "hss1.eyJidW5kbGVzIjpb…",
  "warnings": [],
  "usage": { "input_tokens": 24, "reads": 1, "window_tokens": 4096 }
}

Member	Type	Description
`object`	`"extraction"`	Always `"extraction"`.
`reference`	string \| null	The `reference` you sent when creating it.
`language`	`"en"` \| `"nl"`	The document language used.
`document`	string	The document text. Offsets in `response` count into this text.
`schema`	object	The schema, with defaults filled in.
`options`	object	The options used, with defaults filled in.
`response`	object	The Tickmarks response.
`active_bundles`	integer[]	The evidence bundles in this extraction. A bundle is a batch of evidence added together: the model’s first pass is bundle `0`, and each insert adds another. Use these ids to retract a bundle or withdraw one of its mentions.
`revision`	integer	How many edits have been applied, starting at `0`.
`session`	string	Opaque and signed: the evidence bundles and the edits made so far, which Tickmarks needs to edit the extraction. Leave it as it is. Tickmarks rejects a value that was altered, or that’s sent with a different document or schema.
`warnings`	string[]	Non-fatal notes about the run.
`usage`	object	What this call consumed: `input_tokens`, the document’s size in tokens; `reads`, the number of windows the model read, each holding up to `window_tokens` tokens (4,096) of schema plus document; and `window_tokens`. An edit replays the session and doesn’t run the model, so its `reads` is `0`.

Create an extraction

POST /v1/extractions

Runs a document through Tickmarks with a schema and returns the extraction.

Auth: API key with scope extractions:create.

Body

Parameter	Type	Required	Default	Description
`document`	string	yes		The document text, 1 to 50,000 characters after trimming. Offsets in the response count into this text.
`schema`	object	yes		A schema with 1 to 32 fields.
`language`	`"en"` \| `"nl"`	no	`"en"`	The document’s language.
`reference`	string	no		Your own identifier, up to 128 characters, returned unchanged.
`options.on_conflict`	`"withhold"`	no	`"withhold"`	Conflicting fields get no value, and the response explains the conflict. This is the only value today.
`options.fallback`	`"off"` \| `"quote-first"` \| `"offset"`	no	`"off"`	Ask a second model for evidence on fields that are still incomplete after solving. With `quote-first` it locates its answer by quoting the text; with `offset`, by character position. Fallback isn’t available yet: any value other than `"off"` gets `400 invalid_request`.

Scans, photos and PDFs

Tickmarks reads text, not images. If your document is a scan, a photo or a PDF, turn it into text first and send that text as document:

A PDF with selectable text, one made by software rather than scanned, doesn’t need OCR. Extract its text with a PDF library or a tool such as pdftotext.
A scan or a photo needs OCR (optical character recognition). Any OCR service or library that returns plain text works, for example the open-source engine Tesseract. Set its language to match language.

Keep the text exactly as you sent it. The evidence in the response points at character offsets in that text, so you can show each value next to where it was read. OCR mistakes reach Tickmarks as they are: a misread amount that breaks its field or a relation is dropped or flagged, but one that still fits can’t be told apart from the real value. Relations are what catch it.

Responses

Status	Meaning
`200`	The body is the extraction object.
`400 invalid_request`	The body failed validation; see `error.details`.
`401 unauthorized`	See Authentication.
`429 rate_limited`	Rate limit reached.
`502 upstream_error`	The extraction service failed.
`503 pipeline_unavailable`	The extraction service is unavailable. Try again shortly.

A read is one pass of Hydra Extract over up to 4,096 tokens of schema plus document. A longer document is split into overlapping windows, one read each. See pricing for what that costs.

Example

curl -X POST "$TICKMARKS_API/v1/extractions" \
  -H "Authorization: Bearer $TICKMARKS_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "document": "Invoice INV-004211. Net EUR 100.00, VAT EUR 21.00, total EUR 121.00.",
    "reference": "batch-7/invoice-42",
    "schema": {
      "hydra_schema": "1.0",
      "name": "invoice",
      "fields": [
        { "name": "number", "type": "string", "pattern": "^INV-[0-9]{6}$" },
        { "name": "net", "type": "money", "currency": "EUR", "scale": 2 },
        { "name": "vat", "type": "money", "currency": "EUR", "scale": 2 },
        { "name": "total", "type": "money", "currency": "EUR", "scale": 2 }
      ],
      "relations": [{ "id": "sum", "expr": "total == net + vat" }]
    }
  }'

The response is 200 OK with the extraction object shown above.

A document over 50,000 characters is rejected before anything runs:

{
  "error": {
    "code": "invalid_request",
    "message": "Request validation failed.",
    "details": [
      {
        "origin": "string",
        "code": "too_big",
        "maximum": 50000,
        "inclusive": true,
        "path": ["document"],
        "message": "Too big: expected string to have <=50000 characters"
      }
    ]
  }
}

Edit an extraction

POST /v1/extractions/edit

Use this when a person or another system knows better than the model: to add a value the model missed, to take back evidence that turned out to be wrong, or to settle a conflict. You send an extraction and one change, and Tickmarks applies the change, solves again and returns the new extraction.

Auth: API key with scope extractions:edit.

Body

Parameter	Type	Required	Description
`extraction`	object	yes	An extraction as Tickmarks returned it. Tickmarks reads `document`, `schema`, `language`, `reference`, `options` and `session`, and ignores the other members, so you can leave out `response` to keep the request small.
`edit`	object	yes	The change to make. Its `action` member picks one of the four actions below.

`edit.action`	Does	Returns
`insert`	Adds a value quoted from the document.	A new extraction
`retract`	Withdraws an evidence bundle.	A new extraction
`preview_reconciliation`	Plans how to resolve a conflict.	A preview; the extraction is unchanged
`reconcile`	Applies a previewed plan.	A new extraction

Each edit returns a new extraction with revision one higher. It includes the change you just made, so store it and send it with your next edit if you want that change to carry over.

Candidates

insert and preview_reconciliation take a candidate: a value you propose for a field, quoted from the document so that Tickmarks can check it exists there.

Parameter	Type	Required	Description
`slot`	string	yes	The field name.
`quote`	string	yes	Text that occurs verbatim in the document and parses as the field’s type.
`p`	number	yes	Your confidence, strictly between 0 and 1.

Responses

Status	Meaning
`200`	The new extraction object, or for `preview_reconciliation` a preview (see below).
`400 invalid_request`	The body is invalid, the quote isn’t in the document, the extraction was altered or its `session` doesn’t match its document and schema, or a plan is stale.
`401 unauthorized`	See Authentication.
`429 rate_limited`	Rate limit reached.

Insert evidence

Say the model missed the VAT amount on an invoice and a reviewer supplies it. In this example, the shell variable $EXTRACTION holds the extraction object, as JSON, that you stored earlier:

curl -X POST "$TICKMARKS_API/v1/extractions/edit" \
  -H "Authorization: Bearer $TICKMARKS_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "extraction": '"$EXTRACTION"',
    "edit": { "action": "insert", "candidate": { "slot": "vat", "quote": "EUR 21.00", "p": 0.95 } }
  }'

The new extraction has the inserted evidence as a new bundle in active_bundles, so you can retract it later if the reviewer turns out to be wrong.

p must be strictly between 0 and 1, so even 1 is rejected:

{
  "error": {
    "code": "invalid_request",
    "message": "Request validation failed.",
    "details": [
      {
        "origin": "number",
        "code": "too_big",
        "maximum": 1,
        "inclusive": false,
        "path": ["edit", "candidate", "p"],
        "message": "Too big: expected number to be <1"
      }
    ]
  }
}

If you stored only response and send that back, the session member is missing and you get 400 too:

{
  "error": {
    "code": "invalid_request",
    "message": "Request validation failed.",
    "details": [
      {
        "expected": "string",
        "code": "invalid_type",
        "path": ["extraction", "session"],
        "message": "Invalid input: expected string, received undefined"
      }
    ]
  }
}

Retract a bundle

Withdraws all the evidence in one bundle, such as an insert that turned out to be wrong. To withdraw a single mention instead, for example one misread number from the model’s first pass, reconcile.

Parameter	Type	Required	Description
`bundle`	integer	yes	A bundle id from the extraction’s `active_bundles`.

{ "extraction": { "…": "…" }, "edit": { "action": "retract", "bundle": 1 } }

Reconcile a conflict

When fields have status conflict, the response’s components[].conflicts lists the mentions that disagree (a mention is one piece of text the model read as a value) and the possible repairs. Resolving a conflict takes two requests: the first shows what would change, so a reviewer can check it, and the second applies it.

1. Preview. This changes nothing.

Parameter	Type	Required	Description
`intent`	string	yes	Why you’re making the change, in 1 to 500 characters. It’s kept in the plan and, once applied, in the extraction.
`withdraw`	array	yes	At least one `{ "bundle": integer, "mention": integer }` to withdraw.
`candidate`	object	no	A replacement candidate.

{
  "extraction": { "…": "…" },
  "edit": {
    "action": "preview_reconciliation",
    "intent": "The total line reads 121.00; 112.00 is a typo in the summary.",
    "withdraw": [{ "bundle": 0, "mention": 4 }],
    "candidate": { "slot": "total", "quote": "EUR 121.00", "p": 0.9 }
  }
}

Watch the type of the mention id: responses show it as a string ("4"), but here you send it as an integer (4).

The response is a preview of what the plan would do. impact is the part to show a reviewer: it lists only the fields whose value would change.

{
  "object": "reconciliation_preview",
  "intent": "The total line reads 121.00; 112.00 is a typo in the summary.",
  "withdraw": [{ "bundle": 0, "mention": 4 }],
  "revision": 0,
  "after": { "hydra_response": "1.0", "data": { "total": { "amount": "121.00", "currency": "EUR" } }, "…": "…" },
  "impact": {
    "total": { "before": null, "after": { "amount": "121.00", "currency": "EUR" } }
  },
  "plan": "hsp1.eyJyZXZpc2lvbiI6MCwi…"
}

Member	Type	Description
`object`	`"reconciliation_preview"`	Always `"reconciliation_preview"`.
`intent`, `withdraw`		As you sent them.
`revision`	integer	The extraction revision the plan was made for.
`after`	object	The response the extraction would have after reconciling.
`impact`	object	For each field whose value would change, its `before` and `after` value.
`plan`	string	Opaque and signed. Send it back to apply the plan.

2. Apply. Send the same extraction again, with the plan.

Parameter	Type	Required	Description
`plan`	string	yes	From the preview.

{ "extraction": { "…": "…" }, "edit": { "action": "reconcile", "plan": "hsp1.eyJyZXZpc2lvbiI6MCwi…" } }

A plan only applies to the revision it was previewed at. If you send it with a different extraction, or one that’s been edited since, the plan no longer describes what would happen, so it’s rejected with 400. Preview again with the extraction you have now, and apply that plan.