LLM document transformation

LLM document transformation is also called AI Import.

A Control client can ask the Hub's configured LLM connector to transform one or more documents into a JSON value. Each request has one response sent to the Control session that submitted it. The response repeats the caller's unsigned 64-bit transaction ID, so a client can match concurrent requests without using the text conversation history.

Clients offer the action on modern Hubs and hide it on legacy Hubs. The Control AI availability rule applies before choosing documents or transmitting a transformation: an unavailable action reports an immediate error. The Hub must handle the document transformation messages and have an OpenAI or Gemini connector whose selected model accepts the supplied documents and structured output schema. Available AI does not guarantee those request-specific capabilities; ordinary errors and timeouts still apply after a request is sent.

Request

MSGCtrlLLMDocumentTransformReq has these fields in wire order:

The combined encoded documents must not exceed 8 MiB. This operation accepts LMB1 values with application/pdf, image/png, image/jpeg, or image/webp MIME types. The MIME type must agree with the file signature. The LMB1 value contains the media bytes; the client does not send a path or file URL. The Hub allows up to four pending transformations and limits each provider response body to 2 MiB.

The task instruction says what to extract or transform. The validation JSON supplies values that must be checked against the documents. The schema says which JSON shape the result must have. These are separate fields; clients do not interpolate placeholder tags into the instruction.

For example, a client can use this instruction:

Example

Read the attached employment record. Extract the employee name exactly as shown. Do not guess missing or illegible values.

It can send {"year":"2026"} as the validation JSON, and this output schema:

{
  "type": "object",
  "properties": {
    "employee_name": { "type": "string" }
  },
  "required": ["employee_name"],
  "additionalProperties": false
}

Validation facts need not be returned as result fields. In this example, the Hub asks the model to verify year from the document and returns only the employee name when that check succeeds.

_validationJson and _outputSchema are serialized strings in the Control message. The examples above show the JSON values before string serialization. The client must preserve the 64-bit transaction ID without rounding it through a floating-point number.

The Hub requires _validationJson and _outputSchema to parse as JSON objects. It supplies the schema through the configured provider's structured-output mode: OpenAI uses json_schema with strict: true, and Gemini uses a schema-constrained JSON response format. OpenAI requires every object property in required and additionalProperties: false; represent a genuinely Nullable property with a required type: [T, "null"]. A schema outside a provider's supported subset can cause a provider error. The Hub does not interpret JSON Schema keywords or enforce the schema against the returned result. The client must check a successful result against its supplied schema before using it.

Nullable must reflect the form's actual data model. App authors must not make a destination Nullable solely to accommodate document recognition.

Response and validation

MSGCtrlLLMDocumentTransformRsp contains _transactionId, _json, and _error. On model-reported success, _error is empty and _json is its serialized JSON result. On failure, _json is empty and _error explains the failure. There is no partial result. A malformed request, unsupported media type, provider failure, or malformed model response envelope also produces an error response when the Control session remains connected.

The Hub instructs the model to use the documents as evidence, to check every validation value, and to report a conflict or unverifiable value as an error. The Hub forwards the model's JSON result or error without independently checking the result fields, output schema, or validation facts. Provider schema mode constrains the supported structure; it does not establish that the model read the document correctly. The client checks the result shape and applies a valid result to the editable fields; the user can inspect or edit the filled Request before submitting it.

The request does not read or change the client's text conversation. Document transformations can be submitted concurrently and are correlated by the transaction ID within the originating Control session.

App Schema authoring

An App declares one document action on a named Table struct with LLMDocTrans. Such a struct is a form: mark its direct result children with LLMDocField. A Table whose variants are all LLMDocTrans is a classifier instead, which sorts a pile of documents into the forms it encloses. Each occurrence of that struct in an editable Request or SubscriptionRequest gets its own AI Import action. A screen can therefore contain several independent forms, including repeated rows of one form type. Selected files are temporary editor input; they do not require a Bytes field in the App Request. When a form's row is promoted or suppressed, its action follows the row action placement rule. The action is unavailable in internal calculation requests and inside Hidden, Fixed, CacheIdentity, or ReadOnly subtrees. ReadOnly on a List restricts its membership, not editing an existing item. Studio checks the validation path for every editable occurrence of the annotated Table; a missing or non-Table target in any such occurrence is a schema error. Calculation-only and protected occurrences do not need an otherwise unused validation fact.

LLMDocField has optional allowEmpty or blank, described under blank values. The allowEmpty default is true for a non-Nullable Number and false for String, Boolean, and Enumeration. A marker on a direct child of the annotated Table selects that result field. Within each selected Table, including a List item Table, marking one or more direct children selects only those children for that Table's result object. If none of that Table's direct children is marked, all its unprotected children are selected. Mark each intermediate composite whose descendants are selected. A nested scalar marker also sets its blank rule.

A selected Union imports its variant choice, and a variant's payload fields only when they are marked. The all-children default above does not apply to Union payloads: a variant's payload usually holds facts the chosen option leads to, not values printed on the document. Marking one or more direct variants of a selected Union limits its result schema to those variants; with no marked variant, all unprotected variants remain available, subject to ordinary choice constraints. Marked variants must be capable of producing a result.

A variant's payload is either a Table or a single non-Table value. A Table payload has the same meaning however it is declared (a Rust struct variant V { a, b }, a Rust W2(W2), or a TypeScript interface member): its fields import only when marked, as above. A single non-Table value, such as a Rust tuple variant Allocated(i64), has no field to mark; that value is the variant's one payload field, named by the variant, and it is always marked. A Nil (unit) variant has no payload field.

• When at least one variant has a marked payload field, the result is a variant/value wrapper. value contains that variant's marked payload fields, following the Table selection rule inside them, and is an empty object for a variant with none marked.
• When no variant has a marked payload field, the Union is variant-only and is projected and converted exactly as an Enumeration of its available variant names, so a printed code can be imported without the model answering the facts that code leads to.

When the chosen variant equals the current one, the client keeps the variant and every payload value the result does not supply, in its current state: an unanswered field stays unanswered and is validated as it was before the import, so it does not reject the proposal. A different variant is applied exactly as if the user had chosen it in the editor: its unmarked payload fields receive normal construction and initialization, and required ones without a value stay unanswered. The proposal is not rejected for them. The result is the same state a manual choice of that variant produces: incremental validation runs as it does after that choice, and every rule a manual choice would defer (mandatory values, final-only ValidationRules, and cross-field rules that read an unanswered field) is deferred to Submit's final validation, which requires them as usual. The editor shows the unanswered fields for the user to answer. At least one direct child of a form's annotated root Table must be marked. Normal editor construction supplies the bound struct occurrence before the action runs.

Every supported Libertas value type is supported in result subtrees, including nested Tables and Lists, Enumerations, Unions, Dates, money, dynamic EnumSource, other semantic scalars, and their declared Nullable forms. Hidden, ReadOnly, and Fixed remain ordinary field attributes. Hidden and Fixed fields are excluded from LLM output, including their complete subtrees. Placing LLMDocField directly on either is invalid. A selected composite may contain them: omit those properties from the generated output schema and preserve their normal values when updating the existing record. New List rows or Union payloads receive them through normal field initialization, such as Fixed, Default, or the existing construction rules. If a required value remains unavailable, normal validation rejects the proposal, except for the unmarked payload fields of a newly chosen Union variant, described above. The model never supplies it. ReadOnly results retain their normal visible presentation. For an existing Table or the same Union variant, omitted fields keep their current typed values, including visible fields that were outside the result schema. A newly selected Union variant initializes omitted payload fields by the normal construction rules. An extracted List creates new rows through normal initialization; it must not copy Hidden identities from old rows by position. A composite containing only protected content is omitted, including a List with a protected item type. Marking such a field directly is invalid; no null placeholders are generated.

The client derives standard JSON Schema recursively from the selected result fields and only the types reachable from them: object properties, arrays and items, enum choices, variant alternatives, null, and references for recursive types. It does not send the App's full schema. Declared constraints still apply. This feature adds no arbitrary Table depth, List length, or integer-width restriction; actual schema and wire limits apply, including the 64 KiB output-schema limit. A configured provider that rejects a schema returns a normal request error.

The following TypeScript source illustrates independent W2 and W4 actions:

interface W2Facts { year: string; }

/**
 * @LibertasLLMDocTrans(validation="@.expected", instruction="Read the W2 exactly. Check the supplied facts against the form. Return an empty string for a clearly blank tips box; return decimal numeric text for a legible amount. Report illegible values as errors.", doc_type="application/pdf,image/*")
 */
interface W2 {
    /** @LibertasHidden */
    expected: W2Facts;
    /** @LibertasLLMDocField */
    employee_name: string;
    /** @LibertasNumber(type=f64)
     * @LibertasDefault(0)
     * @LibertasLLMDocField */
    tips: number;
    /** @LibertasSize(max=16)
     * @LibertasLLMDocField */
    boxes: W2Box[];
}

interface W2Box {
    label: string;
    /** @LibertasNumber(type=f64)
     * @LibertasDefault(0) */
    amount: number;
}

/** @LibertasLLMDocTrans(instruction="Read the W4 exactly. Return an empty string for a clearly blank dependents box. Report illegible values as errors.") */
interface W4 {
    /** @LibertasNumber(type=u16)
     * @LibertasDefault(0)
     * @LibertasLLMDocField */
    dependents: number;
}

/** @LibertasRequest */
interface TaxRequest { w2: W2; w4: W4; }

The Rust spellings are #[libertas_llm_doc_trans(instruction = "...", validation = "@.expected", doc_type = "image/*", accept_multiple_inputs = true)] on the struct and #[libertas_llm_doc_field(allow_empty = true)] or #[libertas_llm_doc_field(blank = 0)] on a direct field. Omit the field argument to use its type-dependent default. A marker on a nested scalar inside a selected composite overrides only that scalar's blank rule. Rust and TypeScript emit the same portable Schema attributes.

The selected validation Table may be Hidden and may receive its value through normal Request construction, CopyFrom, or EditRequest. Its children may be complete typed values using the same recursive JSON projection below. An absent or incomplete required value blocks the action. Declared Nullable facts retain their null meaning. Fact names need no relationship to result names: the W2 example verifies year without returning year. The model checks those facts against the document; the Hub does not independently verify them.

Blank values and output types

Every selected result property, including each selected property in a nested Table, is required in the generated strict output JSON Schema; Hidden and Fixed subtrees are omitted. A List is a JSON array whose item schema is derived recursively. Every Table object has exactly its selected children and additionalProperties: false; unselected children do not appear in the model output. The client checks List length against its Schema bounds, converts every item, and replaces the selected List as one typed subtree. It merges selected Table fields into the existing Table or same Union variant as one editor change. A non-Nullable Number normally uses a JSON string result because the form's box might be blank even though the App's data field is required. The model returns decimal numeric text for a legible amount and exact "" for a clearly blank box. The client strictly converts nonempty text to the destination Number type. Exact "" applies the field's effective, typed Schema Default. Thus tips: number with @LibertasDefault(0) stays required and stores zero for a blank tips box. The schema sent to the model does not contain a JSON Schema default keyword or expose the App's Default value.

allowEmpty=true requires a real Schema Default and a non-Nullable scalar whose document representation is a JSON string. An implicit empty initialization or localized DefaultText does not count. A String with default allowEmpty=false requires a nonempty result. Boolean results use JSON Boolean and cannot use allowEmpty=true. Table, List, and Nil fields do not use allowEmpty; a Union may use allowEmpty=true when its Schema Default names one of its variants. Fixed fields are outside the model's result. An Enumeration normally uses a JSON string constrained to its canonical member names. The client maps an exact member name to its declared Fixed integer value, or to its ordinal when no Fixed is declared. The model never supplies a localized display label or an arbitrary integer. Explicit allowEmpty=true also permits exact "" and applies the effective Enumeration Default. If a Number has no meaningful Default, an App author may set allowEmpty=false only when the form establishes that its box cannot be blank; its output then uses JSON integer or number as appropriate and a blank box is an error. Otherwise, leave that field out of automatic extraction until its blank semantics are known. Do not invent a zero Default for this purpose.

An Endpoint's process-provisioning Default does not supply an Endpoint ID, so Endpoint results cannot enable allowEmpty=true.

Nullable remains a domain-model choice, never a document-recognition trick. CodeBots must not make a field Nullable solely to support this action. A truly Nullable result permits JSON null for a genuinely blank value; null clears the destination. A non-null Nullable Table uses its selected child object shape; a non-null Nullable List uses its selected item shape. A Nullable Number uses JSON string or null and converts its non-null numeric text strictly. Nullable Enumeration uses member name or null. allowEmpty is inapplicable to Nullable fields; blank is not.

Nullable fields with no permitted choice

When the current constraints leave a Nullable Union, Enumeration, identity, or EnumSource field with an empty permitted set, its result schema admits exactly null (the model is offered only null) and the import proceeds; any other value is rejected. A non-Nullable field with an empty set refuses the action.

Declared blank value

LLMDocField(blank=VALUE) declares the value that an exact "" result applies when the field's box is blank on the document. VALUE is spelled exactly like a Default literal for that field: $null on a Nullable field, a number literal on a Number, a quoted string, an Enumeration member name, or a Union variant name, with $NAME constant references wherever Default allows them. For example, blank=$null clears a Nullable field when its box is blank, blank=0 stores zero, and blank=ShortTerm on a Union chooses that variant. The portable LLMDocField attribute value is blank: followed by the literal in its portable Default encoding, such as blank:$null, blank:0, or blank:ShortTerm; the other portable values are auto for the bare marker, true, and false.

blank is valid on a selected scalar, Enumeration, or Union field, Nullable or not, and its value must be valid for that field: $null requires Nullable, a member or variant must exist (a Union variant must not be Hidden or Fixed), and a number or text must fit the declared type and bounds. Bytes and identity fields (User, Device, LanDevice, VirtualDevice, Endpoint) accept only blank=$null. blank implies that the field accepts "", so it cannot be combined with allowEmpty; allowEmpty=true remains the shorthand for a blank that applies the field's Schema Default. A field with neither keeps the rules above. Studio rejects an invalid blank at the marker's source location.

A field that accepts "", through allowEmpty=true or blank, adds it to its output schema: exact "" joins an enum (Enumeration, variant-only Union, EnumSource, and permitted choices), a Boolean is JSON Boolean or "", a String drops its nonempty requirement, a Number uses its string form, and a Union with a marked payload field accepts "" in place of its variant/value wrapper. The schema sent to the model never exposes the blank value or the Default. The client applies "" exactly as if the user had typed or chosen that value. On a Union it chooses the blank variant (the Default variant under allowEmpty=true) as a manual choice does: the current variant keeps its payload, and a different variant initializes its payload by the normal construction rules, with required payload fields left unanswered and deferred to Submit as described for a different variant.

For ordinary optional text whose domain meaning is “blank when not supplied,” use a non-Nullable String with an empty Schema Default and explicit allowEmpty=true. In Rust, combine #[libertas_default("")] with #[libertas_llm_doc_field(allow_empty = true)]; the bare field marker still defaults to false for String. Permit length zero in its size bounds. Describe the manual answer as “Leave it empty if absent” and tell the model to return exact "" for genuinely missing text. Reserve JSON null for truly Nullable destinations. Unreadable text is an error, not evidence that the text is absent. Follow the type-selection guidance when manual input and an internal record use different representations.

The client accepts numeric text only when the entire string is a JSON number lexeme, with no surrounding whitespace, grouping separators, currency sign, NaN, or infinity, and checks the destination type and bounds. Conversion failure is an error; it never applies the Default. A whitespace-only String is invalid. The client converts every supported scalar, Table, List item, and declared null into the destination's Libertas type, checks the generated JSON schema, then runs normal typed field and cross-field validation before offering any change. It rejects any partial, extra, mismatched, or unconvertible result.

Complete JSON projection and client conversion

JSON Schema describes the JSON representation. The client converts that representation into the destination's actual Libertas type with its ordinary value codecs and validation rules; JSON primitive types do not replace the App's semantic types.

Every generated schema node carries a description, except a $ref node. It is built from the node's Display Name and Description, the same text the user reads, so the model maps what is printed on the document to the right property. A $ref node carries no sibling keyword, because structured-output providers reject keywords beside $ref (OpenAI does); the referenced definition in $defs carries its type's description instead. A value whose choices are listed by enum (an Enumeration, a variant-only Union, or dynamic EnumSource) also describes every choice in that description: each canonical name followed by its Display Name and Description, for example DirectRollover: G — direct rollover. The node's value-representation hints and its declared answer labels follow, each on its own line; client blank-value guidance comes last.

Physical units retain their ordinary semantics; they do not change the underlying value type. Display Template presentation, including a format pattern, never reaches the model. Hidden and Fixed values never enter the model's output schema or its returned result; normal initialization and validation continue to govern them. Number annotations such as DateOnly, Money, TimeInterval, EnumSource, and Bitflags keep the Number default of allowEmpty=true; their typed Default is applied locally for exact "". Other semantic scalar types default to allowEmpty=false and may explicitly enable it with a meaningful Default.

Dynamic EnumSource is resolved from the bound Request when the request is prepared. Its current choices, allowed-choice constraints, and relevant reference values belong to the pending transaction snapshot. If they change before application, discard the result as stale. A choice cannot be silently reinterpreted against a newer list.

Value-representation hints

AI Import result schemas and AI tool-call descriptions share this one list of value-representation hints. Each hint follows from the node's effective schema attributes, including inherited ones, and its built-in type; App authors never write it. Every applicable hint is one line, in table order, after the node's text and choice list. The words are fixed English; an angle-bracketed part comes from the schema or its document. A hint states the JSON representation only: it never changes storage or encoding, supplies a value, or lists choices beyond those enum already offers.

A Bitflags hint in either form is followed by one line per declared flag, <member> = <value>, with (<Display Name>): <Description> appended when the catalog document lists that member. In the tool-call hint, <one major unit in minor units> is 1 followed by <scale> zeros, such as 100 for USD/2.

Array import

A file often holds several forms of one type: separate forms for several accounts, or a consolidated statement. When the annotated struct occurrence lies anywhere inside an editable List item, directly or through nested Tables and Union payloads, its AI Import action imports an array. The struct type of the occurrence the action is on decides what is imported: only forms of that type.

• The output schema is a JSON object with one required property, `forms`: an array whose every element is that struct's selected result object, as for a single import, with at least one element. The root stays an object because structured-output providers (OpenAI) require one. The client appends to the instruction that the model returns one element per distinct form in the files, that the files may contain duplicate copies of a form which count once, and ignores forms of other types and pages that are not forms.
• The first element replaces the occurrence the action is on, exactly as a single import does.
• Each further element is appended as a new item of the nearest enclosing List. The new item is constructed by normal initialization; every Union on the path from that item down to the struct selects the same variant as in the current item, as a manual choice does, and the element is then applied to the new occurrence as a single import. Other fields of the new item (for example its owner) keep their normal initialization. Existing items are never changed or removed.
• The App's instruction names what a duplicate looks like for its form (for example a tax form's Copy B, Copy C and Copy 2 printed on one page); the client's appended text stays generic.
• Duplicates are dropped: a file may hold several copies of one form, though not always. After conversion the client drops an element to be appended whose imported values exactly equal another element of the same result, or those of an existing item at the same struct position (the same file imported again). The review states how many duplicates were skipped.
• Each element is converted and validated like a single result, and the whole proposal is one editor change: if any element fails, or the appended items would exceed the List's declared size, nothing is applied.
• A struct occurrence outside any List, or inside a List whose membership is ReadOnly, keeps the single import.

Categorized import

A person often holds a pile of different documents at once, such as W-2s, several 1099s and a K-1. One transformation over every form at once is impractical: one instruction cannot carry every form's instructions well, and one result schema with every form's fields exceeds provider schema limits and degrades accuracy. A categorized import first sorts the files into documents of known kinds, then imports each document with its own kind's declaration.

Classifier rule. An LLMDocTrans Table is a classifier if and only if all its variants are LLMDocTrans structs. Its variants are its import positions: each selected field, each selected List's item, and, in an item Table, its one marked form or each variant of its one marked Union (the item's other fields, such as an owner, keep their normal initialization). A variant may itself be a classifier, so classifiers nest, typically one root classifier with one classifier per category. Every other LLMDocTrans Table is a form, with the meaning described above; an LLMDocTrans a form encloses keeps only its own action. Selection is the ordinary rule above (the marked direct children, or all unprotected children when none is marked); a form's requirement to mark at least one direct child does not apply to a classifier.

/// Documents you may have
#[libertas_llm_doc_trans(instruction = "Sort each document into its kind. ...")]
pub struct DocumentCenter {
    /// Wages
    pub wages: WagesCategory,
    /// Interest and dividends
    pub interest_dividends: InterestDividendsCategory,
}

/// Wages
#[libertas_llm_doc_trans(instruction = "Wage statements from an employer. ...")]
pub struct WagesCategory {
    pub items: Vec<WageItem>,
}

pub struct WageItem {
    /// Owner
    pub owner: Owner,               // normal initialization
    /// Document
    #[libertas_llm_doc_field]
    pub document: WageDocument,
}

pub enum WageDocument {
    /// Form W-2
    /// Wage and Tax Statement from an employer.
    W2(W2),                         // W2 is an LLMDocTrans form
    /// Form 8919
    /// Uncollected Social Security and Medicare tax on wages.
    Form8919(Form8919),             // Form8919 is an LLMDocTrans form
}

An editor usually draws a classifier with many categories in the categorized layout: only the categories that hold a document, plus one control that adds any kind.

This schematic Rust omits derives and the form structs. The root classifier is a Table, not a Union, because a Union holds one variant at a time while the editor holds every category at once.

Arguments. On a classifier, instruction (required) says what counts as one document, what to leave out, and how to recognize duplicate copies; the client appends the generic classification text below. doc_type filters the selectable files as for a form. A classifier always accepts several files; a validation path or accept_multiple_inputs = true is invalid on it (the emitted defaults are accepted). Each form keeps its own validation, used when its kind is imported; its doc_type and accept_multiple_inputs govern only its own action.

Assembled instruction. A classification request sends one instruction: the classifier's own instruction, then, in declaration order, each enclosed classifier's native name and instruction, nested the same way down to the categories. Native names keep the assembled bytes independent of locale; the localized display names name the kinds (below). A root classifier's rules for a category therefore reach the model when the root sorts a pile. Leaf forms' instructions are not included; they belong to the import step. The assembled instruction is decided by the schema alone, and the platform generates it; an App author does not budget it.

Kinds. A classifier's kinds are all the forms under it: every non-classifier LLMDocTrans reached through its variants and through every nested classifier. Traversal stops at a form, so declarations a form encloses are not kinds. A kind is offered only while the Union's ordinary choice constraints (Hidden, Fixed, ConstrainedBy) permit it for a new item at its position. A kind inside a List is a list kind, and the nearest List enclosing it is its category List. A form that is a classifier's variant itself, outside any List, is a single-document kind, such as a passport or a birth certificate: the Request holds one occurrence of it. A kind is named for the model by its variant's (or, for a one-kind List, its struct's) localized display name and description, with the display names of the classifiers between the importing classifier and the kind. Write them to say exactly what the document is, such as "Form 1099-S for the sale of your main home". The root classifier offers every leaf kind; a category classifier offers its own.

Item shapes. A List under a classifier holds one of:

• a Union, whose variants with a form payload are kinds;
• a Table with exactly one direct child marked LLMDocField that is a Union (read the same way) or a form (a one-kind List); its other fields, such as an owner, keep their normal initialization; or
• a form struct, making a one-kind List.

Studio validation. The shared validator decides the classifier rule from the emitted schema. A classifier must be a named Table used in an editable Request or SubscriptionRequest, with at least one kind, without a validation path or accept_multiple_inputs = true, and with every enclosed List of an allowed item shape. Studio refuses a classifier List whose membership is ReadOnly, and warns when a single-document kind itself encloses import declarations (it is imported as one form; a record or Nil variant in a category Union causes this). Each failure is reported at the declaration's source location.

Action. Every classifier occurrence offers an Import with AI action (in Chinese it shares the AI Import label), placed as the presentation contract describes. Its availability is AI Import's: absent when AI is unavailable under the Control AI availability rule, disabled while any calculation is pending or deferred, and unavailable in internal calculation requests and inside Hidden, Fixed, CacheIdentity, or ReadOnly subtrees. Each category List keeps its ordinary Add: a one-kind List appends its struct, and a multi-kind List opens its variant chooser.

Workflow.

1. Select. The person takes pictures or opens files, several at once, and can add more before starting. Images are prepared as described under client workflow; PDF bytes are unchanged.

2. Classify. The client sends one transformation per batch of at most eight files and 8 MiB, the request limits above. Its output schema is:

   JSON

   {
     "type": "object",
     "properties": {
       "documents": { "type": "array", "items": {
         "type": "object",
         "properties": {
           "kind": { "type": "string", "enum": ["…offered kinds…"] },
           "parts": { "type": "array", "items": {
             "type": "object",
             "properties": {
               "file": { "type": "integer" },
               "pages": { "type": "array", "items": { "type": "integer" } }
             },
             "required": ["file", "pages"], "additionalProperties": false } }
         },
         "required": ["kind", "parts"], "additionalProperties": false } },
       "not_imported": { "type": "array", "items": {
         "type": "object",
         "properties": {
           "file": { "type": "integer" },
           "pages": { "type": "array", "items": { "type": "integer" } }
         },
         "required": ["file", "pages"], "additionalProperties": false } }
     },
     "required": ["documents", "not_imported"], "additionalProperties": false
   }
   

kind is an enumeration with one unique name per offered kind; its description lists every kind with the name text described under Kinds, as an Enumeration's choices are described. file is the one-based position of a file in that request, and pages its one-based pages; an image has one page. The appended generic text says that the result has one element per distinct document, that duplicate copies of a document count once, and that pages belonging to no offered kind are listed in not_imported instead. The import itself does not open or count the files: it forwards them as selected and relies on the model's page numbers; a document whose pages do not exist fails its import and becomes Enter by hand, and so does a document whose part names a file the request does not contain or no pages; an unusable not_imported entry is listed as given. One bad entry never ends the import. Only the transport of a client provider may convert a file for a provider that cannot take it, such as a PDF sent as one PNG per page; that conversion never decides the import's pages. 3. Import. For each classified document, the client sends that kind's own transformation: the kind's instruction, its array import output schema, and its validation facts resolved in a new item of its category List constructed by normal initialization. The request carries the files the document's pages lie in, and the instruction names those pages, for example "Read only pages 3 to 4 of file 2". Clients never split or rewrite a PDF. Classification and import requests run concurrently, with at most four of an import's transformations pending at once, the Hub's limit. 4. Progress. The blocking busy overlay with Cancel of the client workflow shows determinate text, such as "Sorting 12 files…" and then "Reading 5 of 9 documents…". 5. Review. Before anything changes the editor, one sheet lists one row per document, grouped by category:

• Ready: the kind and a short summary of its imported values, such as "W-2 · Acme Inc.";
• Enter by hand: classified, but its import failed (unreadable pages, a provider error or timeout, a result that fails conversion or validation); a list kind is added with its kind selected and its payload in normal initialization; a single-document kind's occurrence stays unchanged;
• Replaces your current entry: a single-document kind whose occurrence is no longer in normal initialization; its checkbox starts unchecked;
• Not imported: each file and its pages that the classification lists in not_imported, and the count of documents dropped as duplicates.

Each Ready and Enter-by-hand row has a checkbox, checked by default, so a misclassified document can be left out. Add selected adds the checked documents; Cancel discards them all. 6. Apply. Applying is one editor change. A checked single-document kind is applied to its occurrence as a single import; a second document of the same single-document kind is listed as Not imported (only one is allowed). Selection order decides: documents are ordered by the position of their first file and first page in the person's selection, never by when a reply arrives, so "second" is the later one in that order, and the review's rows and the appended items follow the same order. Each checked list-kind document is always appended as a new item of its category List (the import is started from a classifier, outside any document, so there is no current item to replace), its variant selected as a manual choice does, and its result applied under the array import rules: the first form fills the new item, further forms of that kind are appended with the same variant, and duplicates of another imported form or of an existing item at the same position are dropped. Existing items are never changed; a filled single-document occurrence changes only when its Replaces your current entry row is checked. If any List's declared size would be exceeded, nothing is applied and the sheet names that List. 7. After. The Request is not submitted; the person reviews and saves as usual.

The categorized import is one blocking action under the pending and stale rules, through its review: Cancel, navigation, or a disconnect discards every pending reply and leaves the editor unchanged. An error or timeout of any classification request ends the import, is reported, and leaves the editor unchanged; the person can retry. An import step's failure makes only that document Enter by hand.

Without AI, every step above is absent; Add and each item's editor work as usual.

Client workflow

The model request itself, including a client's own provider account, follows LLM requests.

The AI Import action (Chinese: AI导入) first offers Take a picture using ordinary camera capture when the effective MIME filter accepts camera JPEG and a camera is available, and Open a file. Each action selects one file by default, or up to eight in user-selected order when enabled. Captured photos use the same preparation and validation as selected image files. Clients decode and orient selected images, flatten transparency on white, and JPEG-encode them at no more than 3,000,000 content bytes each. Image preparation must keep reducing an oversized image's dimensions until its JPEG fits. JPEG quality stays at or above 70, and the source is never upscaled. An image that cannot be decoded or encoded reports an input error. PDF bytes remain unchanged. Clients check the combined LMB1 limit of 8 MiB before sending.

The client uses a session-unique unsigned 64-bit transaction ID. A pending action belongs to its originating Control session, editor, struct occurrence, selected files, validation value, and result-field state. A cancel, timeout, navigation, or disconnect, or a change to a relevant value while it is pending (for example an arriving calculation result), makes its reply stale. Only the matching pending reply can change the editor.

Like Submit, AI Import is disabled while any calculation is pending or deferred and becomes available when the calculations settle, so no calculation result can arrive during a transformation and make its reply stale. AI Import is blocking. While its transformation is pending, the client shows a busy overlay over the whole editor with a Cancel action; the editor and its actions accept no input, so no edit can make the pending reply stale and no second transformation or Submit can start. The busy state ends, and the editor accepts input again, when the matching reply arrives (a result is applied as above; an error is reported), when the user cancels, when the timeout elapses, or when the Control session disconnects. Cancel, timeout, and disconnect discard the pending action, so a late reply is ignored; an error, a timeout, or a disconnect is reported to the user, who can retry. Navigating away from the editor also discards it. On success, the client checks the JSON shape, strictly converts each value, validates the complete candidate, and applies all selected fields in one normal editor change, including values supplied by a Schema Default. Ordinary drafts and calculations remain active. AI Import does not submit the Request. An error or invalid result leaves the editor unchanged. The user can review and correct the filled fields before submitting the Request.