# LLM document transformation LLM document transformation is also called **AI Import**. A Control client can ask the Hub's configured LLM connector to transform one or more documents into a JSON value. Each request has one response sent to the Control session that submitted it. The response repeats the caller's unsigned 64-bit transaction ID, so a client can match concurrent requests without using the text conversation history. Clients offer the action on modern Hubs and hide it on legacy Hubs. The [Control AI availability rule](protocols.md#ai-navigation-and-ai-search) applies before choosing documents or transmitting a transformation: an unavailable action reports an immediate error. The Hub must handle the document transformation messages and have an OpenAI or Gemini connector whose selected model accepts the supplied documents and structured output schema. Available AI does not guarantee those request-specific capabilities; ordinary errors and timeouts still apply after a request is sent. ## Request `MSGCtrlLLMDocumentTransformReq` has these fields in wire order: | Field | Type | Meaning | | --- | --- | --- | | `_transactionId` | unsigned 64-bit integer | Caller-chosen correlation ID, returned unchanged. | | `_documents` | array of `LibertasLLMDocument` | One to eight documents. Each item has one `_bytes` value containing the full [LMB1 envelope](schema.md#mime-aware-bytes). | | `_instruction` | string | Required, nonempty client task instruction. No separate instruction-size limit. | | `_validationJson` | string containing a JSON object | Facts the documents must confirm. Use `"{}"` when there are no facts. At most 64 KiB. | | `_outputSchema` | string containing a JSON Schema object | Requested result shape for the model and client. At most 64 KiB. | The combined encoded documents must not exceed 8 MiB. This operation accepts LMB1 values with `application/pdf`, `image/png`, `image/jpeg`, or `image/webp` MIME types. The MIME type must agree with the file signature. The LMB1 value contains the media bytes; the client does not send a path or file URL. The Hub allows up to four pending transformations and limits each provider response body to 2 MiB. The task instruction says **what** to extract or transform. The validation JSON supplies values that must be checked against the documents. The schema says **which JSON shape** the result must have. These are separate fields; clients do not interpolate placeholder tags into the instruction. For example, a client can use this instruction: > Read the attached employment record. Extract the employee name exactly as > shown. Do not guess missing or illegible values. It can send `{"year":"2026"}` as the validation JSON, and this output schema: ```json { "type": "object", "properties": { "employee_name": { "type": "string" } }, "required": ["employee_name"], "additionalProperties": false } ``` Validation facts need not be returned as result fields. In this example, the Hub asks the model to verify `year` from the document and returns only the employee name when that check succeeds. `_validationJson` and `_outputSchema` are serialized **strings** in the Control message. The examples above show the JSON values before string serialization. The client must preserve the 64-bit transaction ID without rounding it through a floating-point number. The Hub requires `_validationJson` and `_outputSchema` to parse as JSON objects. It supplies the schema through the configured provider's structured-output mode: OpenAI uses `json_schema` with `strict: true`, and Gemini uses a schema-constrained JSON response format. OpenAI requires every object property in `required` and `additionalProperties: false`; represent a genuinely Nullable property with a required `type: [T, "null"]`. A schema outside a provider's supported subset can cause a provider error. The Hub does not interpret JSON Schema keywords or enforce the schema against the returned result. The client must check a successful result against its supplied schema before using it. Nullable must reflect the form's actual data model. App authors must not make a destination Nullable solely to accommodate document recognition. ## Response and validation `MSGCtrlLLMDocumentTransformRsp` contains `_transactionId`, `_json`, and `_error`. On model-reported success, `_error` is empty and `_json` is its serialized JSON result. On failure, `_json` is empty and `_error` explains the failure. There is no partial result. A malformed request, unsupported media type, provider failure, or malformed model response envelope also produces an error response when the Control session remains connected. The Hub instructs the model to use the documents as evidence, to check every validation value, and to report a conflict or unverifiable value as an error. The Hub forwards the model's JSON result or error without independently checking the result fields, output schema, or validation facts. Provider schema mode constrains the supported structure; it does not establish that the model read the document correctly. The client checks the result shape and applies a valid result to the editable fields; the user can inspect or edit the filled Request before submitting it. The request does not read or change the client's text conversation. Document transformations can be submitted concurrently and are correlated by the transaction ID within the originating Control session. ## App Schema authoring An App declares one document action on a named Table struct with `LLMDocTrans`. Such a struct is a **form**: mark its direct result children with `LLMDocField`. A Table whose variants are all `LLMDocTrans` is a [classifier](#categorized-import) instead, which sorts a pile of documents into the forms it encloses. Each occurrence of that struct in an editable Request or SubscriptionRequest gets its own **AI Import** action. A screen can therefore contain several independent forms, including repeated rows of one form type. Selected files are temporary editor input; they do not require a Bytes field in the App Request. When a form's row is promoted or suppressed, its action follows the [row action placement rule](presentation.md#actions-on-a-promoted-or-suppressed-row). The action is unavailable in internal calculation requests and inside Hidden, Fixed, CacheIdentity, or ReadOnly subtrees. ReadOnly on a List restricts its membership, not editing an existing item. Studio checks the `validation` path for every editable occurrence of the annotated Table; a missing or non-Table target in any such occurrence is a schema error. Calculation-only and protected occurrences do not need an otherwise unused validation fact. | Argument | Meaning | | --- | --- | | `instruction` | Required App-authored task text. Tell the model what to read, what a genuinely blank box means, and when to report an error. The Hub sends it separately from facts and result schema, with no separate instruction-size limit. | | `validation` | Optional [Libertas Schema Path](schema.md#context-and-syntax) to a Table value in the current Request. `$` is the Request root, `@` is this struct occurrence, and `^` follows actual value-tree parents. The Table's **children**, without its own name, become the validation JSON object. Omission sends `{}`. | | `doc_type` | Optional [MIME filter](schema.md#mime-aware-bytes) for selected files. The effective source choices are its intersection with PDF, PNG, JPEG, and WebP; an empty intersection is invalid. Omission accepts all four. Prepared images are sent as JPEG even if selected as PNG or WebP. | | `accept_multiple_inputs` | Optional Boolean, default `false`. One file is the normal case. Set `true` only when a logical document may span several selected files; the Hub limit is eight. | `LLMDocField` has optional `allowEmpty` or `blank`, described under [blank values](#blank-values-and-output-types). The `allowEmpty` default is `true` for a non-Nullable Number and `false` for String, Boolean, and Enumeration. A marker on a **direct child** of the annotated Table selects that result field. Within each selected Table, including a List item Table, marking one or more direct children selects **only those children** for that Table's result object. If none of that Table's direct children is marked, all its unprotected children are selected. Mark each intermediate composite whose descendants are selected. A nested scalar marker also sets its blank rule. A selected Union imports its variant choice, and a variant's payload fields **only when they are marked**. The all-children default above does not apply to Union payloads: a variant's payload usually holds facts the chosen option leads to, not values printed on the document. Marking one or more direct variants of a selected Union limits its result schema to those variants; with no marked variant, all unprotected variants remain available, subject to ordinary choice constraints. Marked variants must be capable of producing a result. A variant's payload is either a Table or a single non-Table value. A Table payload has the same meaning however it is declared (a Rust struct variant `V { a, b }`, a Rust `W2(W2)`, or a TypeScript interface member): its fields import only when marked, as above. A single non-Table value, such as a Rust tuple variant `Allocated(i64)`, has no field to mark; that value is the variant's one payload field, named by the variant, and it is **always marked**. A Nil (unit) variant has no payload field. - When at least one variant has a marked payload field, the result is a `variant`/`value` wrapper. `value` contains that variant's marked payload fields, following the Table selection rule inside them, and is an empty object for a variant with none marked. - When no variant has a marked payload field, the Union is **variant-only** and is projected and converted exactly as an **Enumeration** of its available variant names, so a printed code can be imported without the model answering the facts that code leads to. When the chosen variant equals the current one, the client keeps the variant and every payload value the result does not supply, in its current state: an unanswered field stays unanswered and is validated as it was before the import, so it does not reject the proposal. A different variant is applied exactly as if the user had chosen it in the editor: its unmarked payload fields receive normal construction and initialization, and required ones without a value stay unanswered. The proposal is not rejected for them. The result is the same state a manual choice of that variant produces: incremental validation runs as it does after that choice, and every rule a manual choice would defer (mandatory values, final-only ValidationRules, and cross-field rules that read an unanswered field) is deferred to Submit's final validation, which requires them as usual. The editor shows the unanswered fields for the user to answer. At least one direct child of a form's annotated root Table must be marked. Normal editor construction supplies the bound struct occurrence before the action runs. **Every supported Libertas value type is supported in result subtrees**, including nested Tables and Lists, Enumerations, Unions, Dates, money, dynamic EnumSource, other semantic scalars, and their declared Nullable forms. Hidden, ReadOnly, and Fixed remain ordinary field attributes. **Hidden and Fixed fields are excluded from LLM output**, including their complete subtrees. Placing `LLMDocField` directly on either is invalid. A selected composite may contain them: omit those properties from the generated output schema and preserve their normal values when updating the existing record. New List rows or Union payloads receive them through normal field initialization, such as Fixed, Default, or the existing construction rules. If a required value remains unavailable, normal validation rejects the proposal, except for the unmarked payload fields of a newly chosen Union variant, described above. The model never supplies it. ReadOnly results retain their normal visible presentation. For an existing Table or the same Union variant, omitted fields keep their current typed values, including visible fields that were outside the result schema. A newly selected Union variant initializes omitted payload fields by the normal construction rules. An extracted List creates new rows through normal initialization; it must not copy Hidden identities from old rows by position. A composite containing only protected content is omitted, including a List with a protected item type. Marking such a field directly is invalid; no null placeholders are generated. The client derives standard JSON Schema recursively from the selected result fields and only the types reachable from them: object properties, arrays and items, enum choices, variant alternatives, null, and references for recursive types. It does not send the App's full schema. Declared constraints still apply. This feature adds no arbitrary Table depth, List length, or integer-width restriction; actual schema and wire limits apply, including the 64 KiB output-schema limit. A configured provider that rejects a schema returns a normal request error. The following TypeScript source illustrates independent W2 and W4 actions: ```typescript interface W2Facts { year: string; } /** * @LibertasLLMDocTrans(validation="@.expected", instruction="Read the W2 exactly. Check the supplied facts against the form. Return an empty string for a clearly blank tips box; return decimal numeric text for a legible amount. Report illegible values as errors.", doc_type="application/pdf,image/*") */ interface W2 { /** @LibertasHidden */ expected: W2Facts; /** @LibertasLLMDocField */ employee_name: string; /** @LibertasNumber(type=f64) * @LibertasDefault(0) * @LibertasLLMDocField */ tips: number; /** @LibertasSize(max=16) * @LibertasLLMDocField */ boxes: W2Box[]; } interface W2Box { label: string; /** @LibertasNumber(type=f64) * @LibertasDefault(0) */ amount: number; } /** @LibertasLLMDocTrans(instruction="Read the W4 exactly. Return an empty string for a clearly blank dependents box. Report illegible values as errors.") */ interface W4 { /** @LibertasNumber(type=u16) * @LibertasDefault(0) * @LibertasLLMDocField */ dependents: number; } /** @LibertasRequest */ interface TaxRequest { w2: W2; w4: W4; } ``` The Rust spellings are `#[libertas_llm_doc_trans(instruction = "...", validation = "@.expected", doc_type = "image/*", accept_multiple_inputs = true)]` on the struct and `#[libertas_llm_doc_field(allow_empty = true)]` or `#[libertas_llm_doc_field(blank = 0)]` on a direct field. Omit the field argument to use its type-dependent default. A marker on a nested scalar inside a selected composite overrides only that scalar's blank rule. Rust and TypeScript emit the same portable Schema attributes. The selected validation Table may be Hidden and may receive its value through normal Request construction, CopyFrom, or EditRequest. Its children may be complete typed values using the same recursive JSON projection below. An absent or incomplete required value blocks the action. Declared Nullable facts retain their null meaning. Fact names need no relationship to result names: the W2 example verifies `year` without returning `year`. The model checks those facts against the document; the Hub does not independently verify them. ### Blank values and output types Every selected result property, including each selected property in a nested Table, is required in the generated strict output JSON Schema; Hidden and Fixed subtrees are omitted. A List is a JSON array whose item schema is derived recursively. Every Table object has exactly its selected children and `additionalProperties: false`; unselected children do not appear in the model output. The client checks List length against its Schema bounds, converts every item, and replaces the selected List as one typed subtree. It merges selected Table fields into the existing Table or same Union variant as one editor change. A non-Nullable Number normally uses a **JSON string** result because the form's box might be blank even though the App's data field is required. The model returns decimal numeric text for a legible amount and exact `""` for a clearly blank box. The client strictly converts nonempty text to the destination Number type. Exact `""` applies the field's effective, typed Schema `Default`. Thus `tips: number` with `@LibertasDefault(0)` stays required and stores zero for a blank tips box. The schema sent to the model does not contain a JSON Schema `default` keyword or expose the App's Default value. `allowEmpty=true` requires a real Schema `Default` and a non-Nullable scalar whose document representation is a JSON string. An implicit empty initialization or localized DefaultText does not count. A String with default `allowEmpty=false` requires a nonempty result. Boolean results use JSON Boolean and cannot use `allowEmpty=true`. Table, List, and Nil fields do not use `allowEmpty`; a Union may use `allowEmpty=true` when its Schema `Default` names one of its variants. Fixed fields are outside the model's result. An Enumeration normally uses a JSON string constrained to its canonical member names. The client maps an exact member name to its declared `Fixed` integer value, or to its ordinal when no `Fixed` is declared. The model never supplies a localized display label or an arbitrary integer. Explicit `allowEmpty=true` also permits exact `""` and applies the effective Enumeration Default. If a Number has no meaningful Default, an App author may set `allowEmpty=false` only when the form establishes that its box cannot be blank; its output then uses JSON `integer` or `number` as appropriate and a blank box is an error. Otherwise, leave that field out of automatic extraction until its blank semantics are known. Do not invent a zero Default for this purpose. An Endpoint's process-provisioning `Default` does not supply an Endpoint ID, so Endpoint results cannot enable `allowEmpty=true`. Nullable remains a domain-model choice, never a document-recognition trick. CodeBots must not make a field Nullable solely to support this action. A truly Nullable result permits JSON `null` for a genuinely blank value; `null` clears the destination. A non-null Nullable Table uses its selected child object shape; a non-null Nullable List uses its selected item shape. A Nullable Number uses JSON string or null and converts its non-null numeric text strictly. Nullable Enumeration uses member name or null. `allowEmpty` is inapplicable to Nullable fields; `blank` is not. ##### Nullable fields with no permitted choice When the current constraints leave a Nullable Union, Enumeration, identity, or EnumSource field with an empty permitted set, its result schema admits exactly `null` (the model is offered only `null`) and the import proceeds; any other value is rejected. A non-Nullable field with an empty set refuses the action. #### Declared blank value `LLMDocField(blank=VALUE)` declares the value that an exact `""` result applies when the field's box is blank on the document. `VALUE` is spelled exactly like a `Default` literal for that field: `$null` on a Nullable field, a number literal on a Number, a quoted string, an Enumeration member name, or a Union variant name, with `$NAME` constant references wherever Default allows them. For example, `blank=$null` clears a Nullable field when its box is blank, `blank=0` stores zero, and `blank=ShortTerm` on a Union chooses that variant. The portable `LLMDocField` attribute value is `blank:` followed by the literal in its portable Default encoding, such as `blank:$null`, `blank:0`, or `blank:ShortTerm`; the other portable values are `auto` for the bare marker, `true`, and `false`. `blank` is valid on a selected scalar, Enumeration, or Union field, Nullable or not, and its value must be valid for that field: `$null` requires Nullable, a member or variant must exist (a Union variant must not be Hidden or Fixed), and a number or text must fit the declared type and bounds. Bytes and identity fields (User, Device, LanDevice, VirtualDevice, Endpoint) accept only `blank=$null`. `blank` implies that the field accepts `""`, so it cannot be combined with `allowEmpty`; `allowEmpty=true` remains the shorthand for a blank that applies the field's Schema `Default`. A field with neither keeps the rules above. Studio rejects an invalid `blank` at the marker's source location. A field that accepts `""`, through `allowEmpty=true` or `blank`, adds it to its output schema: exact `""` joins an `enum` (Enumeration, variant-only Union, EnumSource, and permitted choices), a Boolean is JSON Boolean or `""`, a String drops its nonempty requirement, a Number uses its string form, and a Union with a marked payload field accepts `""` in place of its `variant`/`value` wrapper. The schema sent to the model never exposes the blank value or the Default. The client applies `""` exactly as if the user had typed or chosen that value. On a Union it chooses the blank variant (the Default variant under `allowEmpty=true`) as a manual choice does: the current variant keeps its payload, and a different variant initializes its payload by the normal construction rules, with required payload fields left unanswered and deferred to Submit as described for a [different variant](#app-schema-authoring). For ordinary optional text whose domain meaning is “blank when not supplied,” use a non-Nullable String with an empty Schema Default and explicit `allowEmpty=true`. In Rust, combine `#[libertas_default("")]` with `#[libertas_llm_doc_field(allow_empty = true)]`; the bare field marker still defaults to `false` for String. Permit length zero in its size bounds. Describe the manual answer as “Leave it empty if absent” and tell the model to return exact `""` for genuinely missing text. Reserve JSON `null` for truly Nullable destinations. Unreadable text is an error, not evidence that the text is absent. Follow the [type-selection guidance](app-contract.md#user-facing-and-non-user-facing-types) when manual input and an internal record use different representations. The client accepts numeric text only when the entire string is a JSON number lexeme, with no surrounding whitespace, grouping separators, currency sign, `NaN`, or infinity, and checks the destination type and bounds. Conversion failure is an error; it never applies the Default. A whitespace-only String is invalid. The client converts every supported scalar, Table, List item, and declared null into the destination's Libertas type, checks the generated JSON schema, then runs normal typed field and cross-field validation before offering any change. It rejects any partial, extra, mismatched, or unconvertible result. ### Complete JSON projection and client conversion JSON Schema describes the JSON representation. The client converts that representation into the destination's actual Libertas type with its ordinary value codecs and validation rules; JSON primitive types do not replace the App's semantic types. **Every generated schema node carries a `description`**, except a `$ref` node. It is built from the node's Display Name and Description, the same text the user reads, so the model maps what is printed on the document to the right property. A `$ref` node carries no sibling keyword, because structured-output providers reject keywords beside `$ref` (OpenAI does); the referenced definition in `$defs` carries its type's `description` instead. A value whose choices are listed by `enum` (an Enumeration, a variant-only Union, or dynamic EnumSource) also describes every choice in that `description`: each canonical name followed by its Display Name and Description, for example `DirectRollover: G — direct rollover`. The node's [value-representation hints](#value-representation-hints) and its declared [answer labels](documentation.md#answer-labels-in-ai-descriptions) follow, each on its own line; client blank-value guidance comes last. | Libertas value | Model JSON and client conversion | | --- | --- | | String | JSON string; preserve text, applying the declared blank rule. | | Boolean | JSON Boolean; no string truthiness or numeric coercion. | | Number | Decimal numeric string when blank conversion or exact wide-integer representation requires it; otherwise JSON integer/number. Parse the complete value, preserve integer precision, and enforce native width, range, and step. | | DateOnly | `YYYY-MM-DD` string; validate a Gregorian date and convert to the ordinary `yyyyMMdd` typed value. | | DateTime | RFC 3339 string with an explicit offset; convert to the ordinary epoch-second instant without guessing a timezone. | | TimeOnly | `HH:mm:ss` string; convert to seconds from local midnight under the normal TimeOnly rules. | | Money | Decimal major-unit string in the declared currency; convert exactly to integer minor units using the declared scale. Reject excess fractional precision and overflow. | | TimeInterval | Decimal seconds string; convert to the declared numeric duration type and bounds. | | Enumeration | Canonical member-name string constrained by `enum`, with every member's Display Name and Description in the `description`; map to its declared Fixed value or ordinal. | | Dynamic EnumSource | Canonical decimal index string constrained by `enum` to the currently permitted indices; for a List whose member declares a [key](schema.md#enumsource-keys), the member's key as a string (decimal for an integer key) constrained to the current keys instead. Include the corresponding current choice values in schema descriptions. Resolve the returned index or key through that same choice snapshot and normal constraint rules, and store the index or the typed key. A [List host](schema.md#enumsource-list-hosts) is a JSON array of these strings; a repeated element rejects the proposal, and `Size` applies. | | Bitflags | Decimal mask string with the declared catalog described in the schema; convert exactly and reject undeclared bits. | | User, Device, LanDevice, VirtualDevice, Endpoint | Decimal identity string; convert without precision loss and apply the ordinary identity/selector constraints. Include the current permitted candidates and their meanings when the editor provides a choice set. | | Bytes | Base64 string; decode to Bytes and enforce ordinary byte constraints, including LMB1 or formatted-value rules when annotated. | | Table | JSON object with every selected child required and no extra properties; recursively convert children while preserving omitted values in an existing Table or unchanged Union variant. | | List | JSON array; recursively convert every element and enforce its declared size and element constraints. | | Union with a marked payload field | An `anyOf` of variant objects, each with required `variant` and `value` properties and no extras. `variant` is that branch's canonical name; `value` holds that variant's marked payload fields (an empty object when it has none). Convert through the ordinary Union discriminator and payload codec. | | Union with no marked payload field (variant-only) | As an Enumeration of its available variant names. Keep the current variant and payload when it is unchanged; otherwise select it as the editor does, with normal payload initialization. | | Nil | JSON null. | | Nullable | Add null to that value's JSON schema; a returned null clears the typed value. Never add Nullable merely for recognition. | Physical units retain their ordinary semantics; they do not change the underlying value type. Display Template presentation, including a `format` pattern, never reaches the model. Hidden and Fixed values never enter the model's output schema or its returned result; normal initialization and validation continue to govern them. Number annotations such as DateOnly, Money, TimeInterval, EnumSource, and Bitflags keep the Number default of `allowEmpty=true`; their typed Default is applied locally for exact `""`. Other semantic scalar types default to `allowEmpty=false` and may explicitly enable it with a meaningful Default. Dynamic EnumSource is resolved from the bound Request when the request is prepared. Its current choices, allowed-choice constraints, and relevant reference values belong to the pending transaction snapshot. If they change before application, discard the result as stale. A choice cannot be silently reinterpreted against a newer list. #### Value-representation hints AI Import result schemas and [AI tool-call descriptions](presentation.md#ai-tool-call-descriptions) share this one list of value-representation hints. Each hint follows from the node's effective schema attributes, including inherited ones, and its built-in type; App authors never write it. Every applicable hint is one line, in table order, after the node's text and choice list. The words are fixed English; an angle-bracketed part comes from the schema or its document. A hint states the JSON representation only: it never changes storage or encoding, supplies a value, or lists choices beyond those `enum` already offers. | Applies to | AI Import hint | Tool-call hint | | --- | --- | --- | | Money Number | `Money: , scale ; decimal amount in major currency units as a string, with '.' as the decimal separator and at most decimal places; no currency symbol or grouping separators.` | `Money: , scale ; integer minor units ( = 1). Use the exact integer, without a currency symbol or decimal amount.` | | DateOnly Number | `DateOnly: calendar date string in YYYY-MM-DD format, e.g. "2026-09-20"; no timezone conversion.` | Same | | TimeInterval Number | `TimeInterval: duration in seconds as a decimal string; e.g. 5 minutes = "300".` | `TimeInterval: numeric duration in seconds; e.g. 5 minutes = 300.` | | Number in its string form with no hint above and no Bitflags or EnumSource | `Number: decimal number as a string, without grouping separators or a currency sign.` | None: the JSON type is `integer` or `number`. | | PhysicalUnit Number | `PhysicalUnit: . The number is expressed in this unit; no automatic unit conversion.`, followed by ` 50 means 50 percent.` for `percent` | Same | | Bitflags Number | `Bitflags: integer mask as a decimal string; combine declared flags using bitwise OR. Undeclared bits are invalid.` | `Bitflags: integer mask; combine declared flags using bitwise OR. Undeclared bits are invalid.` | | DateTime | `DateTime: RFC 3339 date-time string with an explicit UTC offset or Z, e.g. "2026-09-20T14:30:00Z"; identifies an instant.` | Same | | TimeOnly | `TimeOnly: time string in HH:MM:SS format, e.g. "14:30:00"; no date component.` | Same | | EnumSource Number or String | Keyed List: `EnumSource: the chosen item's key from the choices; never its index or label.` Otherwise: `EnumSource: the chosen item's zero-based index from the choices; never its label.` | Keyed List: `EnumSource: the Key field () value of the chosen item in the List at in the current data tree; use that key, not the item's index or label.` Unkeyed List: `EnumSource: zero-based index into the List at in the current data tree; use the item's index, not its label or value.` List not resolved: `EnumSource: the chosen item of the List at in the current data tree; use the item's Key field value if its member Table declares one, otherwise its zero-based index; not its label.` | | EnumSource List host | Keyed List: `EnumSource list: an array of distinct chosen items' keys from the choices; never labels; an empty array chooses none.` Otherwise: `EnumSource list: an array of distinct chosen items' zero-based indexes from the choices; never labels; an empty array chooses none.` | `EnumSource list: an array of distinct chosen items of the List at in the current data tree; each element is ; never labels; an empty array chooses none.`, where `` is `the item's Key field () value`, `the item's zero-based index`, or, when the List is not resolved, `the item's Key field value if its member Table declares one, otherwise its zero-based index` | | Unordered List | `Unordered: List order has no semantic meaning; duplicates are not prohibited by this attribute.` | Same | | Bytes | `Bytes: base64-encoded bytes.` | None: the schema declares `contentEncoding`. | | User, Device, LanDevice, VirtualDevice, Endpoint | `Identity: exact decimal ID as a string.` | None: the schema lists names. | A Bitflags hint in either form is followed by one line per declared flag, ` = `, with ` (): ` appended when the catalog document lists that member. In the tool-call hint, `` is 1 followed by `` zeros, such as `100` for `USD/2`. ### Array import A file often holds several forms of one type: separate forms for several accounts, or a consolidated statement. When the annotated struct occurrence lies anywhere inside an editable List item, directly or through nested Tables and Union payloads, its **AI Import** action imports an array. The struct type of the occurrence the action is on decides what is imported: only forms of that type. - The output schema is a JSON object with one required property, `forms`: an array whose every element is that struct's selected result object, as for a single import, with at least one element. The root stays an object because structured-output providers (OpenAI) require one. The client appends to the instruction that the model returns one element per distinct form in the files, that the files may contain duplicate copies of a form which count once, and ignores forms of other types and pages that are not forms. - The first element replaces the occurrence the action is on, exactly as a single import does. - Each further element is **appended** as a new item of the nearest enclosing List. The new item is constructed by normal initialization; every Union on the path from that item down to the struct selects the same variant as in the current item, as a manual choice does, and the element is then applied to the new occurrence as a single import. Other fields of the new item (for example its owner) keep their normal initialization. Existing items are never changed or removed. - The App's `instruction` names what a duplicate looks like for its form (for example a tax form's Copy B, Copy C and Copy 2 printed on one page); the client's appended text stays generic. - Duplicates are dropped: a file may hold several copies of one form, though not always. After conversion the client drops an element to be appended whose imported values exactly equal another element of the same result, or those of an existing item at the same struct position (the same file imported again). The review states how many duplicates were skipped. - Each element is converted and validated like a single result, and the whole proposal is one editor change: if any element fails, or the appended items would exceed the List's declared size, nothing is applied. - A struct occurrence outside any List, or inside a List whose membership is ReadOnly, keeps the single import. ### Categorized import A person often holds a pile of different documents at once, such as W-2s, several 1099s and a K-1. One transformation over every form at once is impractical: one instruction cannot carry every form's instructions well, and one result schema with every form's fields exceeds provider schema limits and degrades accuracy. A **categorized import** first sorts the files into documents of known kinds, then imports each document with its own kind's declaration. **Classifier rule.** An `LLMDocTrans` Table is a **classifier** if and only if all its variants are `LLMDocTrans` structs. Its variants are its import positions: each selected field, each selected List's item, and, in an item Table, its one marked form or each variant of its one marked Union (the item's other fields, such as an owner, keep their normal initialization). A variant may itself be a classifier, so classifiers nest, typically one root classifier with one classifier per **category**. Every other `LLMDocTrans` Table is a **form**, with the meaning described above; an `LLMDocTrans` a form encloses keeps only its own action. Selection is the ordinary rule above (the marked direct children, or all unprotected children when none is marked); a form's requirement to mark at least one direct child does not apply to a classifier. ```rust /// Documents you may have #[libertas_llm_doc_trans(instruction = "Sort each document into its kind. ...")] pub struct DocumentCenter { /// Wages pub wages: WagesCategory, /// Interest and dividends pub interest_dividends: InterestDividendsCategory, } /// Wages #[libertas_llm_doc_trans(instruction = "Wage statements from an employer. ...")] pub struct WagesCategory { pub items: Vec, } pub struct WageItem { /// Owner pub owner: Owner, // normal initialization /// Document #[libertas_llm_doc_field] pub document: WageDocument, } pub enum WageDocument { /// Form W-2 /// Wage and Tax Statement from an employer. W2(W2), // W2 is an LLMDocTrans form /// Form 8919 /// Uncollected Social Security and Medicare tax on wages. Form8919(Form8919), // Form8919 is an LLMDocTrans form } ``` An editor usually draws a classifier with many categories in the [categorized layout](presentation.md#categorized-layout): only the categories that hold a document, plus one control that adds any kind. This schematic Rust omits derives and the form structs. The root classifier is a [Table, not a Union](#app-schema-authoring), because a Union holds one variant at a time while the editor holds every category at once. **Arguments.** On a classifier, `instruction` (required) says what counts as one document, what to leave out, and how to recognize duplicate copies; the client appends the generic classification text below. `doc_type` filters the selectable files as for a form. A classifier always accepts several files; a `validation` path or `accept_multiple_inputs = true` is invalid on it (the emitted defaults are accepted). Each form keeps its own `validation`, used when its kind is imported; its `doc_type` and `accept_multiple_inputs` govern only its own action. **Assembled instruction.** A classification request sends one instruction: the classifier's own `instruction`, then, in declaration order, each enclosed classifier's native name and `instruction`, nested the same way down to the categories. Native names keep the assembled bytes independent of locale; the localized display names name the kinds (below). A root classifier's rules for a category therefore reach the model when the root sorts a pile. Leaf forms' instructions are not included; they belong to the import step. The assembled instruction is decided by the schema alone, and the platform generates it; an App author does not budget it. **Kinds.** A classifier's kinds are **all the forms under it**: every non-classifier `LLMDocTrans` reached through its variants and through every nested classifier. Traversal stops at a form, so declarations a form encloses are not kinds. A kind is offered only while the Union's ordinary choice constraints (Hidden, Fixed, `ConstrainedBy`) permit it for a new item at its position. A kind inside a List is a **list kind**, and the nearest List enclosing it is its **category List**. A form that is a classifier's variant itself, outside any List, is a **single-document kind**, such as a passport or a birth certificate: the Request holds one occurrence of it. A kind is named for the model by its variant's (or, for a one-kind List, its struct's) localized display name and description, with the display names of the classifiers between the importing classifier and the kind. Write them to say exactly what the document is, such as "Form 1099-S for the sale of your main home". The root classifier offers every leaf kind; a category classifier offers its own. **Item shapes.** A List under a classifier holds one of: - a Union, whose variants with a form payload are kinds; - a Table with exactly one direct child marked `LLMDocField` that is a Union (read the same way) or a form (a one-kind List); its other fields, such as an owner, keep their normal initialization; or - a form struct, making a one-kind List. **Studio validation.** The shared validator decides the classifier rule from the emitted schema. A classifier must be a [named Table](#app-schema-authoring) used in an editable Request or SubscriptionRequest, with at least one kind, without a [`validation` path or `accept_multiple_inputs = true`](#categorized-import), and with every enclosed List of an [allowed item shape](#categorized-import). Studio refuses a classifier List whose membership is ReadOnly, and warns when a single-document kind itself encloses import declarations (it is imported as one form; a record or Nil variant in a category Union causes this). Each failure is reported at the declaration's source location. **Action.** Every classifier occurrence offers an **Import with AI** action (in Chinese it shares the AI Import label), placed as the [presentation contract](presentation.md#actions-on-a-promoted-or-suppressed-row) describes. Its availability is AI Import's: absent when AI is unavailable under the [Control AI availability rule](protocols.md#ai-navigation-and-ai-search), disabled while any calculation is pending or deferred, and unavailable in internal calculation requests and inside Hidden, Fixed, CacheIdentity, or ReadOnly subtrees. Each category List keeps its ordinary **Add**: a one-kind List appends its struct, and a multi-kind List opens its variant chooser. **Workflow.** 1. **Select.** The person takes pictures or opens files, several at once, and can add more before starting. Images are prepared as described under [client workflow](#client-workflow); PDF bytes are unchanged. 2. **Classify.** The client sends one transformation per batch of at most eight files and 8 MiB, the request limits above. Its output schema is: ```json { "type": "object", "properties": { "documents": { "type": "array", "items": { "type": "object", "properties": { "kind": { "type": "string", "enum": ["…offered kinds…"] }, "parts": { "type": "array", "items": { "type": "object", "properties": { "file": { "type": "integer" }, "pages": { "type": "array", "items": { "type": "integer" } } }, "required": ["file", "pages"], "additionalProperties": false } } }, "required": ["kind", "parts"], "additionalProperties": false } }, "not_imported": { "type": "array", "items": { "type": "object", "properties": { "file": { "type": "integer" }, "pages": { "type": "array", "items": { "type": "integer" } } }, "required": ["file", "pages"], "additionalProperties": false } } }, "required": ["documents", "not_imported"], "additionalProperties": false } ``` `kind` is an enumeration with one unique name per offered kind; its `description` lists every kind with the name text described under **Kinds**, as an Enumeration's choices are described. `file` is the one-based position of a file in that request, and `pages` its one-based pages; an image has one page. The appended generic text says that the result has one element per distinct document, that duplicate copies of a document count once, and that pages belonging to no offered kind are listed in `not_imported` instead. The import itself does not open or count the files: it forwards them as selected and relies on the model's page numbers; a document whose pages do not exist fails its import and becomes Enter by hand, and so does a document whose part names a file the request does not contain or no pages; an unusable `not_imported` entry is listed as given. One bad entry never ends the import. Only the transport of a [client provider](llm-client-requests.md#provider-mapping) may convert a file for a provider that cannot take it, such as a PDF sent as one PNG per page; that conversion never decides the import's pages. 3. **Import.** For each classified document, the client sends that kind's own transformation: the kind's instruction, its [array import](#array-import) output schema, and its `validation` facts resolved in a new item of its category List constructed by normal initialization. The request carries the files the document's pages lie in, and the instruction names those pages, for example "Read only pages 3 to 4 of file 2". Clients never split or rewrite a PDF. Classification and import requests run concurrently, with at most four of an import's transformations pending at once, the Hub's limit. 4. **Progress.** The blocking busy overlay with **Cancel** of the [client workflow](#client-workflow) shows determinate text, such as "Sorting 12 files…" and then "Reading 5 of 9 documents…". 5. **Review.** Before anything changes the editor, one sheet lists one row per document, grouped by category: - **Ready**: the kind and a short summary of its imported values, such as "W-2 · Acme Inc."; - **Enter by hand**: classified, but its import failed (unreadable pages, a provider error or timeout, a result that fails conversion or validation); a list kind is added with its kind selected and its payload in normal initialization; a single-document kind's occurrence stays unchanged; - **Replaces your current entry**: a single-document kind whose occurrence is no longer in normal initialization; its checkbox starts unchecked; - **Not imported**: each file and its pages that the classification lists in `not_imported`, and the count of documents dropped as duplicates. Each Ready and Enter-by-hand row has a checkbox, checked by default, so a misclassified document can be left out. **Add selected** adds the checked documents; **Cancel** discards them all. 6. **Apply.** Applying is one editor change. A checked single-document kind is applied to its occurrence as a single import; a second document of the same single-document kind is listed as **Not imported** (only one is allowed). **Selection order** decides: documents are ordered by the position of their first file and first page in the person's selection, never by when a reply arrives, so "second" is the later one in that order, and the review's rows and the appended items follow the same order. Each checked list-kind document is **always appended** as a new item of its category List (the import is started from a classifier, outside any document, so there is no current item to replace), its variant selected as a manual choice does, and its result applied under the [array import](#array-import) rules: the first form fills the new item, further forms of that kind are appended with the same variant, and duplicates of another imported form or of an existing item at the same position are dropped. Existing items are never changed; a filled single-document occurrence changes only when its **Replaces your current entry** row is checked. If any List's declared size would be exceeded, nothing is applied and the sheet names that List. 7. **After.** The Request is not submitted; the person reviews and saves as usual. The categorized import is one blocking action under the [pending and stale rules](#client-workflow), through its review: Cancel, navigation, or a disconnect discards every pending reply and leaves the editor unchanged. An error or timeout of any classification request ends the import, is reported, and leaves the editor unchanged; the person can retry. An import step's failure makes only that document Enter by hand. Without AI, every step above is absent; **Add** and each item's editor work as usual. ## Client workflow The model request itself, including a client's own provider account, follows [LLM requests](llm-client-requests.md). The **AI Import** action (Chinese: **AI导入**) first offers **Take a picture** using ordinary camera capture when the effective MIME filter accepts camera JPEG and a camera is available, and **Open a file**. Each action selects one file by default, or up to eight in user-selected order when enabled. Captured photos use the same preparation and validation as selected image files. Clients decode and orient selected images, flatten transparency on white, and JPEG-encode them at no more than **3,000,000 content bytes each**. Image preparation must keep reducing an oversized image's dimensions until its JPEG fits. JPEG quality stays at or above 70, and the source is never upscaled. An image that cannot be decoded or encoded reports an input error. PDF bytes remain unchanged. Clients check the combined LMB1 limit of 8 MiB before sending. The client uses a session-unique unsigned 64-bit transaction ID. A pending action belongs to its originating Control session, editor, struct occurrence, selected files, validation value, and result-field state. A cancel, timeout, navigation, or disconnect, or a change to a relevant value while it is pending (for example an arriving calculation result), makes its reply stale. Only the matching pending reply can change the editor. Like Submit, AI Import is disabled while any calculation is pending or deferred and becomes available when the calculations settle, so no calculation result can arrive during a transformation and make its reply stale. AI Import is blocking. While its transformation is pending, the client shows a busy overlay over the whole editor with a **Cancel** action; the editor and its actions accept no input, so no edit can make the pending reply stale and no second transformation or Submit can start. The busy state ends, and the editor accepts input again, when the matching reply arrives (a result is applied as above; an error is reported), when the user cancels, when the timeout elapses, or when the Control session disconnects. Cancel, timeout, and disconnect discard the pending action, so a late reply is ignored; an error, a timeout, or a disconnect is reported to the user, who can retry. Navigating away from the editor also discards it. On success, the client checks the JSON shape, strictly converts each value, validates the complete candidate, and applies all selected fields in one normal editor change, including values supplied by a Schema Default. Ordinary drafts and calculations remain active. AI Import does not submit the Request. An error or invalid result leaves the editor unchanged. The user can review and correct the filled fields before submitting the Request.