{
  "schemaVersion": "1.0",
  "title": "Runtime",
  "description": "Use this page to understand what the reviewed AIP runtime owns between trusted gateway dispatch and connector execution. It explains how capability contracts, policy, idempotency, approvals, transactions, queues, callbacks, and recovery for",
  "canonical": "https://getaip.org/docs/architecture/runtime",
  "route": "/docs/architecture/runtime",
  "source": "docs/architecture/runtime.md",
  "protocol": "Agent Interoperability Protocol",
  "protocolVersion": "1.0",
  "section": "Architecture and Security",
  "documentType": "Architecture",
  "language": "en",
  "revision": {
    "lastReviewedRevision": "97be86e9efedf07ecf1783b03800f683f107fb04",
    "documentationSourceRevision": "9192fef3695ad294994f2712f6d156241e5e92fb",
    "basis": "frontmatter"
  },
  "downloads": {
    "md": "/docs/download/architecture/runtime.md",
    "txt": "/docs/download/architecture/runtime.txt",
    "json": "/docs/download/architecture/runtime.json",
    "pdf": "/docs/download/architecture/runtime.pdf"
  },
  "content": {
    "format": "text/markdown",
    "markdown": "---\ntitle: Runtime\ndescription: Understand how trusted actions become durable lifecycle state and restartable execution\nkind: explanation\naudience: developer\nappliesTo: \"1.x\"\nwritingStandard: \"aip-docs/1.0\"\nlastReviewedRevision: \"97be86e9efedf07ecf1783b03800f683f107fb04\"\n---\n\n# Runtime\n\nUse this page to understand what the reviewed AIP runtime owns between trusted\ngateway dispatch and connector execution. It explains how capability contracts,\npolicy, idempotency, approvals, transactions, queues, callbacks, and recovery\nform one lifecycle without turning the runtime into a transport server or a\nproduct adapter.\n\nThe page describes version `1.0.0` of the Rust workspace at source revision\n`97be86e9efedf07ecf1783b03800f683f107fb04`. `aip-runtime` is a reusable\nin-process service composed by gateways and SDKs. Its service layout and storage\nchoices are implementation architecture, not a topology required by AIP 1.0.\n\n## Place the runtime after trusted ingress\n\nThe gateway establishes the authenticated actor, verified tenant, credential\nhandle, identity context, session, and correlation coordinates. The runtime\nuses that trusted context to govern work, select a capability implementation,\nrecord lifecycle evidence, and return a native AIP result or queued\nacknowledgement.\n\n| Boundary | Runtime owns | Runtime does not own |\n|---|---|---|\n| Admission | Capability lookup, schema and contract checks, policy, idempotency, approval, and transaction gates | Listener authentication, signature verification, or payload-to-identity trust |\n| Execution | Bounded handler invocation with trusted context, cancellation, streaming, and checkpoints | Product data-transfer objects, provider credentials, or provider API semantics |\n| Lifecycle | Results, events, approvals, transactions, delegations, queue state, callbacks, and query views | External observability storage or deployment-specific reporting |\n| Recovery | Lease-aware queued work, callback retry, delegation replay, and a structured restart report | Process supervision or proof that an uncertain provider effect did not occur |\n\nThe runtime can call a local handler or a catalog-resolved remote handler. Both\npaths enter the same action governance. Moving a connector into a fleet does\nnot create a second authorization or lifecycle model.\n\n## Compose one coherent state bundle\n\n`RuntimeStores` supplies 13 services as one deployment contract. The runtime\nrecords its durability class for the complete bundle; an embedder must not mix\nephemeral and durable implementations when replay or side-effect fencing\ndepends on restart safety.\n\n| Service | State or decision it owns |\n|---|---|\n| Storage health | A bounded read-and-write readiness check for the selected backend |\n| Maintenance | Retention of events, replay claims, dead letters, callback records, and published outbox records |\n| Replay | Atomic claims for signed envelope message IDs |\n| Profile state | Versioned, compare-and-set projections and bindings owned by protocol profiles |\n| Sessions | Session creation, state, and resume data |\n| Idempotency | Action intent reservations, ownership, input binding, and resolved results |\n| Events | Append-only lifecycle and operational events with cursor reads |\n| Lifecycle | Results, chunks, cancellations, escalations, conversations, receipts, audits, and settlements |\n| Delegations | Parent-child graph records and durable delegation outcomes |\n| Approvals | Requests, decisions, leases, and resumable approval state |\n| Transactions | Plans, state transitions, provider checkpoints, and recoverable records |\n| Action queue | Queued and running actions, leases, retries, terminal records, and dead letters |\n| Callback delivery | Durable callback attempts, lease state, retry windows, and terminal views |\n\nThe discovery service, admitted handler map, active-action map, policy object,\nrouters, dispatchers, and work semaphores are runtime composition rather than\nadditional entries in `RuntimeStores`.\n\nThree durability classes make the deployment promise explicit:\n\n- `ephemeral` is process-local and loses state on restart;\n- `durable_single_process` survives restart but coordinates one process at a\n  time; and\n- `durable_shared` survives restart and relies on backend-enforced coordination\n  between processes.\n\nThe built-in local file bundle is `durable_single_process`. The reviewed\nPostgreSQL bundle is `durable_shared` and supplies all 13 services from one\npool. Its queue uses transactions, row locks, skip-locked selection, and\nlease-identity checks. Merely serializing queue records does not provide the\nshared coordination guarantee.\n\nThe default runtime uses in-memory services. That is useful for embedding and\ntests, but its always-ready health implementation and no-op retention service\nare not evidence of durable production readiness.\n\n## Give handlers bounded trusted context\n\nAn action payload is not the handler's complete authority. Before invocation,\nthe runtime builds a non-serializable `ExecutionContext` from trusted and\nvalidated state.\n\n| Context value | Why it is present |\n|---|---|\n| Authenticated actor and verified tenant | Bind execution to transport-established identity and membership |\n| Opaque credential handle | Let the deployment credential provider resolve secret material without placing it in protocol state |\n| Deadline and cancellation token | Bound execution and support cooperative cancellation |\n| Idempotency reservation | Identify the one owned action intent for this attempt |\n| Verified approval set | Carry durable decisions and policy hashes that passed runtime verification |\n| Transaction context | Carry the stable transaction ID and recovered provider-operation coordinates |\n| Transaction checkpoint publisher | Persist a provider operation and reconciliation cursor at the uncertainty boundary |\n| Execution checkpoint publisher | Expose ordered, bounded crash-qualification observations without trusting payload telemetry |\n| Stream publisher | Persist and publish incremental chunks through runtime lifecycle state |\n| Trace and redaction policy | Carry trusted correlation and deny sensitive diagnostic fields |\n\nThe context is deliberately not an AIP schema and is not serialized into an\naction. Durable stores retain the verified source records needed to reconstruct\nit for another attempt. A connector cannot obtain more authority by adding\nlook-alike fields to its action input.\n\n## Admit capability and implementation together\n\nA local manifest becomes callable only when every non-resource capability has\na handler whose declared implementation support satisfies the capability\ncontract. Runtime admission validates the complete manifest-handler set under\none admission lock and publishes discovery and handler state together.\n\nA compatibility registration method revalidates the owning manifest before it\nreplaces one handler. A local capability present in discovery without its\nadmitted handler fails as `HandlerNotFound`; the runtime does not silently send\nit to a remote process.\n\nFor fleet capabilities, the runtime looks in bounded local discovery first and\nthen asks the tenant-scoped capability catalog. Only a capability absent from\nlocal discovery can use the configured catch-all remote handler. This ordering\nprevents a remote catalog entry from shadowing a local contract.\n\n## Follow one synchronous action\n\nThe synchronous path is ordered so an external handler sees only an action that\nhas passed all applicable gates.\n\n~~~mermaid\nflowchart LR\n    T[\"Trusted message context\"] --> C[\"Resolve and validate capability\"]\n    C --> P[\"Authorize policy\"]\n    P --> I[\"Reserve idempotent intent\"]\n    I --> A[\"Validate or request approval\"]\n    A --> X[\"Apply transaction gate\"]\n    X --> H[\"Select admitted handler\"]\n    H --> E[\"Execute with bounded context\"]\n    E --> V[\"Validate output\"]\n    V --> D[\"Persist lifecycle and transaction state\"]\n    D --> S[\"Settle idempotency result\"]\n~~~\n\nThe text equivalent is:\n\n1. The runtime replaces action identity with the gateway's resolved identity.\n   It normalizes transaction identifiers and restores reconciliation\n   coordinates when applicable.\n2. It rejects an action ID that aliases a different durable queued action.\n3. It resolves the capability from local discovery or the tenant-scoped remote\n   catalog, then validates input schema, mode, contract, and credential context.\n4. It applies capability policy and any transaction-specific policy\n   restrictions.\n5. It derives the idempotency claim and either owns a bounded reservation,\n   returns the already resolved result, or rejects a conflicting intent.\n6. It runs any transaction preview that must precede approval, validates\n   supplied approval evidence, and creates a pending approval when policy\n   requires a human decision.\n7. It applies the remaining pre-execution transaction operation. A dry run,\n   plan, or other non-executing branch can return here with durable evidence.\n8. It enriches only the connector-facing action, resolves compensation routing\n   when needed, and selects the admitted local or remote handler.\n9. It constructs trusted execution context, marks the action active, invokes\n   the handler under the effective timeout, and removes the active entry.\n10. It validates the output contract, persists the result and terminal action\n    record, records transaction outcome, and settles the idempotency reservation.\n\nThe runtime records ordered execution checkpoints around intent persistence,\nprovider invocation, response receipt, result persistence, and reconciliation.\nThese observations help qualify crash boundaries; they do not make a provider\neffect atomic with local storage.\n\n## Queue asynchronous work before execution\n\nAn asynchronous submission performs capability, contract, credential, policy,\nidempotency, approval, transaction, and handler-availability checks before it\nreturns `queued`. It then stores the action, trusted principal and message\ncontext, retry policy, timestamps, and owned idempotency reservation in the\naction queue.\n\nThe acknowledgement means the reviewed queue accepted the action. It does not\nmean that a worker started, the provider accepted the operation, or a callback\nwas delivered.\n\nA worker atomically leases either the next eligible action or a named action.\nWhile execution is active, a heartbeat renews both the queue lease and the\ntransferred idempotency reservation. Loss of the lease prevents that worker\nfrom claiming terminal ownership.\n\nThe worker re-enters the normal execution path with the transferred reservation\ninstead of acquiring a new intent. It then completes the attempt against the\nsame lease identity.\n\n| Attempt outcome | Queue behavior | Lifecycle meaning |\n|---|---|---|\n| Success or terminal non-retryable result | Settle the queue record | Persist terminal result and emit worker evidence |\n| Retryable result within policy | Remove the lease and set `next_attempt_at` | Keep the action non-terminal; do not publish a final callback yet |\n| Exhausted or unretryable failure | Move the attempt to dead-letter state | Persist terminal failure evidence |\n| Cancellation won before completion | Preserve cancelled state | Do not let the stale worker overwrite cancellation |\n| Lease lost | Reject worker settlement | Another worker or recovery cycle may own the record |\n\nRetry eligibility considers the capability policy, error category, attempt and\nelapsed budgets, idempotency requirements, and the future retry window. The\nfinite worker drain stops after bounded idle polling or a claim limit. A daemon\nor process supervisor owns the long-running loop, shutdown, and backpressure\npolicy.\n\n## Keep streams, callbacks, and completion separate\n\nThe stream publisher persists chunks through lifecycle state and can publish\nthem without waiting for the terminal handler result. Cancellation reaches the\nactive handler through the same trusted execution context.\n\nAfter a terminal asynchronous result, the runtime records an event and, when a\ncallback exists, creates or advances durable callback delivery state. Callback\nwork has an independent concurrency budget, lease, retry schedule, receipt\nchain, and terminal view. A completed action and a delivered callback are\ntherefore distinct facts.\n\nCallback recovery claims only a bounded batch that fits the available callback\nbudget. Reconciliation has a separate semaphore, so a callback storm cannot\nconsume the capacity reserved for uncertain transaction work.\n\n## Treat provider uncertainty as recoverable state\n\nThe runtime can persist a provider operation ID and optional reconciliation\ncursor before a connector awaits an uncertain commit response. A later\ntransaction reconciliation reconstructs that context and runs under its own\nwork budget.\n\nThis boundary reduces blind retries, but it cannot guarantee exactly-once\nexternal effects. A crash can still occur between a provider effect and its\ndurable checkpoint, and a provider may not offer an idempotent lookup. The\nconnector and provider contract determine whether reconciliation can establish\nthe outcome.\n\nApproval, idempotency, replay, and transactions solve different problems:\n\n| Mechanism | Question answered |\n|---|---|\n| Replay claim | Has this signed envelope message ID already been accepted in its time window? |\n| Idempotency reservation | Does this caller-owned action intent already have an owner or result? |\n| Approval record | Did an authorized decision satisfy the immutable policy and action binding? |\n| Transaction record | What plan, external-operation evidence, outcome, or compensation state survives? |\n\nNone of these records alone proves provider success.\n\n## Recover explicitly after restart\n\n`recover_runtime_state` produces one auditable report rather than hiding\nrestart work inside construction. Its configuration independently controls five\ncategories:\n\n1. include pending approvals in the report;\n2. include recoverable transaction or saga records;\n3. lease and replay queued or running actions;\n4. replay running delegations through the remote router or local fallback; and\n5. claim and retry recoverable callback deliveries.\n\nThe report separates successful queued and delegation results from typed\nerrors, and includes callback delivery views. Listing a pending approval or\nrecoverable transaction is not the same as deciding or reconciling it. The\nembedding supervisor remains responsible for sequencing recovery after handlers,\ncatalogs, routers, dispatchers, and durable stores are ready.\n\nThe local file store can recover state for one process. Shared workers require\na backend whose lease, renewal, completion, compare-and-set, and idempotency\noperations coordinate every process. The reviewed PostgreSQL implementation is\nthe reference shared backend for that contract.\n\n## Operate readiness and retention as narrow controls\n\nStorage health checks only that the configured runtime backend can perform its\nbounded readiness operation. It does not probe a connector host, provider,\ncallback destination, registry, transport listener, or process supervisor.\n\nRetention maintenance deletes bounded operational families according to the\nconfigured event, dead-letter, callback, and outbox windows, and also expires\nreplay claims. Operators must choose windows that preserve their audit,\nrecovery, and replay requirements. The protocol does not prescribe those\ndeployment values.\n\n## Failure ownership and non-implications\n\n| Failure | Owning runtime decision |\n|---|---|\n| Capability absent or local handler missing | Reject before connector execution |\n| Input, mode, credential, or policy mismatch | Return a typed failed result or runtime error before execution |\n| Idempotency conflict | Reject the different intent without reusing its result |\n| Approval required | Persist pending state and stop before execution |\n| Queue lease or reservation lost | Refuse stale worker settlement |\n| Retry budget exhausted | Persist terminal failure and dead-letter evidence |\n| Callback target unavailable | Retain independent callback retry state |\n| Provider outcome uncertain | Preserve transaction coordinates for explicit reconciliation when supported |\n| Durable backend unavailable | Fail the state transition rather than claim durability |\n\nThe runtime does not authenticate transport connections, hold raw provider\nsecrets, define product payloads, start connector processes, or supervise an\ninfinite worker loop. Its support for a store, handler, transaction mode,\ncallback, or recovery method does not prove that a deployment enabled or\nqualified that path.\n\n## Related pages\n\n- [Actions and sessions](../concepts/actions-and-sessions.md)\n- [Approvals and policy](../concepts/approvals-and-policy.md)\n- [Transactions and compensation](../concepts/transactions-and-compensation.md)\n- [Gateway](gateway.md)\n- [Observe and recover](../guides/observe-and-recover.md)\n",
    "text": "Runtime\n\nUse this page to understand what the reviewed AIP runtime owns between trusted\ngateway dispatch and connector execution. It explains how capability contracts,\npolicy, idempotency, approvals, transactions, queues, callbacks, and recovery\nform one lifecycle without turning the runtime into a transport server or a\nproduct adapter.\n\nThe page describes version 1.0.0 of the Rust workspace at source revision\n97be86e9efedf07ecf1783b03800f683f107fb04. aip-runtime is a reusable\nin-process service composed by gateways and SDKs. Its service layout and storage\nchoices are implementation architecture, not a topology required by AIP 1.0.\n\nPlace the runtime after trusted ingress\n\nThe gateway establishes the authenticated actor, verified tenant, credential\nhandle, identity context, session, and correlation coordinates. The runtime\nuses that trusted context to govern work, select a capability implementation,\nrecord lifecycle evidence, and return a native AIP result or queued\nacknowledgement.\n\n| Boundary | Runtime owns | Runtime does not own |\n\n| Admission | Capability lookup, schema and contract checks, policy, idempotency, approval, and transaction gates | Listener authentication, signature verification, or payload-to-identity trust |\n| Execution | Bounded handler invocation with trusted context, cancellation, streaming, and checkpoints | Product data-transfer objects, provider credentials, or provider API semantics |\n| Lifecycle | Results, events, approvals, transactions, delegations, queue state, callbacks, and query views | External observability storage or deployment-specific reporting |\n| Recovery | Lease-aware queued work, callback retry, delegation replay, and a structured restart report | Process supervision or proof that an uncertain provider effect did not occur |\n\nThe runtime can call a local handler or a catalog-resolved remote handler. Both\npaths enter the same action governance. Moving a connector into a fleet does\nnot create a second authorization or lifecycle model.\n\nCompose one coherent state bundle\n\nRuntimeStores supplies 13 services as one deployment contract. The runtime\nrecords its durability class for the complete bundle; an embedder must not mix\nephemeral and durable implementations when replay or side-effect fencing\ndepends on restart safety.\n\n| Service | State or decision it owns |\n\n| Storage health | A bounded read-and-write readiness check for the selected backend |\n| Maintenance | Retention of events, replay claims, dead letters, callback records, and published outbox records |\n| Replay | Atomic claims for signed envelope message IDs |\n| Profile state | Versioned, compare-and-set projections and bindings owned by protocol profiles |\n| Sessions | Session creation, state, and resume data |\n| Idempotency | Action intent reservations, ownership, input binding, and resolved results |\n| Events | Append-only lifecycle and operational events with cursor reads |\n| Lifecycle | Results, chunks, cancellations, escalations, conversations, receipts, audits, and settlements |\n| Delegations | Parent-child graph records and durable delegation outcomes |\n| Approvals | Requests, decisions, leases, and resumable approval state |\n| Transactions | Plans, state transitions, provider checkpoints, and recoverable records |\n| Action queue | Queued and running actions, leases, retries, terminal records, and dead letters |\n| Callback delivery | Durable callback attempts, lease state, retry windows, and terminal views |\n\nThe discovery service, admitted handler map, active-action map, policy object,\nrouters, dispatchers, and work semaphores are runtime composition rather than\nadditional entries in RuntimeStores.\n\nThree durability classes make the deployment promise explicit:\n• ephemeral is process-local and loses state on restart;\n• durablesingleprocess survives restart but coordinates one process at a\n  time; and\n• durableshared survives restart and relies on backend-enforced coordination\n  between processes.\n\nThe built-in local file bundle is durablesingleprocess. The reviewed\nPostgreSQL bundle is durableshared and supplies all 13 services from one\npool. Its queue uses transactions, row locks, skip-locked selection, and\nlease-identity checks. Merely serializing queue records does not provide the\nshared coordination guarantee.\n\nThe default runtime uses in-memory services. That is useful for embedding and\ntests, but its always-ready health implementation and no-op retention service\nare not evidence of durable production readiness.\n\nGive handlers bounded trusted context\n\nAn action payload is not the handler's complete authority. Before invocation,\nthe runtime builds a non-serializable ExecutionContext from trusted and\nvalidated state.\n\n| Context value | Why it is present |\n\n| Authenticated actor and verified tenant | Bind execution to transport-established identity and membership |\n| Opaque credential handle | Let the deployment credential provider resolve secret material without placing it in protocol state |\n| Deadline and cancellation token | Bound execution and support cooperative cancellation |\n| Idempotency reservation | Identify the one owned action intent for this attempt |\n| Verified approval set | Carry durable decisions and policy hashes that passed runtime verification |\n| Transaction context | Carry the stable transaction ID and recovered provider-operation coordinates |\n| Transaction checkpoint publisher | Persist a provider operation and reconciliation cursor at the uncertainty boundary |\n| Execution checkpoint publisher | Expose ordered, bounded crash-qualification observations without trusting payload telemetry |\n| Stream publisher | Persist and publish incremental chunks through runtime lifecycle state |\n| Trace and redaction policy | Carry trusted correlation and deny sensitive diagnostic fields |\n\nThe context is deliberately not an AIP schema and is not serialized into an\naction. Durable stores retain the verified source records needed to reconstruct\nit for another attempt. A connector cannot obtain more authority by adding\nlook-alike fields to its action input.\n\nAdmit capability and implementation together\n\nA local manifest becomes callable only when every non-resource capability has\na handler whose declared implementation support satisfies the capability\ncontract. Runtime admission validates the complete manifest-handler set under\none admission lock and publishes discovery and handler state together.\n\nA compatibility registration method revalidates the owning manifest before it\nreplaces one handler. A local capability present in discovery without its\nadmitted handler fails as HandlerNotFound; the runtime does not silently send\nit to a remote process.\n\nFor fleet capabilities, the runtime looks in bounded local discovery first and\nthen asks the tenant-scoped capability catalog. Only a capability absent from\nlocal discovery can use the configured catch-all remote handler. This ordering\nprevents a remote catalog entry from shadowing a local contract.\n\nFollow one synchronous action\n\nThe synchronous path is ordered so an external handler sees only an action that\nhas passed all applicable gates.\n\nmermaid\nflowchart LR\n    T[\"Trusted message context\"] --> C[\"Resolve and validate capability\"]\n    C --> P[\"Authorize policy\"]\n    P --> I[\"Reserve idempotent intent\"]\n    I --> A[\"Validate or request approval\"]\n    A --> X[\"Apply transaction gate\"]\n    X --> H[\"Select admitted handler\"]\n    H --> E[\"Execute with bounded context\"]\n    E --> V[\"Validate output\"]\n    V --> D[\"Persist lifecycle and transaction state\"]\n    D --> S[\"Settle idempotency result\"]\n\nThe text equivalent is:\n1. The runtime replaces action identity with the gateway's resolved identity.\n   It normalizes transaction identifiers and restores reconciliation\n   coordinates when applicable.\n2. It rejects an action ID that aliases a different durable queued action.\n3. It resolves the capability from local discovery or the tenant-scoped remote\n   catalog, then validates input schema, mode, contract, and credential context.\n4. It applies capability policy and any transaction-specific policy\n   restrictions.\n5. It derives the idempotency claim and either owns a bounded reservation,\n   returns the already resolved result, or rejects a conflicting intent.\n6. It runs any transaction preview that must precede approval, validates\n   supplied approval evidence, and creates a pending approval when policy\n   requires a human decision.\n7. It applies the remaining pre-execution transaction operation. A dry run,\n   plan, or other non-executing branch can return here with durable evidence.\n8. It enriches only the connector-facing action, resolves compensation routing\n   when needed, and selects the admitted local or remote handler.\n9. It constructs trusted execution context, marks the action active, invokes\n   the handler under the effective timeout, and removes the active entry.\n10. It validates the output contract, persists the result and terminal action\n    record, records transaction outcome, and settles the idempotency reservation.\n\nThe runtime records ordered execution checkpoints around intent persistence,\nprovider invocation, response receipt, result persistence, and reconciliation.\nThese observations help qualify crash boundaries; they do not make a provider\neffect atomic with local storage.\n\nQueue asynchronous work before execution\n\nAn asynchronous submission performs capability, contract, credential, policy,\nidempotency, approval, transaction, and handler-availability checks before it\nreturns queued. It then stores the action, trusted principal and message\ncontext, retry policy, timestamps, and owned idempotency reservation in the\naction queue.\n\nThe acknowledgement means the reviewed queue accepted the action. It does not\nmean that a worker started, the provider accepted the operation, or a callback\nwas delivered.\n\nA worker atomically leases either the next eligible action or a named action.\nWhile execution is active, a heartbeat renews both the queue lease and the\ntransferred idempotency reservation. Loss of the lease prevents that worker\nfrom claiming terminal ownership.\n\nThe worker re-enters the normal execution path with the transferred reservation\ninstead of acquiring a new intent. It then completes the attempt against the\nsame lease identity.\n\n| Attempt outcome | Queue behavior | Lifecycle meaning |\n\n| Success or terminal non-retryable result | Settle the queue record | Persist terminal result and emit worker evidence |\n| Retryable result within policy | Remove the lease and set nextattemptat | Keep the action non-terminal; do not publish a final callback yet |\n| Exhausted or unretryable failure | Move the attempt to dead-letter state | Persist terminal failure evidence |\n| Cancellation won before completion | Preserve cancelled state | Do not let the stale worker overwrite cancellation |\n| Lease lost | Reject worker settlement | Another worker or recovery cycle may own the record |\n\nRetry eligibility considers the capability policy, error category, attempt and\nelapsed budgets, idempotency requirements, and the future retry window. The\nfinite worker drain stops after bounded idle polling or a claim limit. A daemon\nor process supervisor owns the long-running loop, shutdown, and backpressure\npolicy.\n\nKeep streams, callbacks, and completion separate\n\nThe stream publisher persists chunks through lifecycle state and can publish\nthem without waiting for the terminal handler result. Cancellation reaches the\nactive handler through the same trusted execution context.\n\nAfter a terminal asynchronous result, the runtime records an event and, when a\ncallback exists, creates or advances durable callback delivery state. Callback\nwork has an independent concurrency budget, lease, retry schedule, receipt\nchain, and terminal view. A completed action and a delivered callback are\ntherefore distinct facts.\n\nCallback recovery claims only a bounded batch that fits the available callback\nbudget. Reconciliation has a separate semaphore, so a callback storm cannot\nconsume the capacity reserved for uncertain transaction work.\n\nTreat provider uncertainty as recoverable state\n\nThe runtime can persist a provider operation ID and optional reconciliation\ncursor before a connector awaits an uncertain commit response. A later\ntransaction reconciliation reconstructs that context and runs under its own\nwork budget.\n\nThis boundary reduces blind retries, but it cannot guarantee exactly-once\nexternal effects. A crash can still occur between a provider effect and its\ndurable checkpoint, and a provider may not offer an idempotent lookup. The\nconnector and provider contract determine whether reconciliation can establish\nthe outcome.\n\nApproval, idempotency, replay, and transactions solve different problems:\n\n| Mechanism | Question answered |\n\n| Replay claim | Has this signed envelope message ID already been accepted in its time window? |\n| Idempotency reservation | Does this caller-owned action intent already have an owner or result? |\n| Approval record | Did an authorized decision satisfy the immutable policy and action binding? |\n| Transaction record | What plan, external-operation evidence, outcome, or compensation state survives? |\n\nNone of these records alone proves provider success.\n\nRecover explicitly after restart\n\nrecoverruntimestate produces one auditable report rather than hiding\nrestart work inside construction. Its configuration independently controls five\ncategories:\n1. include pending approvals in the report;\n2. include recoverable transaction or saga records;\n3. lease and replay queued or running actions;\n4. replay running delegations through the remote router or local fallback; and\n5. claim and retry recoverable callback deliveries.\n\nThe report separates successful queued and delegation results from typed\nerrors, and includes callback delivery views. Listing a pending approval or\nrecoverable transaction is not the same as deciding or reconciling it. The\nembedding supervisor remains responsible for sequencing recovery after handlers,\ncatalogs, routers, dispatchers, and durable stores are ready.\n\nThe local file store can recover state for one process. Shared workers require\na backend whose lease, renewal, completion, compare-and-set, and idempotency\noperations coordinate every process. The reviewed PostgreSQL implementation is\nthe reference shared backend for that contract.\n\nOperate readiness and retention as narrow controls\n\nStorage health checks only that the configured runtime backend can perform its\nbounded readiness operation. It does not probe a connector host, provider,\ncallback destination, registry, transport listener, or process supervisor.\n\nRetention maintenance deletes bounded operational families according to the\nconfigured event, dead-letter, callback, and outbox windows, and also expires\nreplay claims. Operators must choose windows that preserve their audit,\nrecovery, and replay requirements. The protocol does not prescribe those\ndeployment values.\n\nFailure ownership and non-implications\n\n| Failure | Owning runtime decision |\n\n| Capability absent or local handler missing | Reject before connector execution |\n| Input, mode, credential, or policy mismatch | Return a typed failed result or runtime error before execution |\n| Idempotency conflict | Reject the different intent without reusing its result |\n| Approval required | Persist pending state and stop before execution |\n| Queue lease or reservation lost | Refuse stale worker settlement |\n| Retry budget exhausted | Persist terminal failure and dead-letter evidence |\n| Callback target unavailable | Retain independent callback retry state |\n| Provider outcome uncertain | Preserve transaction coordinates for explicit reconciliation when supported |\n| Durable backend unavailable | Fail the state transition rather than claim durability |\n\nThe runtime does not authenticate transport connections, hold raw provider\nsecrets, define product payloads, start connector processes, or supervise an\ninfinite worker loop. Its support for a store, handler, transaction mode,\ncallback, or recovery method does not prove that a deployment enabled or\nqualified that path.\n\nRelated pages\n• Actions and sessions (../concepts/actions-and-sessions.md)\n• Approvals and policy (../concepts/approvals-and-policy.md)\n• Transactions and compensation (../concepts/transactions-and-compensation.md)\n• Gateway (gateway.md)\n• Observe and recover (../guides/observe-and-recover.md)\n"
  },
  "integrity": {
    "algorithm": "sha256",
    "sourceDigest": "1bead11ff612ba9b4df7f0300607360a7091779b2efe6ff4bb8cdb1945290c19"
  }
}