{
  "schemaVersion": "1.0",
  "title": "Troubleshoot the CrewAI connector",
  "description": "Use this index to move from one observed CrewAI connector symptom to a bounded check and safe response. Preserve the original Action, host runtime, and sidecar journal before restarting, retrying, archiving, or replacing state.",
  "canonical": "https://getaip.org/docs/connectors/crewai/troubleshooting",
  "route": "/docs/connectors/crewai/troubleshooting",
  "source": "docs/connectors/crewai/troubleshooting/README.md",
  "protocol": "Agent Interoperability Protocol",
  "protocolVersion": "1.0",
  "section": "Connectors",
  "documentType": "Connector",
  "language": "en",
  "downloads": {
    "md": "/docs/download/connectors/crewai/troubleshooting.md",
    "txt": "/docs/download/connectors/crewai/troubleshooting.txt",
    "json": "/docs/download/connectors/crewai/troubleshooting.json",
    "pdf": "/docs/download/connectors/crewai/troubleshooting.pdf"
  },
  "content": {
    "format": "text/markdown",
    "markdown": "---\ntitle: Troubleshoot the CrewAI connector\ndescription: >-\n  Diagnose CrewAI startup, readiness, journal, stream, timeout, cancellation,\n  and provider failures without unsafe retries\nkind: runbook\naudience: operator\nappliesTo: \"1.x\"\nwritingStandard: \"aip-docs/1.0\"\nlastReviewedRevision: \"97be86e9efedf07ecf1783b03800f683f107fb04\"\nconnector: crewai\naliases:\n  - CrewAI connector troubleshooting\n  - CrewAI connector errors\n  - Diagnose aip-host-crewai\n---\n\n# Troubleshoot the CrewAI connector\n\nUse this index to move from one observed CrewAI connector symptom to a bounded\ncheck and safe response. Preserve the original Action, host runtime, and\nsidecar journal before restarting, retrying, archiving, or replacing state.\n\nThe connector marks every reviewed `ConnectorFailure` as non-retryable. A\ntemporary category or HTTP status does not authorize automatic replay.\n\n## Capture the incident identity first\n\nRecord:\n\n- UTC start time and first observed symptom;\n- tenant, connector type, version, instance, and replica;\n- host and sidecar image digests;\n- AIP source and CrewAI method-verification revisions;\n- configuration and credential revisions;\n- crew ID, capability ID, operation, and Action ID;\n- sidecar status, event sequence, and request hash when available;\n- host `/ready`, metrics, registry lease, and route assignment;\n- provider, model, tool, and artifact identifiers;\n- every operator action already taken.\n\nDo not copy Bearer tokens, signing seeds, model credentials, prompts, or\ngoverned output into an incident ticket.\n\n## Use the symptom index\n\n| Symptom | First check | Containment | Next section |\n|---|---|---|---|\n| Sidecar will not start | Process error and mounted paths | Keep host out of routing | Startup failures |\n| Host is alive but not ready | Structured host `/ready` | Stop new assignments | Readiness failures |\n| Sidecar returns `401` | Host and sidecar token identity | Quarantine the route | Authentication failures |\n| Capability is absent | Descriptor and sidecar operation policy | Do not widen policy during incident | Discovery failures |\n| Start returns `409` | Original Action and request hash | Preserve existing record | Identity collisions |\n| Start returns `507` | Journal count, bytes, and persistence | Stop new job admission | Journal failures |\n| Stream fails or closes early | Event sequence, byte limits, terminal event | Reconcile admitted work | Stream failures |\n| Action times out or is cancelled | Execution mechanism and provider evidence | Assume effects may continue | Uncertain outcomes |\n| Terminal job failed | Sidecar event and restricted logs | Preserve traceback securely | Provider failures |\n\n## Diagnose startup failures\n\n| Message family | Source-backed cause | Safe correction |\n|---|---|---|\n| Registry reference is required or malformed | Missing value or not `module:attribute` | Mount the intended registry module and exact attribute |\n| Registry missing, empty, or invalid | Attribute does not exist, is not a mapping, or has invalid factories | Correct code or registry identity; do not substitute an unrelated crew |\n| Durable state file is required | Non-loopback deployment lacks `AIP_CREWAI_STATE_FILE` | Mount the protected persistent volume and configure the file |\n| Journal already owned | Another process holds the companion lock | Find the active owner; never delete the lock while it runs |\n| Journal path, mode, or size invalid | Symlink, non-regular file, mode other than `0600`, or configured bound exceeded | Preserve bytes, repair ownership or restore from verified state |\n| Training directory required or unsafe | `train` enabled without protected storage | Provide a non-symlink directory without group or world write permission |\n| Bearer authentication required | No valid token outside explicit loopback development | Project one owner-controlled token into host and sidecar |\n| Runtime bound invalid | Concurrency, run, event, checkpoint, or state setting is outside code bounds | Restore a reviewed in-range configuration |\n\nIf journal decoding or validation fails, preserve the file before repair.\nStarting with an empty journal can remove duplicate-work protection.\n\n## Diagnose host construction failures\n\n| Connector error | Meaning | Correction |\n|---|---|---|\n| `NoCrews` | Host descriptor array is empty | Provide one through 1,000 reviewed descriptors |\n| `InvalidCrew` | Descriptor ID, name, capability ID, or allowlist is invalid | Correct the descriptor source and repeat admission review |\n| `InvalidUrl` | Sidecar URL cannot be parsed | Supply one absolute valid URL |\n| `InvalidBaseUrl` | Credentials, query, fragment, or insecure non-loopback HTTP violates policy | Remove forbidden URL parts and use private HTTPS |\n| `InvalidClient` | Hardened HTTP client construction failed | Inspect TLS and client configuration at startup |\n| `InvalidCredential` | Host token material is empty, contains controls, or is not UTF-8 | Rotate the projected token file and restart deliberately |\n\nThe host validates descriptors before registration. A startup correction can\nchange the manifest digest, so re-run admission when capability material\nchanges.\n\n## Diagnose discovery and readiness failures\n\nThe CrewAI connector readiness probe fails when:\n\n- authenticated sidecar health cannot be read;\n- sidecar `status` is not `ready`;\n- `durable_journal` is not true;\n- a configured crew is absent from sidecar health;\n- a descriptor operation is absent from sidecar policy.\n\nCompare the host crew file with the sidecar registry and\n`AIP_CREWAI_ALLOWED_OPERATIONS`. Keep the narrower policy during diagnosis.\nWiden it only through a reviewed configuration and admission change.\n\nHost `/ready` can also fail because the replica drains, its lease expires, or\nPostgreSQL is not readable and writable. Diagnose those fields before changing\nCrewAI configuration.\n\n## Diagnose authentication failures\n\nSidecar HTTP `401` maps to `connector.crewai.authentication`. Confirm:\n\n1. host and sidecar read the intended token projections;\n2. neither projection changed only on one process;\n3. the token is non-empty UTF-8 within 16 KiB;\n4. the private ingress preserves the `Authorization` header;\n5. old and new token windows follow the deployment rotation procedure.\n\nDo not print the header or token. Compare secret versions or digests through an\napproved secret-management channel.\n\nSidecar HTTP `403` means the startable operation is disabled by sidecar policy.\nIt maps to the same auth category at the Rust boundary, but token rotation will\nnot repair it.\n\n## Diagnose configuration or input rejection\n\n`connector.crewai.configuration_or_input` covers several local failures:\n\n- capability ID is not configured;\n- operation is disabled for the crew or execution path;\n- `run_action_id` is missing or invalid;\n- required idempotency metadata is absent;\n- descriptor, URL, or client configuration is invalid.\n\nSidecar HTTP `400`, `404`, `409`, `413`, `416`, `422`, and `428` map to\n`connector.crewai.remote_rejected`. Inspect the retained sidecar response body\nand classify it with this table:\n\n| Status | Typical correction |\n|---:|---|\n| `400` | Correct an invalid key or mismatched path and body Action ID |\n| `404` | Use an admitted crew or an existing target Action ID |\n| `409` | Reuse exact request material or preserve the existing terminal record |\n| `413` | Reduce encoded operation input below 1 MiB |\n| `416` | Resume from a cursor between zero and current event count |\n| `422` | Match the exact operation-specific input contract |\n| `428` | Supply the required idempotency key or archive confirmation |\n\nDo not replace an Action ID to make a `409` disappear. The collision protects\nthe original provider-effect boundary.\n\n## Diagnose journal failures\n\nHTTP `507` occurs when retained jobs reach the configured count or the sidecar\ncannot persist the admission fence. It maps to\n`connector.crewai.remote_temporary` but remains non-retryable.\n\nFor capacity exhaustion:\n\n1. stop new assignments;\n2. record retained and maximum run counts;\n3. identify only terminal records beyond the AIP replay-retention window;\n4. retain their evidence externally;\n5. archive each through exact Action confirmation;\n6. verify the journal and health before restoring traffic.\n\nFor persistence failure, inspect volume capacity, inode capacity, ownership,\nmode, filesystem errors, temporary-file creation, replacement, and directory\n`fsync`. Preserve the last valid journal.\n\nNever truncate or edit `runs.json` in place while the sidecar owns its lock.\n\n## Diagnose stream failures\n\n`connector.crewai.invalid_stream` covers:\n\n- transport chunks that are not valid SSE framing;\n- non-UTF-8 frames or invalid JSON data;\n- missing event name or sequence;\n- a sequence gap or duplicate;\n- more than 10,000 accepted events;\n- an SSE buffer over 1 MiB;\n- stream closure before a terminal event;\n- downstream chunk-emission failure.\n\n`connector.crewai.response_too_large` means a response or complete stream\nexceeded the host-configured byte bound.\n\nAfter admitted streamed work fails, the connector attempts sidecar cancellation\nfor five cancellable operations. A successful attempt preserves the original\nstream error; a failed attempt becomes `connector.crewai.cancelled_unconfirmed`.\n\nReconcile the target Action through its status and events. Do not create a new\nAction merely because the client lost the stream.\n\n## Diagnose timeout and cancellation\n\nSidecar HTTP `504` means the blocking route exceeded its wait duration. HTTP\n`502` means the durable job reached failed state before the blocking response.\n\nBoth map to a temporary connector category for invocation, with uncertainty\nset for `5xx`. The connector still sets `retryable` to false.\n\nFor `cancelled_unconfirmed`, preserve:\n\n- whether an HTTP response arrived before local cancellation;\n- whether the operation used native async work or a worker thread;\n- the sidecar record and terminal event;\n- model, tool, training, or evaluator effects;\n- any artifact created after the cancellation timestamp.\n\nReplay, train, test, memory reset, and fallback paths can continue in worker\nthreads after their asyncio waiter ends. A successful cancel route does not\nprove those calls stopped.\n\n## Diagnose terminal provider failure\n\nA sidecar execution exception appends a `failed` event with:\n\n- `status: failed`;\n- Python exception class in `error_type`;\n- a generic message directing the operator to logs by Action ID.\n\nThe blocking route returns HTTP `502`. A stream returns a failed AIP result\nwith code `connector.crewai.execution_failed` and the event details.\n\nReview restricted sidecar logs for the matching Action ID. The log includes the\nprovider exception and traceback, so treat it as confidential.\n\nDo not infer retry safety from the exception class alone. First determine\nwhether models, tools, memory, training artifacts, or downstream systems\nreceived effects.\n\n## Escalate with complete evidence\n\nEscalate when identity cannot be reconciled, the journal is corrupted, a\nthread-backed call continues after containment, a stream violates sequence, or\nprovider effects remain unknown.\n\nInclude:\n\n- the incident identity captured at the start;\n- exact AIP and CrewAI revisions;\n- redacted host readiness and sidecar health;\n- relevant metric deltas;\n- sidecar record and event slice;\n- restricted log location, not copied secret-bearing lines;\n- provider and downstream evidence;\n- containment and recovery decisions;\n- a clear statement of what remains unproved.\n\n## Related documentation\n\n- [CrewAI health, observability, and recovery](../operations/health-observability-and-recovery.md)\n- [CrewAI sidecar contract and lockfile](../reference/sidecar-contract-and-lockfile.md)\n- [CrewAI idempotency, cancellation, and process fencing](../reference/idempotency-cancellation-and-process-fencing.md)\n- [CrewAI capability index](../capabilities/README.md)\n- [Errors and retry decisions](../../../reference/errors.md)\n",
    "text": "Troubleshoot the CrewAI connector\n\nUse this index to move from one observed CrewAI connector symptom to a bounded\ncheck and safe response. Preserve the original Action, host runtime, and\nsidecar journal before restarting, retrying, archiving, or replacing state.\n\nThe connector marks every reviewed ConnectorFailure as non-retryable. A\ntemporary category or HTTP status does not authorize automatic replay.\n\nCapture the incident identity first\n\nRecord:\n• UTC start time and first observed symptom;\n• tenant, connector type, version, instance, and replica;\n• host and sidecar image digests;\n• AIP source and CrewAI method-verification revisions;\n• configuration and credential revisions;\n• crew ID, capability ID, operation, and Action ID;\n• sidecar status, event sequence, and request hash when available;\n• host /ready, metrics, registry lease, and route assignment;\n• provider, model, tool, and artifact identifiers;\n• every operator action already taken.\n\nDo not copy Bearer tokens, signing seeds, model credentials, prompts, or\ngoverned output into an incident ticket.\n\nUse the symptom index\n\n| Symptom | First check | Containment | Next section |\n\n| Sidecar will not start | Process error and mounted paths | Keep host out of routing | Startup failures |\n| Host is alive but not ready | Structured host /ready | Stop new assignments | Readiness failures |\n| Sidecar returns 401 | Host and sidecar token identity | Quarantine the route | Authentication failures |\n| Capability is absent | Descriptor and sidecar operation policy | Do not widen policy during incident | Discovery failures |\n| Start returns 409 | Original Action and request hash | Preserve existing record | Identity collisions |\n| Start returns 507 | Journal count, bytes, and persistence | Stop new job admission | Journal failures |\n| Stream fails or closes early | Event sequence, byte limits, terminal event | Reconcile admitted work | Stream failures |\n| Action times out or is cancelled | Execution mechanism and provider evidence | Assume effects may continue | Uncertain outcomes |\n| Terminal job failed | Sidecar event and restricted logs | Preserve traceback securely | Provider failures |\n\nDiagnose startup failures\n\n| Message family | Source-backed cause | Safe correction |\n\n| Registry reference is required or malformed | Missing value or not module:attribute | Mount the intended registry module and exact attribute |\n| Registry missing, empty, or invalid | Attribute does not exist, is not a mapping, or has invalid factories | Correct code or registry identity; do not substitute an unrelated crew |\n| Durable state file is required | Non-loopback deployment lacks AIPCREWAISTATEFILE | Mount the protected persistent volume and configure the file |\n| Journal already owned | Another process holds the companion lock | Find the active owner; never delete the lock while it runs |\n| Journal path, mode, or size invalid | Symlink, non-regular file, mode other than 0600, or configured bound exceeded | Preserve bytes, repair ownership or restore from verified state |\n| Training directory required or unsafe | train enabled without protected storage | Provide a non-symlink directory without group or world write permission |\n| Bearer authentication required | No valid token outside explicit loopback development | Project one owner-controlled token into host and sidecar |\n| Runtime bound invalid | Concurrency, run, event, checkpoint, or state setting is outside code bounds | Restore a reviewed in-range configuration |\n\nIf journal decoding or validation fails, preserve the file before repair.\nStarting with an empty journal can remove duplicate-work protection.\n\nDiagnose host construction failures\n\n| Connector error | Meaning | Correction |\n\n| NoCrews | Host descriptor array is empty | Provide one through 1,000 reviewed descriptors |\n| InvalidCrew | Descriptor ID, name, capability ID, or allowlist is invalid | Correct the descriptor source and repeat admission review |\n| InvalidUrl | Sidecar URL cannot be parsed | Supply one absolute valid URL |\n| InvalidBaseUrl | Credentials, query, fragment, or insecure non-loopback HTTP violates policy | Remove forbidden URL parts and use private HTTPS |\n| InvalidClient | Hardened HTTP client construction failed | Inspect TLS and client configuration at startup |\n| InvalidCredential | Host token material is empty, contains controls, or is not UTF-8 | Rotate the projected token file and restart deliberately |\n\nThe host validates descriptors before registration. A startup correction can\nchange the manifest digest, so re-run admission when capability material\nchanges.\n\nDiagnose discovery and readiness failures\n\nThe CrewAI connector readiness probe fails when:\n• authenticated sidecar health cannot be read;\n• sidecar status is not ready;\n• durablejournal is not true;\n• a configured crew is absent from sidecar health;\n• a descriptor operation is absent from sidecar policy.\n\nCompare the host crew file with the sidecar registry and\nAIPCREWAIALLOWEDOPERATIONS. Keep the narrower policy during diagnosis.\nWiden it only through a reviewed configuration and admission change.\n\nHost /ready can also fail because the replica drains, its lease expires, or\nPostgreSQL is not readable and writable. Diagnose those fields before changing\nCrewAI configuration.\n\nDiagnose authentication failures\n\nSidecar HTTP 401 maps to connector.crewai.authentication. Confirm:\n1. host and sidecar read the intended token projections;\n2. neither projection changed only on one process;\n3. the token is non-empty UTF-8 within 16 KiB;\n4. the private ingress preserves the Authorization header;\n5. old and new token windows follow the deployment rotation procedure.\n\nDo not print the header or token. Compare secret versions or digests through an\napproved secret-management channel.\n\nSidecar HTTP 403 means the startable operation is disabled by sidecar policy.\nIt maps to the same auth category at the Rust boundary, but token rotation will\nnot repair it.\n\nDiagnose configuration or input rejection\n\nconnector.crewai.configurationorinput covers several local failures:\n• capability ID is not configured;\n• operation is disabled for the crew or execution path;\n• runactionid is missing or invalid;\n• required idempotency metadata is absent;\n• descriptor, URL, or client configuration is invalid.\n\nSidecar HTTP 400, 404, 409, 413, 416, 422, and 428 map to\nconnector.crewai.remoterejected. Inspect the retained sidecar response body\nand classify it with this table:\n\n| Status | Typical correction |\n\n| 400 | Correct an invalid key or mismatched path and body Action ID |\n| 404 | Use an admitted crew or an existing target Action ID |\n| 409 | Reuse exact request material or preserve the existing terminal record |\n| 413 | Reduce encoded operation input below 1 MiB |\n| 416 | Resume from a cursor between zero and current event count |\n| 422 | Match the exact operation-specific input contract |\n| 428 | Supply the required idempotency key or archive confirmation |\n\nDo not replace an Action ID to make a 409 disappear. The collision protects\nthe original provider-effect boundary.\n\nDiagnose journal failures\n\nHTTP 507 occurs when retained jobs reach the configured count or the sidecar\ncannot persist the admission fence. It maps to\nconnector.crewai.remotetemporary but remains non-retryable.\n\nFor capacity exhaustion:\n1. stop new assignments;\n2. record retained and maximum run counts;\n3. identify only terminal records beyond the AIP replay-retention window;\n4. retain their evidence externally;\n5. archive each through exact Action confirmation;\n6. verify the journal and health before restoring traffic.\n\nFor persistence failure, inspect volume capacity, inode capacity, ownership,\nmode, filesystem errors, temporary-file creation, replacement, and directory\nfsync. Preserve the last valid journal.\n\nNever truncate or edit runs.json in place while the sidecar owns its lock.\n\nDiagnose stream failures\n\nconnector.crewai.invalidstream covers:\n• transport chunks that are not valid SSE framing;\n• non-UTF-8 frames or invalid JSON data;\n• missing event name or sequence;\n• a sequence gap or duplicate;\n• more than 10,000 accepted events;\n• an SSE buffer over 1 MiB;\n• stream closure before a terminal event;\n• downstream chunk-emission failure.\n\nconnector.crewai.responsetoolarge means a response or complete stream\nexceeded the host-configured byte bound.\n\nAfter admitted streamed work fails, the connector attempts sidecar cancellation\nfor five cancellable operations. A successful attempt preserves the original\nstream error; a failed attempt becomes connector.crewai.cancelledunconfirmed.\n\nReconcile the target Action through its status and events. Do not create a new\nAction merely because the client lost the stream.\n\nDiagnose timeout and cancellation\n\nSidecar HTTP 504 means the blocking route exceeded its wait duration. HTTP\n502 means the durable job reached failed state before the blocking response.\n\nBoth map to a temporary connector category for invocation, with uncertainty\nset for 5xx. The connector still sets retryable to false.\n\nFor cancelledunconfirmed, preserve:\n• whether an HTTP response arrived before local cancellation;\n• whether the operation used native async work or a worker thread;\n• the sidecar record and terminal event;\n• model, tool, training, or evaluator effects;\n• any artifact created after the cancellation timestamp.\n\nReplay, train, test, memory reset, and fallback paths can continue in worker\nthreads after their asyncio waiter ends. A successful cancel route does not\nprove those calls stopped.\n\nDiagnose terminal provider failure\n\nA sidecar execution exception appends a failed event with:\n• status: failed;\n• Python exception class in errortype;\n• a generic message directing the operator to logs by Action ID.\n\nThe blocking route returns HTTP 502. A stream returns a failed AIP result\nwith code connector.crewai.executionfailed and the event details.\n\nReview restricted sidecar logs for the matching Action ID. The log includes the\nprovider exception and traceback, so treat it as confidential.\n\nDo not infer retry safety from the exception class alone. First determine\nwhether models, tools, memory, training artifacts, or downstream systems\nreceived effects.\n\nEscalate with complete evidence\n\nEscalate when identity cannot be reconciled, the journal is corrupted, a\nthread-backed call continues after containment, a stream violates sequence, or\nprovider effects remain unknown.\n\nInclude:\n• the incident identity captured at the start;\n• exact AIP and CrewAI revisions;\n• redacted host readiness and sidecar health;\n• relevant metric deltas;\n• sidecar record and event slice;\n• restricted log location, not copied secret-bearing lines;\n• provider and downstream evidence;\n• containment and recovery decisions;\n• a clear statement of what remains unproved.\n\nRelated documentation\n• CrewAI health, observability, and recovery (../operations/health-observability-and-recovery.md)\n• CrewAI sidecar contract and lockfile (../reference/sidecar-contract-and-lockfile.md)\n• CrewAI idempotency, cancellation, and process fencing (../reference/idempotency-cancellation-and-process-fencing.md)\n• CrewAI capability index (../capabilities/README.md)\n• Errors and retry decisions (../../../reference/errors.md)\n"
  },
  "integrity": {
    "algorithm": "sha256",
    "sourceDigest": "19c67f04cc1d20150f64575e15c92ab83cb1b666b7932e0cf18d51ad42ae19e4"
  }
}