# Live Product End-to-End Tests

Contract tests and live product tests provide different evidence and are kept
separate:

- connector unit tests validate HTTP payloads, authentication headers,
  cancellation, streaming, webhook verification, and retry declarations
  against deterministic protocol peers;
- orchestration contract tests validate native AIP lifecycle behavior without
  claiming that an external product was deployed;
- the ignored live suite invokes real Dify, CrewAI sidecar, and Chatwoot
  endpoints through an AIP `Gateway` and performs customer-visible writes.

## Required Environment

The live suite requires dedicated non-production tenants and these variables:

| Variable | Purpose |
| --- | --- |
| `AIP_E2E_DIFY_URL` | Dify application API base URL |
| `AIP_E2E_DIFY_API_KEY` | Dify application API credential |
| `AIP_E2E_DIFY_APP_ID` | Dify application identifier exposed by the connector |
| `AIP_E2E_DIFY_APP_MODE` | Dify mode, such as `workflow` or `chat` |
| `AIP_E2E_DIFY_INPUT_JSON` | Optional complete Dify test input |
| `AIP_E2E_CREWAI_URL` | CrewAI sidecar base URL |
| `AIP_E2E_CREWAI_CREW_ID` | Crew identifier configured in the sidecar |
| `AIP_E2E_CREWAI_BEARER_TOKEN` | Optional sidecar bearer token; required when sidecar auth is enabled |
| `AIP_E2E_CHATWOOT_URL` | Chatwoot base URL |
| `AIP_E2E_CHATWOOT_ACCOUNT_ID` | Dedicated Chatwoot test account |
| `AIP_E2E_CHATWOOT_TOKEN` | Chatwoot API access token |
| `AIP_E2E_CHATWOOT_CONVERSATION_ID` | Disposable conversation receiving the private note |
| `AIP_E2E_CAL_DIY_URL` | Isolated Cal.diy API base URL, including `/api` when required |
| `AIP_E2E_CAL_DIY_API_KEY` | Dedicated Cal.diy API key or access token |
| `AIP_E2E_CAL_DIY_EVENT_TYPE_ID` | Event type used for reversible slot reservation |
| `AIP_E2E_CAL_DIY_SLOT_START` | Currently available future slot in ISO 8601 format |
| `AIP_E2E_CAL_DIY_SLOT_DURATION_MINUTES` | Optional slot duration, default 30 minutes |

Run only the ignored live test:

```sh
cargo test -p aip-tests --test e2e_chatwoot_dify_crewai \
  live_chatwoot_dify_crewai_workflow_runs_through_aip_gateway \
  -- --ignored --exact --nocapture
```

The test fails when a required variable is absent, a product health probe is
not ready, approval state is not persisted, or any product operation does not
complete. A normal workspace test run reports this test as ignored; it must not
be cited as live-product evidence unless the explicit command above completed
against identified product instances.

## Safety

Use disposable data. The suite invokes a Dify application, executes a CrewAI
crew, and posts a private Chatwoot message. Credentials must come from the
environment or a deployment secret store and must never be committed to the
repository or emitted in test logs.

## Exact-Upstream Cal.diy Qualification

The release qualification is a repository-owned, isolated Docker campaign. It
admits the exact clean Cal.diy source revision and matching image, creates fresh
dedicated AIP and Cal.diy databases, and exercises both native AIP and MCP
Streamable HTTP through `aipd`:

```sh
examples/cal-diy-qualification/qualify.sh
```

The driver fails unless it proves all of these boundaries: verified
tenant/account and opaque credential-handle routing; 81 admitted capabilities;
real slot discovery; plan validation without mutation; an independent
authenticated approval decision; exactly one booking after approval resume;
exact idempotent replay; signed webhook acceptance, duplicate suppression,
invalid-HMAC rejection, and timestamp-skew rejection; durable action,
transaction, event, and provider facts after `aipd` restart.

Runtime secrets and raw evidence are written only under the ignored
`examples/cal-diy-qualification/.runtime/` directory. The redacted retained
record is
[`cal-diy-isolated-live.md`](cal-diy-isolated-live.md), with its machine-readable
artifact linked from that report. A passing deterministic provider fixture or a
skipped ignored test is not live Cal.diy evidence.

The older environment-driven ignored test remains available as a narrow manual
probe for an operator-owned Cal.diy deployment. It verifies authenticated
reservation/replay/release behavior but does not replace the complete release
qualification:

```sh
AIP_E2E_CAL_DIY_URL=http://127.0.0.1:5555/api \
AIP_E2E_CAL_DIY_API_KEY="$CAL_DIY_TEST_API_KEY" \
AIP_E2E_CAL_DIY_EVENT_TYPE_ID=42 \
AIP_E2E_CAL_DIY_SLOT_START=2050-01-01T10:00:00Z \
cargo test -p aip-connector-cal-diy \
  live_cal_diy_reservation_round_trip_uses_the_aip_connector \
  -- --ignored --exact --nocapture
```

## Exact-Upstream CrewAI Qualification

The CrewAI connector can be qualified independently before the three-product
workflow is available. Build or install CrewAI from the exact revision in
`docs/connectors/upstream-baselines.md`, start the qualification sidecar, and
run:

```sh
AIP_E2E_CREWAI_URL=http://127.0.0.1:8090 \
AIP_E2E_CREWAI_CREW_ID=support-qualification \
AIP_E2E_CREWAI_BEARER_TOKEN="$AIP_CREWAI_BEARER_TOKEN" \
cargo test -p aip-connector-crewai \
  exact_upstream_sidecar_executes_a_real_crewai_crew \
  -- --ignored --nocapture
```

This test is product evidence only when the sidecar was built from the recorded
upstream revision. It asserts a real terminal `CrewOutput` and token usage over
the Rust connector's authenticated HTTP boundary. Unit tests remain the
authoritative deterministic checks for stream ordering, cancellation, and
restart fencing.