{ "schemaVersion": "1.0", "title": "Manage Dify knowledge", "description": "Use this procedure to create or select one Dify knowledge base, ingest an approved source, wait for indexing, enrich its records, and verify retrieval. An optional final branch updates or removes provider state.", "canonical": "https://getaip.org/docs/connectors/dify/guides/manage-knowledge", "route": "/docs/connectors/dify/guides/manage-knowledge", "source": "docs/connectors/dify/guides/manage-knowledge.md", "protocol": "Agent Interoperability Protocol", "protocolVersion": "1.0", "section": "Connectors", "documentType": "Connector", "language": "en", "downloads": { "md": "/docs/download/connectors/dify/guides/manage-knowledge.md", "txt": "/docs/download/connectors/dify/guides/manage-knowledge.txt", "json": "/docs/download/connectors/dify/guides/manage-knowledge.json", "pdf": "/docs/download/connectors/dify/guides/manage-knowledge.pdf" }, "content": { "format": "text/markdown", "markdown": "---\ntitle: Manage Dify knowledge\ndescription: >-\n Create or select one knowledge base, ingest approved content, settle\n indexing, enrich records, retrieve chunks, and maintain provider state\nkind: how-to\naudience: application-developer\nappliesTo: \"1.x\"\nwritingStandard: \"aip-docs/1.0\"\nlastReviewedRevision: \"97be86e9efedf07ecf1783b03800f683f107fb04\"\nconnector: dify\n---\n\n# Manage Dify knowledge\n\nUse this procedure to create or select one Dify knowledge base, ingest an\napproved source, wait for indexing, enrich its records, and verify retrieval.\nAn optional final branch updates or removes provider state.\n\nEach mutation uses its own Action ID, immutable input, approval, and\nidempotency key. A lost response never authorizes a replacement Action.\n\n## Before you begin\n\nObtain:\n\n- one authenticated tenant and admitted Dify connector release;\n- one discovered Knowledge capability prefix for the intended workspace;\n- authority to read model, dataset, document, and metadata inventory;\n- approved source content and its retention classification;\n- an approver for every non-GET operation;\n- a protected location for Action, provider, batch, and readback evidence.\n\nThe Knowledge prefix is:\n\n```text\ncap:dify:knowledge::\n```\n\nDo not use an app alias or app API key. The configured Knowledge credential\nselects the workspace and never belongs in Action input.\n\n## 1. Pin capability and workspace identity\n\nRead authenticated tenant discovery and record the exact Knowledge prefix,\nconnector revision, credential ID, and required operations. Stop if any\ncapability is absent.\n\nRecord the credential revision without copying secret bytes. Dataset,\ndocument, segment, metadata, upload, and model IDs remain valid only inside\nthat workspace boundary.\n\nFor every mutation, create one Action ID and one tenant-scoped idempotency key\nbefore approval. Keep the same Action, key, and canonical input until its\noutcome is settled.\n\n## 2. Discover models and existing datasets\n\nRead available embedding or reranking models with `model.list`. Then call\n`dataset.list` with a narrow keyword or tag filter.\n\n```json\n{\n \"query\": {\n \"page\": 1,\n \"limit\": 20,\n \"keyword\": \"customer support\"\n }\n}\n```\n\nUse an existing dataset only after `dataset.get` confirms its workspace,\nprovider, permission, indexing technique, models, and retrieval settings.\nName similarity alone is insufficient.\n\nModel discovery proves current visibility, not quota or future availability.\nRetain the exact provider and model identifiers selected for indexing.\n\n## 3. Create a dataset only when needed\n\nSkip this step when an approved dataset already exists.\n\nPrepare the smallest creation body that fixes ownership and indexing intent:\n\n```json\n{\n \"body\": {\n \"name\": \"Customer support knowledge\",\n \"description\": \"Reviewed support policy and product facts.\",\n \"provider\": \"vendor\",\n \"permission\": \"partial_members\",\n \"indexing_technique\": \"high_quality\",\n \"embedding_model_provider\": \"provider-name\",\n \"embedding_model\": \"embedding-model-name\"\n }\n}\n```\n\nApprove the exact name, permission, provider class, models, retrieval settings,\nand summary configuration. Successful creation returns an empty dataset, not\nindexed knowledge.\n\nIf the response is lost, list by the exact name and compare full settings.\nDo not create a second dataset merely because the first Action is uncertain.\n\n## 4. Choose one ingestion branch\n\nChoose exactly one branch for each approved source:\n\n| Source | Initial capability | Completion evidence |\n|---|---|---|\n| Authorized raw text | `document.text.create` | Returned document and batch, then terminal indexing status |\n| Authorized local file | `document.file.create` | Returned document and batch, then terminal indexing status |\n| Pipeline local file | `pipeline.file.upload`, then `pipeline.run` | Upload ID, run result, then authoritative document reads |\n| Pipeline remote source | `pipeline.run` | Run result, then authoritative document reads |\n\nDirect file ingestion is unavailable for an external dataset. Pipeline upload\ncreates a workspace file first; it does not attach that file to a dataset by\nitself.\n\nDo not combine branches to recover a lost response. Inspect provider state and\nthe retained IDs from the original branch.\n\n## 5. Ingest text or a file directly\n\nFor text, supply name, content, document form, language, and the intended\nindexing technique:\n\n```json\n{\n \"dataset_id\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"body\": {\n \"name\": \"Refund policy\",\n \"text\": \"Approved customers may request a refund within thirty days.\",\n \"doc_form\": \"text_model\",\n \"doc_language\": \"English\",\n \"indexing_technique\": \"high_quality\"\n }\n}\n```\n\nFor a file, provide one safe filename, MIME type, base64 content, and serialized\nprocessing settings. The connector caps decoded input at 32 MiB; provider\ntype, quota, and upload limits can be stricter.\n\nThe first document needs an indexing technique when the dataset does not have\none. Later documents inherit the dataset setting.\n\nBoth operations return `document` and `batch`. Persist both immediately.\nTheir HTTP completion acknowledges ingestion but does not complete indexing.\n\n## 6. Run a knowledge pipeline when selected\n\nFor a local pipeline source, upload the file and retain its returned ID. Use\nthat ID as `reference` in `datasource_info_list`.\n\nRun the published pipeline in blocking mode for a structured connector result:\n\n```json\n{\n \"dataset_id\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"body\": {\n \"inputs\": {},\n \"datasource_type\": \"local_file\",\n \"datasource_info_list\": [\n {\n \"reference\": \"upload-0190\",\n \"name\": \"refund-policy.pdf\"\n }\n ],\n \"start_node_id\": \"datasource-node-1\",\n \"is_published\": true,\n \"response_mode\": \"blocking\"\n }\n}\n```\n\nDatasource discovery and execution must use the same published or draft\nselection. Verify every remote URL, workspace, page, drive, bucket, plugin,\nand credential before approval.\n\nAfter the run, list and read generated documents. Provider pipeline completion\ndoes not replace document and indexing evidence.\n\n## 7. Settle indexing\n\nFor direct ingestion, poll `document.indexing_status` with the returned batch\nand same dataset:\n\n```json\n{\n \"dataset_id\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"batch\": \"batch-0190c42f\"\n}\n```\n\nNormal progress is `waiting`, `parsing`, `cleaning`, `splitting`, `indexing`,\nthen `completed`. Preserve paused, stopped, or error evidence instead of\nreporting success.\n\nRequire a terminal state for every intended document. The connector exposes no\nprovider indexing cancellation operation.\n\nRead each document after completion. Confirm dataset, source type, name,\ndisplay status, form, segment count, metadata, error, and summary state needed\nby the task.\n\n## 8. Inspect chunks before enrichment\n\nUse `segment.list` to inspect representative content, answers, keywords,\nattachments, summaries, enabled state, and errors. Use `child_chunk.list` when\nthe document uses a parent-child structure.\n\nDo not modify generated chunks only to make a failed ingestion appear\ncomplete. Repair the source or processing configuration under a new approved\nintent.\n\nA segment update can return HTTP success with `data.status: error` after an\nindexing failure. Inspect returned status and perform a follow-up read.\n\n## 9. Apply tags and metadata\n\nUse tags for workspace organization and metadata for structured document\nattributes. Discover current identities before writing either.\n\nFor document metadata, use the exact field ID, name, and type returned by\n`metadata.list`. Set `partial_update: true` to preserve unspecified custom\nvalues.\n\n```json\n{\n \"dataset_id\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"body\": {\n \"operation_data\": [\n {\n \"document_id\": \"0190c42f-2d5a-7000-8000-000000001002\",\n \"metadata_list\": [\n {\n \"id\": \"0190c42f-2d5a-7000-8000-000000001003\",\n \"name\": \"approved_region\",\n \"value\": \"emea\"\n }\n ],\n \"partial_update\": true\n }\n ]\n }\n}\n```\n\nThe default false replaces custom metadata and bindings. Multi-document input\ncommits each document separately, so a later failure can leave earlier entries\nchanged.\n\nRead fields, tags, and every intended document after mutation. Provider success\nalone is insufficient for metadata helpers that can suppress internal errors.\n\n## 10. Verify governed retrieval\n\n`dataset.retrieve` performs a read-oriented provider search, but the connector\nclassifies its POST route as a medium mutation. It requires approval and an\nidempotency key and publishes no safe retry.\n\nUse one representative query of at most 250 characters. Bind approval to the\ndataset, query, model settings, attachments, expected result count, and data\nhandling.\n\nVerify that returned segments belong to the intended dataset and documents.\nRecord scores and fields needed for acceptance without retaining unrelated\ncontent.\n\n## 11. Maintain or remove state\n\nUse partial dataset update for approved configuration changes. Re-read the\ncomplete dataset after changing permissions, models, retrieval, or external\nknowledge settings.\n\nDocument replacement returns a new batch and must repeat indexing settlement.\nStatus actions require readback of every changed document.\n\nDelete only after dependencies and retention duties are cleared. Document\ndelete removes all chunks and is blocked during indexing or while archived.\nDataset delete removes all documents and is blocked while an app uses it.\n\nBoth deletes are high risk, unsafe to retry, and have no rollback. Confirm\nabsence through authoritative reads after settlement.\n\n## Recover uncertain work\n\n| Observation | Read first | Decision |\n|---|---|---|\n| Dataset create response lost | Dataset list and matching detail | Reuse the confirmed dataset or escalate |\n| Direct ingestion response lost | Document list, source identity, and recent batches | Do not ingest the source again blindly |\n| Pipeline response lost | Upload inventory, pipeline evidence, and document list | Reconcile generated state before another run |\n| Indexing pauses or fails | Batch status and document detail | Repair the cause under a new approved intent |\n| Metadata batch fails partway | Every intended document | Reconcile only the divergent entries |\n| Retrieval response is lost | Original Action state and provider evidence | Do not repeat automatically |\n| Delete response is lost | Exact dataset or document read | Accept confirmed absence or escalate |\n| Data crosses a workspace boundary | Credential, route chain, and audit evidence | Stop traffic and treat it as an isolation incident |\n\n## Completion checklist\n\n- Capability and credential identity came from authenticated discovery.\n- Model and dataset settings matched the reviewed intent.\n- Exactly one ingestion branch owned each source.\n- Document and batch identities were retained before polling.\n- Every intended document reached a verified terminal indexing state.\n- Tags, metadata, and retrieval results passed authoritative readback.\n- Each mutation kept its original Action, input, approval, and key.\n- Sensitive content and identifiers were redacted from general logs.\n- Any destructive change passed dependency and retention checks.\n\n## Related documentation\n\n- [Dify knowledge bases, tags, and retrieval](../capabilities/knowledge-datasets-tags-and-retrieval.md)\n- [Dify knowledge documents and indexing](../capabilities/knowledge-documents-and-indexing.md)\n- [Dify segments and child chunks](../capabilities/knowledge-segments-and-child-chunks.md)\n- [Dify models and knowledge metadata](../capabilities/knowledge-metadata-and-models.md)\n- [Dify knowledge pipelines and uploads](../capabilities/knowledge-pipelines-and-uploads.md)\n", "text": "Manage Dify knowledge\n\nUse this procedure to create or select one Dify knowledge base, ingest an\napproved source, wait for indexing, enrich its records, and verify retrieval.\nAn optional final branch updates or removes provider state.\n\nEach mutation uses its own Action ID, immutable input, approval, and\nidempotency key. A lost response never authorizes a replacement Action.\n\nBefore you begin\n\nObtain:\n• one authenticated tenant and admitted Dify connector release;\n• one discovered Knowledge capability prefix for the intended workspace;\n• authority to read model, dataset, document, and metadata inventory;\n• approved source content and its retention classification;\n• an approver for every non-GET operation;\n• a protected location for Action, provider, batch, and readback evidence.\n\nThe Knowledge prefix is:\n\ncap:dify:knowledge::\n\nDo not use an app alias or app API key. The configured Knowledge credential\nselects the workspace and never belongs in Action input.\n1. Pin capability and workspace identity\n\nRead authenticated tenant discovery and record the exact Knowledge prefix,\nconnector revision, credential ID, and required operations. Stop if any\ncapability is absent.\n\nRecord the credential revision without copying secret bytes. Dataset,\ndocument, segment, metadata, upload, and model IDs remain valid only inside\nthat workspace boundary.\n\nFor every mutation, create one Action ID and one tenant-scoped idempotency key\nbefore approval. Keep the same Action, key, and canonical input until its\noutcome is settled.\n2. Discover models and existing datasets\n\nRead available embedding or reranking models with model.list. Then call\ndataset.list with a narrow keyword or tag filter.\n\n{\n \"query\": {\n \"page\": 1,\n \"limit\": 20,\n \"keyword\": \"customer support\"\n }\n}\n\nUse an existing dataset only after dataset.get confirms its workspace,\nprovider, permission, indexing technique, models, and retrieval settings.\nName similarity alone is insufficient.\n\nModel discovery proves current visibility, not quota or future availability.\nRetain the exact provider and model identifiers selected for indexing.\n3. Create a dataset only when needed\n\nSkip this step when an approved dataset already exists.\n\nPrepare the smallest creation body that fixes ownership and indexing intent:\n\n{\n \"body\": {\n \"name\": \"Customer support knowledge\",\n \"description\": \"Reviewed support policy and product facts.\",\n \"provider\": \"vendor\",\n \"permission\": \"partialmembers\",\n \"indexingtechnique\": \"highquality\",\n \"embeddingmodelprovider\": \"provider-name\",\n \"embeddingmodel\": \"embedding-model-name\"\n }\n}\n\nApprove the exact name, permission, provider class, models, retrieval settings,\nand summary configuration. Successful creation returns an empty dataset, not\nindexed knowledge.\n\nIf the response is lost, list by the exact name and compare full settings.\nDo not create a second dataset merely because the first Action is uncertain.\n4. Choose one ingestion branch\n\nChoose exactly one branch for each approved source:\n\n| Source | Initial capability | Completion evidence |\n\n| Authorized raw text | document.text.create | Returned document and batch, then terminal indexing status |\n| Authorized local file | document.file.create | Returned document and batch, then terminal indexing status |\n| Pipeline local file | pipeline.file.upload, then pipeline.run | Upload ID, run result, then authoritative document reads |\n| Pipeline remote source | pipeline.run | Run result, then authoritative document reads |\n\nDirect file ingestion is unavailable for an external dataset. Pipeline upload\ncreates a workspace file first; it does not attach that file to a dataset by\nitself.\n\nDo not combine branches to recover a lost response. Inspect provider state and\nthe retained IDs from the original branch.\n5. Ingest text or a file directly\n\nFor text, supply name, content, document form, language, and the intended\nindexing technique:\n\n{\n \"datasetid\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"body\": {\n \"name\": \"Refund policy\",\n \"text\": \"Approved customers may request a refund within thirty days.\",\n \"docform\": \"textmodel\",\n \"doclanguage\": \"English\",\n \"indexingtechnique\": \"highquality\"\n }\n}\n\nFor a file, provide one safe filename, MIME type, base64 content, and serialized\nprocessing settings. The connector caps decoded input at 32 MiB; provider\ntype, quota, and upload limits can be stricter.\n\nThe first document needs an indexing technique when the dataset does not have\none. Later documents inherit the dataset setting.\n\nBoth operations return document and batch. Persist both immediately.\nTheir HTTP completion acknowledges ingestion but does not complete indexing.\n6. Run a knowledge pipeline when selected\n\nFor a local pipeline source, upload the file and retain its returned ID. Use\nthat ID as reference in datasourceinfolist.\n\nRun the published pipeline in blocking mode for a structured connector result:\n\n{\n \"datasetid\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"body\": {\n \"inputs\": {},\n \"datasourcetype\": \"localfile\",\n \"datasourceinfolist\": [\n {\n \"reference\": \"upload-0190\",\n \"name\": \"refund-policy.pdf\"\n }\n ],\n \"startnodeid\": \"datasource-node-1\",\n \"ispublished\": true,\n \"responsemode\": \"blocking\"\n }\n}\n\nDatasource discovery and execution must use the same published or draft\nselection. Verify every remote URL, workspace, page, drive, bucket, plugin,\nand credential before approval.\n\nAfter the run, list and read generated documents. Provider pipeline completion\ndoes not replace document and indexing evidence.\n7. Settle indexing\n\nFor direct ingestion, poll document.indexingstatus with the returned batch\nand same dataset:\n\n{\n \"datasetid\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"batch\": \"batch-0190c42f\"\n}\n\nNormal progress is waiting, parsing, cleaning, splitting, indexing,\nthen completed. Preserve paused, stopped, or error evidence instead of\nreporting success.\n\nRequire a terminal state for every intended document. The connector exposes no\nprovider indexing cancellation operation.\n\nRead each document after completion. Confirm dataset, source type, name,\ndisplay status, form, segment count, metadata, error, and summary state needed\nby the task.\n8. Inspect chunks before enrichment\n\nUse segment.list to inspect representative content, answers, keywords,\nattachments, summaries, enabled state, and errors. Use childchunk.list when\nthe document uses a parent-child structure.\n\nDo not modify generated chunks only to make a failed ingestion appear\ncomplete. Repair the source or processing configuration under a new approved\nintent.\n\nA segment update can return HTTP success with data.status: error after an\nindexing failure. Inspect returned status and perform a follow-up read.\n9. Apply tags and metadata\n\nUse tags for workspace organization and metadata for structured document\nattributes. Discover current identities before writing either.\n\nFor document metadata, use the exact field ID, name, and type returned by\nmetadata.list. Set partialupdate: true to preserve unspecified custom\nvalues.\n\n{\n \"datasetid\": \"0190c42f-2d5a-7000-8000-000000001001\",\n \"body\": {\n \"operationdata\": [\n {\n \"documentid\": \"0190c42f-2d5a-7000-8000-000000001002\",\n \"metadatalist\": [\n {\n \"id\": \"0190c42f-2d5a-7000-8000-000000001003\",\n \"name\": \"approvedregion\",\n \"value\": \"emea\"\n }\n ],\n \"partialupdate\": true\n }\n ]\n }\n}\n\nThe default false replaces custom metadata and bindings. Multi-document input\ncommits each document separately, so a later failure can leave earlier entries\nchanged.\n\nRead fields, tags, and every intended document after mutation. Provider success\nalone is insufficient for metadata helpers that can suppress internal errors.\n10. Verify governed retrieval\n\ndataset.retrieve performs a read-oriented provider search, but the connector\nclassifies its POST route as a medium mutation. It requires approval and an\nidempotency key and publishes no safe retry.\n\nUse one representative query of at most 250 characters. Bind approval to the\ndataset, query, model settings, attachments, expected result count, and data\nhandling.\n\nVerify that returned segments belong to the intended dataset and documents.\nRecord scores and fields needed for acceptance without retaining unrelated\ncontent.\n11. Maintain or remove state\n\nUse partial dataset update for approved configuration changes. Re-read the\ncomplete dataset after changing permissions, models, retrieval, or external\nknowledge settings.\n\nDocument replacement returns a new batch and must repeat indexing settlement.\nStatus actions require readback of every changed document.\n\nDelete only after dependencies and retention duties are cleared. Document\ndelete removes all chunks and is blocked during indexing or while archived.\nDataset delete removes all documents and is blocked while an app uses it.\n\nBoth deletes are high risk, unsafe to retry, and have no rollback. Confirm\nabsence through authoritative reads after settlement.\n\nRecover uncertain work\n\n| Observation | Read first | Decision |\n\n| Dataset create response lost | Dataset list and matching detail | Reuse the confirmed dataset or escalate |\n| Direct ingestion response lost | Document list, source identity, and recent batches | Do not ingest the source again blindly |\n| Pipeline response lost | Upload inventory, pipeline evidence, and document list | Reconcile generated state before another run |\n| Indexing pauses or fails | Batch status and document detail | Repair the cause under a new approved intent |\n| Metadata batch fails partway | Every intended document | Reconcile only the divergent entries |\n| Retrieval response is lost | Original Action state and provider evidence | Do not repeat automatically |\n| Delete response is lost | Exact dataset or document read | Accept confirmed absence or escalate |\n| Data crosses a workspace boundary | Credential, route chain, and audit evidence | Stop traffic and treat it as an isolation incident |\n\nCompletion checklist\n• Capability and credential identity came from authenticated discovery.\n• Model and dataset settings matched the reviewed intent.\n• Exactly one ingestion branch owned each source.\n• Document and batch identities were retained before polling.\n• Every intended document reached a verified terminal indexing state.\n• Tags, metadata, and retrieval results passed authoritative readback.\n• Each mutation kept its original Action, input, approval, and key.\n• Sensitive content and identifiers were redacted from general logs.\n• Any destructive change passed dependency and retention checks.\n\nRelated documentation\n• Dify knowledge bases, tags, and retrieval (../capabilities/knowledge-datasets-tags-and-retrieval.md)\n• Dify knowledge documents and indexing (../capabilities/knowledge-documents-and-indexing.md)\n• Dify segments and child chunks (../capabilities/knowledge-segments-and-child-chunks.md)\n• Dify models and knowledge metadata (../capabilities/knowledge-metadata-and-models.md)\n• Dify knowledge pipelines and uploads (../capabilities/knowledge-pipelines-and-uploads.md)\n" }, "integrity": { "algorithm": "sha256", "sourceDigest": "ec847eb5cb5f1c8ef5f36c139d652655f5985d318a77b2ba4a1d880c22fbc4d6" } }