> ## Documentation Index
> Fetch the complete documentation index at: https://agenticbanking.backbase.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Build an intelligent document processing agent

> Connect an agent to the Document Retrieval Model Context Protocol (MCP) server, add template instructions, and send documents.

This guide shows you how to build an intelligent document processing (IDP) agent that classifies a banking document and extracts a requested value. The template on this page returns one value. To extract several values in one pass, change the template instructions. The agent reads the document through the Document Retrieval Model Context Protocol (MCP) server, then classifies the document and extracts the value from that text. You set the JSON the agent returns in the template instructions. See [Add the template instructions](#add-the-template-instructions).

## Prerequisites

* **Subscription key** for the Document Retrieval MCP API. To get a key, contact your Backbase support team.
* **AI gateway access** for the language model that powers your agent. See [AI gateway](https://agenticbanking.backbase.io/agentic-ai/bb-ai-sdk/ai-gateway.md).
* **Document URLs** that the optical character recognition (OCR) service can download. See [Send documents to the agent](#send-documents-to-the-agent).

## Connect the agent to the MCP server

Add the Document Retrieval MCP server to your agent as a tool source, at `{baseURL}/document-retrieval-mcp/v0/mcp` with your subscription key in the `api-key` header. The server exposes two tools: `convert_file_to_markdown` for PDFs and images, and `fetch_text_content` for text files. For client setup, see [Connect an MCP client](https://agenticbanking.backbase.io/agentic-ai/foundation-services/intelligent-document-processing/use-the-services.md#connect-an-mcp-client).

Set the tool call timeout high enough for long documents, because OCR on a large PDF can take minutes.

## Add the template instructions

The following template covers one use case: **classify a banking document and extract a single requested value from it**. For example, the agent returns the registration number from a Dutch Chamber of Commerce extract (Kamer van Koophandel, KvK). The agent can also return the net pay from a payslip, or total assets from a balance sheet. The template returns the value exactly as printed, as a JSON object with the document type and the value.

```markdown theme={"system"}
You are a banking and fintech document extraction assistant.

## Input
A JSON object with the user's query and one or more document URLs in "assets".

## Tools
- Use convert_file_to_markdown for PDFs and images (.pdf, .jpg, .jpeg, .png, .tiff).
- Use fetch_text_content for text files (.txt, .json, .csv, .md, .xml).
- Use mode "default". Use mode "premium" only if the user asks for it, or if the
  default result is clearly missing content such as tables or charts.
- Read every document with a tool. Never answer from memory.

## Tasks
1. Classify the document. Possible types: kvk_extract, payslips, balance_sheets,
   ids, other. Use structural cues to choose the type, such as headings like
   "Assets", payroll terms like "Net Pay", or registry fields like "KvK".
2. Extract only the exact value that the user's query asks for.

## Extraction rules
- Return only the requested value, with no labels, commentary, or other text.
- Currency: Keep the currency symbol or code exactly as printed (€, £, $, USD,
  EUR, GBP). If no currency is printed, infer it from context such as the address
  or country, if possible.
- Scale: If a header or note states a scale ("in thousands", "£'000",
  "in millions", "$m"), multiply by 1,000 or 1,000,000 and apply the currency
  consistently to all values in that scope.
- Number format: Use the document's number format. Keep the original decimal and
  thousands separators. Never swap commas and periods, and never add or remove
  separators.
- Units and percentages: Include units or "%" only if the document shows them.
- Spacing: Keep spacing as printed, for example "£505,00" versus "£ 505,00".
  Don't truncate the value or add characters or spaces.

## Output
Return only a JSON object with exactly two keys:
{
  "document_type": "<kvk_extract|payslips|balance_sheets|ids|other>",
  "extracted_value": "<the exact extracted value or NOT_FOUND>"
}

## Checks before you answer
All checks must pass. If one fails, revise your answer once.
- Currency: no symbol or code switching, and no missed or invented currency.
- Separators: valid for the document's style, with no comma and period swaps.
- Scale: applied only if the document states it, and never applied twice.
- Units and "%": only if printed, with no added spaces, labels, or wrappers.

## Formatting examples
Document value "22,00"      -> extracted_value "22,00"      (not "22")
Document value "1.588,17"   -> extracted_value "1.588,17"   (not "1.588.17")
Document value "£3,000.00"  -> extracted_value "£3,000.00"  (not "£3,000.0")
Document value "£103,00"    -> extracted_value "£103,00"    (not "£ 103,00")
Document value "€303,83"    -> extracted_value "€303,83"    (not "€303.83")
Document value "€1.756,20"  -> extracted_value "€1.756,20"  (not "€1.756.20")
```

To use the template, replace the document types with the types that your process expects, and keep `other` as the last type.

> [!NOTE]
>
> Other use cases need different instructions. For example, extracting several fields at once, summarizing a document, comparing two documents, or answering open questions about a document each need their own tasks and output format. You can reuse the **Tools** section of this template in any instructions that use the Document Retrieval MCP server.

When the agent can't extract a value, add a third key such as `error_message` to the output. Tell the agent when to fill that key. For example: "If the document isn't the type the user asked about, set `extracted_value` to `NOT_FOUND` and explain why in `error_message`."

## Send documents to the agent

The agent reads documents from URLs, so your app stores the uploaded file first and then passes a URL to the agent. Use one of these options:

* **Signed URLs from Document Storage**: Store the file with Backbase Document Storage and pass a time-limited signed URL. See [Use signed URLs](https://backbase.io/documentation/foundation-services/2025.09-LTS/doc-storage/developer-guides/use-signed-urls).
* **Azure Blob Storage shared access signature (SAS) URLs**: Pass a URL with a SAS token.
* **Allowed Blob Storage containers**: Pass a plain blob URL for containers that your installation allows. The OCR service reads these with its managed identity.

Send the query and URLs as a JSON message:

```json theme={"system"}
{
  "query": "What is the total net pay?",
  "assets": [
    { "url": "https://<your-storage>/payslip-2026-09.pdf?<sas-token>" }
  ]
}
```

> [!WARNING]
>
> Documents often contain personal data. Use short-lived signed URLs, and don't log full document URLs that contain access tokens.

## Handle the result

The agent returns a JSON object like the following:

```json theme={"system"}
{
  "document_type": "kvk_extract",
  "extracted_value": "27002562"
}
```

Before you use the value, check the result in your app:

* If `document_type` isn't the type you expected, ask the customer for the right document.
* If `extracted_value` is `NOT_FOUND`, route the document to a person for review.
* If the response isn't valid JSON, retry once, and then route the document for review.

## Related topics

* [Overview](https://agenticbanking.backbase.io/agentic-ai/foundation-services/intelligent-document-processing/overview.md) for how IDP works and when to use it.
* [Use the OCR API and MCP server](https://agenticbanking.backbase.io/agentic-ai/foundation-services/intelligent-document-processing/use-the-services.md) to call the services without an agent.
* [Reference](https://agenticbanking.backbase.io/agentic-ai/foundation-services/intelligent-document-processing/reference.md) for supported formats, document sources, limits, and error codes.