Skip to main content
This guide shows you how to build an intelligent document processing (IDP) agent that classifies a banking document and extracts a requested value. The template on this page returns one value. To extract several values in one pass, change the template instructions. The agent reads the document through the Document Retrieval Model Context Protocol (MCP) server, then classifies the document and extracts the value from that text. You set the JSON the agent returns in the template instructions. See Add the template instructions.

Prerequisites

  • Subscription key for the Document Retrieval MCP API. To get a key, contact your Backbase support team.
  • AI gateway access for the language model that powers your agent. See AI gateway.
  • Document URLs that the optical character recognition (OCR) service can download. See Send documents to the agent.

Connect the agent to the MCP server

Add the Document Retrieval MCP server to your agent as a tool source, at {baseURL}/document-retrieval-mcp/v0/mcp with your subscription key in the api-key header. The server exposes two tools: convert_file_to_markdown for PDFs and images, and fetch_text_content for text files. For client setup, see Connect an MCP client. Set the tool call timeout high enough for long documents, because OCR on a large PDF can take minutes.

Add the template instructions

The following template covers one use case: classify a banking document and extract a single requested value from it. For example, the agent returns the registration number from a Dutch Chamber of Commerce extract (Kamer van Koophandel, KvK). The agent can also return the net pay from a payslip, or total assets from a balance sheet. The template returns the value exactly as printed, as a JSON object with the document type and the value.
To use the template, replace the document types with the types that your process expects, and keep other as the last type.
Other use cases need different instructions. For example, extracting several fields at once, summarizing a document, comparing two documents, or answering open questions about a document each need their own tasks and output format. You can reuse the Tools section of this template in any instructions that use the Document Retrieval MCP server.
When the agent can’t extract a value, add a third key such as error_message to the output. Tell the agent when to fill that key. For example: “If the document isn’t the type the user asked about, set extracted_value to NOT_FOUND and explain why in error_message.”

Send documents to the agent

The agent reads documents from URLs, so your app stores the uploaded file first and then passes a URL to the agent. Use one of these options:
  • Signed URLs from Document Storage: Store the file with Backbase Document Storage and pass a time-limited signed URL. See Use signed URLs.
  • Azure Blob Storage shared access signature (SAS) URLs: Pass a URL with a SAS token.
  • Allowed Blob Storage containers: Pass a plain blob URL for containers that your installation allows. The OCR service reads these with its managed identity.
Send the query and URLs as a JSON message:
Documents often contain personal data. Use short-lived signed URLs, and don’t log full document URLs that contain access tokens.

Handle the result

The agent returns a JSON object like the following:
Before you use the value, check the result in your app:
  • If document_type isn’t the type you expected, ask the customer for the right document.
  • If extracted_value is NOT_FOUND, route the document to a person for review.
  • If the response isn’t valid JSON, retry once, and then route the document for review.