Scaling Enterprise AI: Centralizing Data Access Through Model Context Protocol on AWS Fargate
Organizations building applications powered by artificial intelligence face a recurring, systemic challenge: providing large language model (LLM) agents with seamless, secure, and performant access to enterprise data lakes without the burden of duplicating integration logic across every individual agent. Traditionally, development teams have resorted to wiring agents directly to databases and application programming interfaces (APIs), a fragmented approach that lacks a shared gateway. This "spaghetti" integration method necessitates custom code for every new dataset and redundant implementation of access control policies. As the number of datasets and agents scales within an enterprise, permission enforcement becomes inconsistent, and onboarding a single new data source can balloon into a multi-day engineering effort, stalling productivity and increasing technical debt.
To resolve these inefficiencies, the engineering team supporting Amazon’s North America Sort Center (NASC) network has developed a serverless Model Context Protocol (MCP) server deployed on Amazon Elastic Container Service (Amazon ECS) using AWS Fargate. This solution serves as a centralized, agent-agnostic gateway between generative AI agents and the enterprise data lake. By moving from a code-heavy integration model to a metadata-driven architecture, the team has fundamentally altered how data is exposed to conversational interfaces, reducing onboarding times from days to a simple JSON file update.
The Challenge of Fragmented Integration at Scale
The NASC network represents a complex operational environment where planning teams rely on high-velocity data to manage capacity, labor allocation, and package flow. With thousands of datasets spanning operational metrics, business analytics, and domain-specific information, the demand for conversational AI assistants to query this data has risen sharply. However, the initial state of development was characterized by silos. Each team tasked with building an AI agent would effectively reinvent the wheel, embedding SQL generation logic, schema knowledge, and connection handling directly into the agent’s code.
This lack of standardization meant that even minor schema changes to an underlying database required patching every individual agent, creating an unsustainable maintenance loop. Furthermore, access control was fractured across multiple services, leaving the organization without a single point of audit for data access decisions. Security teams, tasked with maintaining strict governance, faced difficulty tracking who was accessing what data when every agent managed its own authentication and query validation.
Architectural Transformation and Design Goals
The design of the new system was predicated on four core requirements: the implementation of an agent-agnostic gateway, the enforcement of centralized security, a metadata-driven onboarding process, and full serverless scalability. By decoupling the tool-calling capability from the agent logic, the architecture allows for independent scaling of each layer.
The architecture is divided into four distinct layers: the client layer, the MCP server layer, the tools layer, and the knowledge-base refresh layer. This modularity ensures that the system is not only resilient but also adaptable. The client layer, which interfaces with users via web chat or IDE plugins, handles authentication through Amazon Cognito and session management via Amazon DynamoDB. The MCP server layer, powered by the FastMCP Python framework, acts as the orchestrator. It manages query analysis, tool selection, and response generation, ensuring that every request is routed through a secure, internal, and controlled path.
The Metadata-Driven Paradigm
The most significant innovation in this architecture is the transition to a metadata-driven data access model. Instead of hardcoding table structures into the agent, the system maintains a central catalog where schema descriptions, column-level semantics, and table relationships are stored. When an agent needs to query a dataset, it retrieves the necessary context from this catalog at query time.
This catalog is refreshed automatically through a pipeline that ingests metadata JSON files. An engineer simply uploads a file defining the table’s purpose, business context, and column details—such as unique identifiers or timestamp formats—and the system indexes this information using Amazon Titan Text Embeddings. Once indexed in Amazon Bedrock Knowledge Bases, the data becomes immediately discoverable by all connected agents. This eliminates the need for redeploying agent code when the database schema evolves, effectively treating data integration as a configuration task rather than a software development project.
Security, Compliance, and Responsible AI
A critical focus of the NASC project was ensuring that centralized access did not compromise security. The system employs a "defense-in-depth" strategy. Because all tools are registered on the central MCP server, every interaction is subject to the same guardrails. Amazon Bedrock Guardrails are applied to all model interactions, filtering out harmful content and blocking potential prompt injection attacks.
Furthermore, the system enforces a strict "read-only" policy at the tool level. The Athena query tools are restricted to SELECT operations, meaning that even if an LLM were coerced into attempting a write operation, the request would be blocked before reaching the data lake. Because all traffic flows through an internal Application Load Balancer in a private subnet, the system remains isolated from the public internet. Every invocation is logged in structured JSON to Amazon CloudWatch, providing a comprehensive audit trail that meets the high compliance standards required for large-scale logistics operations.
Scalability in a Multi-Tenant Environment
As an enterprise-wide solution, the serverless nature of the architecture is vital. By leveraging AWS Fargate, the system removes the need for managing underlying virtual machine infrastructure. The client and MCP server layers utilize ECS Service Auto Scaling, which monitors CPU and memory utilization to dynamically adjust capacity. Whether the system is handling a single query from a planning analyst or hundreds of concurrent requests during peak operational hours, the infrastructure scales automatically.
The multi-tenant nature of the server is supported by the standardized /mcp endpoint. This allows various teams to build their own specialized agents using different frameworks while all tapping into the same secure, pre-validated toolset. By standardizing the interface, the organization has effectively commoditized data access for AI, allowing new teams to plug into the system without requiring bespoke engineering support.
Operational Implications and Future Outlook
The deployment of this centralized MCP server signals a shift in how large organizations approach the "agentic" era of computing. By moving the complexity of data integration into a specialized, middle-tier server, the NASC team has effectively commoditized access to enterprise data. This architectural pattern reduces the risk of "shadow AI," where developers bypass standard protocols to get data quickly, and instead encourages a secure, unified approach.
Looking forward, the team is working on two key advancements. The first is a more granular, attribute-based access control (ABAC) system. By allowing data owners to mark sensitive columns within the metadata catalog, the system will be able to automatically exclude restricted information from generated SQL, ensuring that users only see the data they are authorized to view, regardless of the agent they use.
The second advancement involves closing the operational feedback loop. The system is being enhanced to detect query failures and low-confidence responses. By capturing these patterns, the architecture will eventually be able to provide automated suggestions for improving metadata, such as adding specific query hints for complex tables or refining column descriptions.
This initiative represents a maturing phase in enterprise AI deployment. It demonstrates that the path to reliable AI in a corporate setting is not found in the sophistication of the models themselves, but in the robustness and cleanliness of the infrastructure that connects them to the organization’s most valuable asset: its data. By standardizing the interface between intelligence and information, organizations can ensure that their AI agents remain performant, secure, and truly scalable.
