AtlasMind

AtlasMind is an AWS-based knowledge assistant for asking natural-language questions about business documents. It combines retrieval-augmented generation (RAG), structured entity extraction, SQL analysis, conversation memory, and an LLM agent that can choose the appropriate capability for a question.

Why AtlasMind?

Business knowledge is usually split between long documents and structured facts. A conventional chatbot may summarize a document well but struggle with questions that require filtering, aggregation, or calculation. A database can answer analytical questions precisely but does not understand the surrounding narrative in a report.

AtlasMind addresses both needs in one workflow:

• Document text and tables are embedded for semantic retrieval.
• Extracted entities are loaded into PostgreSQL for structured analysis.
• An agent can select document search, SQL, web search, or a calculator.
• Conversation history is persisted by session so follow-up questions retain context.

The repository uses company financial reports as its example domain, but the ingestion and agent components are intended to be adapted to other document collections.

What It Does

• Answers conversational questions with an Amazon Bedrock foundation model.
• Retrieves relevant document passages and tables using Amazon Titan Text Embeddings and Aurora PostgreSQL with pgvector.
• Converts analytical questions into PostgreSQL queries against the extracted_entities table.
• Uses a ReAct-style LangChain agent for multi-step questions.
• Provides web search through DuckDuckGo for current events, news, and people.
• Evaluates mathematical expressions with a restricted numexpr calculator.
• Stores chat history in DynamoDB by session ID.
• Provides a Cognito-authenticated Next.js chat application hosted through AWS Amplify.

Architecture

flowchart LR
		User --> UI[Next.js chat app]
		UI -->|Cognito ID token| API[API Gateway POST /]
		API --> Lambda[Lambda agent executor]
		Lambda --> Memory[DynamoDB chat history]
		Lambda --> Bedrock[Amazon Bedrock Claude]
		Lambda --> RAG[SemanticSearch]
		Lambda --> SQL[SQLQA]
		Lambda --> Search[DuckDuckGo Search]
		Lambda --> Calc[Calculator]
		RAG --> Vector[(Aurora PostgreSQL + pgvector)]
		SQL --> Entities[(extracted_entities)]
		SageMaker[SageMaker processing jobs] -->|embeddings and entities| Vector
		S3[S3 data bucket] --> SageMaker

Loading

Runtime request flow

1. A user signs up or signs in through Amazon Cognito in the Next.js application.
2. The frontend obtains the Cognito ID token and sends a POST request to API Gateway.
3. The request contains user_input, the Cognito user sub as session_id, and optionally clean_history.
4. API Gateway authorizes the request with a Cognito user-pool authorizer and invokes the Lambda container.
5. Lambda loads the Bedrock region, model ID, and Aurora credentials from Parameter Store and Secrets Manager.
6. In basic mode, a Bedrock conversation chain answers using the persisted DynamoDB history. In agentic mode, the ReAct agent chooses among the tools described below.
7. Lambda returns a JSON response containing statusCode and response; the frontend renders the response and keeps the visible conversation in the browser.

Agent tools

Tool	Used for	Implementation
SemanticSearch	Questions about report content	LangChain RetrievalQA over the Aurora pgvector index, retrieving up to five results
SQLQA	Aggregations, filters, and structured facts	Bedrock generates SQL from the extracted_entities schema, then Aurora executes it with a limit of five
Search	Current events, news, or people	LangChain DuckDuckGo search
Calculator	Arithmetic	Restricted numexpr evaluation with pi and e available

Data Preparation

The data workflow has two outputs that share the same Aurora PostgreSQL database:

1. PDF reports are downloaded and converted to page-level text and tables.
2. Text and tables are split into 512-token chunks with 64-token overlap.
3. Amazon Titan Text Embeddings generates vectors for the chunks.
4. The vectors and metadata are stored in the agentic_assistant_vector_store pgvector collection.
5. Structured entities, such as revenue, employee counts, risks, company, and year, are loaded into the extracted_entities SQL table.
6. The SageMaker pipeline combines the embedding and SQL-loading jobs so the knowledge base can be refreshed from S3.

The notebooks in experiments demonstrate these stages independently. The production-oriented SageMaker jobs and pipeline definitions are in data-pipelines.

AWS Resources

The backend CDK stack provisions:

• A VPC with public, private-with-egress, and isolated subnets, plus S3, DynamoDB, and Secrets Manager VPC endpoints.
• Aurora PostgreSQL 15.3 Serverless v2 for vectors and structured entities.
• A Docker-based Python 3.9 Lambda function with a five-minute timeout and 2 GB memory.
• DynamoDB for conversation history.
• An S3 bucket for processed data and SQL-table inputs.
• Amazon Cognito user pool and app client.
• API Gateway with Cognito authorization and CORS configuration.
• SSM parameters for Bedrock configuration, database access metadata, the Lambda name, Cognito IDs, API URL, and SageMaker networking details.
• A managed IAM policy for the SageMaker execution role.

The frontend CDK stack creates a CodeCommit repository from frontend/chat-app and deploys it to AWS Amplify with the backend Cognito and API settings.

Repository Guide

Path	Purpose
backend/	Backend CDK application and infrastructure
backend/lib/assistant-backend-stack.ts	Aurora, VPC, Lambda, API Gateway, Cognito, DynamoDB, S3, SSM, and IAM resources
backend/lib/lambda-functions/agent-executor-lambda-container/	Lambda Docker image and Python agent implementation
frontend/	Frontend CDK application for Amplify deployment
frontend/chat-app/	Next.js 14 chat UI with Amplify authentication
data-pipelines/	SageMaker notebooks, processing jobs, and pipeline workflow
experiments/	Step-by-step notebooks and a SageMaker Studio Streamlit client

Deployment

Prerequisites

• An AWS account with permissions to create the resources in both CDK stacks.
• AWS CDK 2, Node.js, npm, Docker, and Python 3.9-compatible environments.
• Amazon Bedrock model access in the selected AWS region for the configured Claude model and Titan embedding model.
• SageMaker Studio or a SageMaker notebook for running the data-processing jobs.
• An execution role for SageMaker with the managed policy produced by the backend stack.

1. Deploy the backend

cd backend
npm install
npx cdk bootstrap
npx cdk deploy

The backend deployment outputs the Cognito IDs, API endpoint, and the SageMaker policy ARN. The stack also writes the values needed by the notebooks to SSM Parameter Store. Review the CDK context in backend/cdk.json or pass -c context:parameters=... as appropriate to set the Bedrock region and model ID.

2. Prepare and load data

Open the notebooks in experiments to download or replace the example PDFs, convert them to processed text, and extract structured entities. Then use the notebooks in data-pipelines to validate the SageMaker-to-Aurora connection, load embeddings, load SQL tables, and optionally create or run the combined SageMaker pipeline.

Attach the sageMakerPostgresDBAccessIAMPolicyARN output from the backend stack to the SageMaker execution role before running jobs. The jobs expect processed documents and SQL-table inputs in the S3 bucket created by the backend stack.

3. Deploy the chat application

cd frontend
npm install
npx cdk deploy

The output AmplifyAppURL is the hosted application URL. For local frontend work:

cd frontend/chat-app
npm install
npm run dev

The frontend CDK stack normally supplies the deployed environment variables. A local run requires the Cognito user-pool IDs and API endpoint to be available as the NEXT_PUBLIC_* variables read by app/aws-exports.tsx and ChatApp.tsx.

Calling the Agent Directly

The Lambda accepts this payload:

{
	"user_input": "What was the revenue in 2021?",
	"session_id": "example-session",
	"chatbot_type": "agentic",
	"clean_history": false
}

chatbot_type can be basic or agentic and defaults to basic. The notebook experiments/06-invoke-the-agent-executor-lambda.ipynb shows synchronous invocation through Boto3. The Streamlit client in experiments/streamlit-ui exposes both chat modes and a conversation-clear control.

Important Notes

• The current Next.js chat UI does not send chatbot_type, so its API requests use the backend default, basic. Use Streamlit or the invocation notebook to exercise the agentic tools.
• The current frontend uses the Cognito user sub as the session ID. That gives each user one stored conversation; supporting multiple conversations per user requires a separate chat/session identifier.
• The example CDK stacks use DESTROY removal policies for the Aurora cluster, S3 bucket, DynamoDB table, and Cognito pool. Change these to retention policies before using the deployment with data that must survive stack deletion.
• API Gateway CORS currently allows all origins. Restrict it to the deployed frontend origin for production.
• Responses are generated by an LLM and should be verified before being used for business decisions. Retrieved source documents are not currently returned with RAG responses.
• The code uses LangChain APIs that match the versions pinned in the Lambda image requirements. Updating those dependencies may require prompt or chain changes.

Development Commands

Backend and frontend infrastructure:

npm run build
npm test
npx cdk synth
npx cdk diff

Next.js application:

npm run dev
npm run build
npm run lint

License

This project is licensed under the MIT License. See LICENSE.