RAG Developer London

As Associate Director of Applied AI/ML at Fitch Ratings, I design and build production RAG systems that enable analysts to semantically query large-scale corpora of credit rating reports, financial filings, and regulatory documents. RAG at Fitch is not a demo: it is a production system handling real analyst workflows in a regulated financial environment.

What Makes a Production RAG System

A RAG system that works in a demo often fails in production. The differences that matter are chunking strategy, retrieval quality, re-ranking, context assembly, citation accuracy, and evaluation rigor. Every component has to be correct.

Document Extraction and Processing

pymupdf4llm

The foundation of the document intelligence pipeline at Fitch. pymupdf4llm provides high-fidelity extraction from PDFs that preserves document structure, tables, and layout, which matters for financial documents where table data is often the most important content.

• Markdown-format extraction preserving document structure
• Table extraction and normalization for financial data
• Header and section detection for hierarchical chunking
• Page-level metadata preservation for citation

Chunking Strategies

• Semantic chunking aligned to document section boundaries
• Sliding window chunking with overlap for context preservation
• Recursive character splitting with configurable separators
• Domain-specific chunking for financial document patterns

Vector Storage and Retrieval

Azure AI Search

The primary vector store at Fitch Ratings, chosen for enterprise compliance, Azure ecosystem integration, and hybrid search capability.

• Vector indexing with HNSW for approximate nearest neighbour search
• Hybrid search combining semantic and keyword (BM25) retrieval
• Semantic re-ranking with Azure cross-encoder models
• Metadata filtering for document type, date, and entity filtering
• Integrated with Azure AI Foundry for end-to-end RAG pipelines

Embedding Models

• Selection and benchmarking of embedding models for financial domain text
• Domain-specific fine-tuning evaluation for financial terminology
• Embedding dimension and model size trade-off analysis
• Batch embedding pipelines for large document corpora

LangChain and LlamaIndex RAG Patterns

LangChain

• Document loaders, text splitters, and embedding integrations
• Retrieval chains with custom retrievers and re-rankers
• Conversational RAG with memory for multi-turn analyst interactions
• Citation and source attribution in generated responses

LlamaIndex

• Query routing across multiple indices
• Knowledge graph construction from document corpora
• Sub-question decomposition for complex analytical queries
• Response synthesis with structured output

Evaluation and Quality Assurance

RAG system quality is measured at every layer:

• Retrieval metrics: recall at k, mean reciprocal rank, NDCG
• Generation metrics: faithfulness, answer relevance, context utilization
• End-to-end metrics: answer correctness on ground truth QA sets
• Citation accuracy: do responses correctly attribute source documents
• Regression test suites run before every deployment

Infrastructure

• FastAPI for RAG-serving endpoints
• Redis for retrieval result caching
• Celery for async embedding and ingestion pipelines
• Airflow for scheduled corpus refresh and re-indexing
• Docker and Kubernetes for containerized deployment
• AWS and Azure for hybrid cloud hosting

Financial Services RAG

Building RAG for financial services requires understanding the domain: rating methodologies, regulatory frameworks, financial statement structure, and the precision requirements of analyst workflows. Generic RAG tutorials do not prepare you for this. Production experience does.

Connect

Happy to discuss RAG architecture, retrieval quality, evaluation strategy, or document intelligence systems. Reach out at [email protected].