MamaOpe Medicals · Knowledge engineering · Applied AI
Empirico
Engineering an AI-powered medical knowledge platform.
- My role
- Engineer on the platform and model service
- Status
- In production
- Areas
- Software & Product Engineering, AI & Machine Learning, Healthcare & Biomedical
- Stack
- Python · FastAPI · AWS S3 · Azure OpenAI · Zilliz / Milvus
The problem
Medical knowledge is spread across long clinical guidelines, formularies and reference texts. Finding the right passage quickly, and knowing where it came from, is hard, especially at the point of care.
General-purpose language models answer fluently but without reliable sourcing, which is not acceptable for clinical information.
Context
Empirico is a distinct product from HealthNavi. It addresses knowledge access and retrieval, and its model service also powers other applications, including HealthNavi.
The public product at empirico.ai offers drug dosing and interactions, clinical guidelines and treatment options, and states that it is for healthcare professionals and not a substitute for clinical judgement.
The solution
A retrieval-augmented generation (RAG) platform: medical PDFs are ingested, chunked and embedded into a vector database; each question is embedded, matched against the knowledge base, and the retrieved evidence is passed to an LLM to compose a referenced answer.
Two search depths trade speed against thoroughness: a quick mode, and a deep mode that retrieves more evidence across more sources.
Architecture
- 01
Document store
Raw medical PDFs in AWS S3
- 02
Ingestion
Parsing, chunking, page-level metadata
- 03
Embeddings
Azure OpenAI embedding models
- 04
Vector database
Zilliz (managed Milvus), hybrid retrieval
- 05
Model service
Python / FastAPI · context construction
- 06
LLM response
Gemini · answer + references + follow-ups
Components reflect the architecture during development; production configuration evolves.
What I did
Knowledge pipeline
- Designed document ingestion and processing workflows for medical PDFs, with storage in S3.
- Implemented document and query embedding workflows and worked with vector search and hybrid retrieval.
Model service & APIs
- Developed the Python/FastAPI model service: evidence retrieval, context construction, LLM integration and response generation.
- Built API access so other applications can use the platform.
Performance, cost & reliability
- Worked on caching, concurrency and rate limiting; containerised services with Docker.
- Investigated retrieval performance and infrastructure costs.
Engineering decisions
- Diversify sources, not just similarity
- Retrieval caps how many chunks come from any single document so an answer draws on several sources instead of repeating one book. That gives better coverage and more useful references.
- Quick vs. deep search
- Rather than one fixed retrieval budget, the service exposes a fast mode for routine questions and a deep mode that retrieves more evidence from more sources when thoroughness matters.
- Cache at the right layers
- Query embeddings and repeated responses are cached, cutting latency and embedding/LLM cost for common questions without changing answers.
- Page-level sources
- Each chunk keeps its source file and page number, so answers can cite exactly where the evidence came from.
Challenges
- Latency vs. thoroughness
- More retrieved context improves grounding but slows responses and raises cost. Search modes, streaming responses and caching keep the experience responsive.
- Infrastructure cost
- Managed vector search, embeddings and LLM calls each carry cost. Investigating retrieval performance and caching repeated work help keep running costs in check.
Outcomes
- Live at empirico.ai, serving evidence-backed medical information to healthcare professionals.
- Provides the knowledge layer behind HealthNavi's clinical guidance.
Screens
Technologies
- Python
- FastAPI
- AWS S3
- Azure OpenAI
- Zilliz / Milvus
- Hybrid search
- Gemini
- Docker
- DigitalOcean
What I took away
How you find and cite sources (chunking, variety and page references) matters more to answer quality than swapping models.
Cost is an engineering constraint like any other; measure it early.