The problem
Internal LLM tools kept confidently answering questions using outdated or generic knowledge, because they had no access to the product’s actual documentation, architecture decisions, or historical context. Every new agent project meant re-solving the same retrieval problem from scratch.
What I built
A shared retrieval-augmented generation pipeline: Confluence pages and S3-stored artifacts are chunked, embedded with OpenAI’s embedding models, and stored in a pgvector-backed PostgreSQL instance. Any downstream tool — the Jira triage agent, internal chat assistants, ad hoc scripts — queries the same knowledge base rather than each maintaining its own retrieval logic.
How it works
- Source ingestion: scheduled jobs pull updated Confluence pages and S3 documents.
- Chunking: documents are split into overlapping semantic chunks sized for retrieval quality, not just token limits.
- Embedding: each chunk is embedded and stored alongside source metadata (page, section, last-updated date) so retrieved context can be traced back to its origin.
- Retrieval: consuming applications run a similarity search against pgvector and inject the top-matching chunks into the model’s context window.
Impact
Answers from downstream tools became traceable to a real source instead of model guesswork, and every new agent project stopped having to rebuild its own retrieval layer — they just query the shared knowledge base.
Stack
PostgreSQL + pgvector, OpenAI embeddings, Confluence API, AWS S3, Python.