LLM applications
Grounded summaries for complex business reports
My contribution
I built the LLM summarization workflow for CLEAR Business AI: selective XML extraction, targeted calls for individual report sections, and source links for summary claims. I integrated search and LLM verification to help distinguish entities with the same name, then refined extraction and prompts through subject matter expert feedback.
-
Select context
Relevant XML fields and report sections
-
Summarize
Targeted LLM calls for each section
-
Ground claims
Source links and entity verification
Explore the technical details
The engineering challenge was deciding which context each call needed, keeping claims traceable, and incorporating expert feedback into the next iteration.
Built the LLM summarization workflow for CLEAR Business entity reports, live in production processing hundreds of reports daily. Designed selective XML extraction for targeted per-section context covering business overview, personnel, adverse media, and legal records. Integrated Bing Search and LLM verification to reduce same-name entity mismatches and linked claims to report sources. Refined extraction and prompts through SME annotation rounds against exemplars for 10-15 companies, and compared GPT-4 and Claude Sonnet on quality, latency, and cost to inform model selection.
Tools & technologies: Python · GPT-4 · Claude Sonnet · Bing Search API · Prompt & Context Engineering · LLM Evaluation · XML Processing