RAG tooling and generation API for customer support chatbot
Jan '25 – pres. |
CHECK24 |
Tech Lead of team of 3 devs
Fullstack dev
(Note: 4 months parental leave during this period)
Led team scaling from single dev to 3-person fullstack team.
Established CI/CD practices, mentored members, defined roadmap.
Made critical Azure LLM API pricing/performance decisions.
Maintained and expanded MCP Client implementation.
Scope: 150 Users (Content Managers), 100+ RAG configurations, 200k daily requests for LLM generation.
Azure Cognitive Services
AWS CloudFormation
FastAPI
asyncio
Postgres+pgvector
React
MCP
AWS CloudWatch dashboard + alarms
Livechat model comparison dashboard and fine-tuning
Mar '24 – Dec '24 |
CHECK24 |
Data Scientist API dev
Built interactive ELO scoring dashboard to evaluate and compare GPT, Claude, Gemini and other models based on human agents' selection preferences from multiple model suggestions.
Developed end-to-end fine-tuning (GPT-3.5/4o) and evaluation pipeline.
Designed strategy for systematic model rollouts based on real-time (daily) preference data from agents.
Built API serving 4000+ suggestions daily from 10+ varied models/providers.
Azure Cognitive Services (OpenAI models)
OpenAI fine-tuning API
Vertex AI (Claude and Gemini models)
Streamlit
Search Query Spellchecker
Mar '22 – Mar '24 |
CHECK24 |
Data Scientist API dev
Designed and built ensemble spellchecker with custom seq2seq transformer, owning the complete roadmap: dataset engineering, model training, evaluation, deployment, monitoring, and KPI definition. Delivered production system: 10+ req/s at 200ms median response time, <1% overcorrection rate. Revisited twice over 18 months for retraining and optimization.
Huggingface Transformers
Elasticsearch
Docker Compose
Pandas