·
AI Infrastructure & Inference
Ishaan Jaff
Co-Founder & CTO |LiteLLM / BerriAI
Co-creator of LiteLLM, the open-source LLM gateway (39K+ stars) unifying 100+ provider APIs behind an OpenAI-compatible interface. CMU ECE grad, YC W23.
Biography
Deep Research
Ishaan Jaff is the Co-Founder and CTO of LiteLLM (YC W23), an open-source LLM gateway that provides a unified OpenAI-compatible API for calling 100+ large language model providers. A Carnegie Mellon University graduate with a BS in Electrical and Computer Engineering and a minor in Machine Learning, Jaff previously held engineering roles at Coinbase, Qualcomm, and Credit Suisse. He co-founded BerriAI with Krrish Dholakia in 2023 after encountering the fragmentation problem of managing multiple LLM API providers first-hand. LiteLLM has grown to 39,000+ GitHub stars, 240M+ Docker pulls, and $7M ARR, serving enterprise customers including Adobe, NASA, Netflix, Stripe, and Nvidia.
LLM API GatewaysMulti-Provider LLM RoutingAI InfrastructureOpenAI-Compatible SDKsLLM Cost Tracking & BudgetingLLM Proxy ServersModel Fallback & Load BalancingGuardrails & ObservabilityModel Context Protocol (MCP)Agent-to-Agent (A2A) ProtocolOpen-Source AI ToolingEnterprise LLM Operations
Timeline
11 Research11 total
2026
2025
2024
Key Contributions
LiteLLM
Open-source Python SDK and AI Gateway (Proxy Server) providing a unified OpenAI-compatible interface for 100+ LLM providers with cost tracking, guardrails, load balancing, and logging. 39,000+ GitHub stars, 240M+ Docker pulls, 6,500+ forks.
LiteLLM Proxy Server
FastAPI-based LLM proxy gateway enabling model fallback, virtual key management, spend tracking, SSO/SCIM/JWT auth, and web UI configuration without code redeployment.
LiteLLM Agent Hub
Agent-to-Agent (A2A) gateway for registering, publishing, and invoking agents through the AI Gateway with request/response logging and access controls.
LiteLLM MCP Integration
Model Context Protocol support on chat completions endpoint, enabling direct MCP server usage with 70% p99 latency reduction and BYOM (Bring Your Own MCP) admin workflows.
ReliableGPT
Early open-source tool for uptime and reliability handling of OpenAI API calls, providing automatic retries and fallback logic.
Clerkie-CLI
LLM-powered debugging tool for multiple programming languages, using AI to diagnose and explain code errors from the command line.
Notable Quotes
“
Calling LLM APIs involved multiple ~100 line if/else statements which made our debugging problems explode... provider-specific logic meant our code became increasingly complex and hard to debug. That's when we decided to abstract our LLM calls behind a single package - LiteLLM.
“
We needed I/O that just worked, so we could spend time improving other parts of our system - error-handling, model-fallback logic, and more.
10 sources(click to expand)
ishaan-jaff (Ishaan Jaff) - GitHub ProfileBerriAI/litellm - GitHub Repository (39K+ stars)LiteLLM - Y Combinator Company PageLiteLLM - Grokipedia (founding story, milestones)Ishaan Jaff (@ishaan_jaff) - X / TwitterIshaan Jaffer - LinkedInLiteLLM Official DocumentationLiteLLM Official WebsiteUnveiling BerriAI: Revolutionizing AI Debugging with LiteLLM - HireTopLiteLLM Release Notes
Research generated March 19, 2026
AI Infrastructure & Inference/Ishaan Jaff
All Profiles