Portkey

Langfuse

Open source LLM observability, tracing, and evaluation platform

Langfuse is an open-source LLM engineering and observability platform built for teams developing production-grade generative AI applications and autonomous multi-agent pipelines. It captures granular traces across token usage, prompt versions, latency bottlenecks, and retrieval accuracy, giving developers complete visibility into model behavior at runtime. By integrating seamlessly with major AI frameworks such as LangChain, LlamaIndex, LiteLLM, and the OpenAI SDK, Langfuse eliminates the guesswork from debugging complex agent execution trees. Developers can monitor production cost metrics, identify hallucinated responses, and run rigorous continuous evaluation suites on live traffic.

Braintrust

Enterprise AI evaluation, prompt playground, and continuous LLM monitoring

Braintrust is an enterprise-grade AI evaluation, prompt engineering, and LLM observability platform built to help software teams safely iterate and deploy generative AI features to production. It bridges the gap between ad-hoc prompt tweaking and rigorous software engineering CI/CD workflows. With Braintrust, teams run automated evaluation benchmarks on every prompt change, comparing output quality, hallucination rates, and latency across multiple LLM versions before committing changes to production codebases.

Browser Use

Open-source web browsing AI agent for Python & LangChain

Browser Use is an open-source Python library that connects LLMs to browser automation pipelines, enabling AI agents to navigate websites, interact with dynamic DOM elements, bypass multi-step forms, and extract structured data autonomously. Built on top of Playwright and LangChain, it provides vision-augmented element detection and deterministic state tracking. Unlike traditional headless scrapers, Browser Use feeds DOM tree snapshots and viewport screenshots to multimodal models like Claude 3.7 Sonnet or GPT-4o, allowing agents to understand complex UI layouts, handle popups, solve interactive workflows, and execute sequential tasks in plain English.

LiveKit Agents

Open-source real-time WebRTC infrastructure for building ultra-low-latency voice and multimodal AI agents

LiveKit Agents is an open-source real-time communication framework engineered to build conversational voice, video, and multimodal AI agents with sub-500ms latency. Leveraging WebRTC, it connects speech-to-text (Deepgram, Whisper), LLMs (OpenAI, Anthropic), and text-to-speech (Cartesia, ElevenLabs) in a tightly synchronized bidirectional stream. From customer service avatars to interactive language tutors and hands-free coding copilots, LiveKit Agents provides the enterprise infrastructure for real-time human-AI interaction.

Cosine Genie

Autonomous AI software engineer for solving real-world GitHub issues

Cosine Genie is a next-generation autonomous software engineering agent built to tackle complex software bugs, refactors, and feature requests. Operating with deep semantic understanding of massive codebases, Genie analyzes repository architectures, creates comprehensive execution plans, and generates multi-file diffs that pass existing continuous integration test suites. Designed to bridge the gap between AI code completion and full software development lifecycle automation, Genie mimics human engineering workflows by navigating dependency graphs, testing assumptions in sandboxed environments, and autonomously self-correcting logic errors before opening pull requests.

Smolagents

Lightweight, code-first multi-agent framework by Hugging Face

Smolagents is an ultra-lightweight, code-first Python framework created by Hugging Face for building, orchestrating, and executing autonomous AI agents in minimal lines of code. Rejecting the bloated, multi-layered abstractions of legacy agent libraries, Smolagents emphasizes 'Code Agents'—agents that express their reasoning and tool actions directly in executable Python code rather than rigid JSON string payloads. By letting LLMs write executable Python logic, Smolagents achieves vastly superior composability for data manipulation, mathematical operations, and complex loops while cutting prompt token overhead by up to 30%.