Comet

What it does

Comet is an AI developer platform built around Opik, an open source tool for LLM observability and evaluation. It logs agent traces, groups recurring errors, suggests fixes, and scores results with test suites and metrics. It also tracks coding agent spend and model training runs.

Comet is aimed at teams building and running GenAI apps and agents. Opik logs traces across 60+ integrations and supports OpenTelemetry, so you can see context retrieval, tool calls, and user feedback in one place. Diagnostics groups similar recurring issues and recommends fixes, while the Ollie agent suggests code changes inside Opik. Test Suites give pass or fail results, and 40+ LLM-as-a-judge metrics score traces against goals. Production dashboards track model costs and alerts. The open source version shares the same codebase as the hosted plans, so self-hosting is possible. The free cloud plan allows up to 10 team members, 25k spans per month, and 60-day data retention. Pro Cloud costs $19 per month with 100k spans per month and 60-day retention. Enterprise adds custom usage, flexible deployments, SSO, and compliance. Check span limits and retention before committing.