Every free llm evaluation tier that clears our bar in 2026: 6 offers meet the criteria and 1 offer is demoted with a named reason. Our membership test: scores model or agent output against a dataset, a rubric or a judge model on runs you trigger. We could not confirm today's terms for 3 of them: on 1 the page we cite did not answer, and on 2 our own read did not confirm them. Each row says why.
Open-source LLM engineering platform for tracing, evaluating and debugging AI applications. The Langfuse Cloud Hobby plan is free with no credit card: 50k units a month (each trace, observation and score counts as one unit), 30 days of data access, 2 users, community support via GitHub, and all platform features with limits. Core is $29/month with 100k units, then $8 per 100k units up to 1M units and less above that. The open-source self-hosted edition is free under the MIT license.
Listed in AI / ML, and on this page because we labelled it llm_evaluation. We read that on 2026-09-01 from https://langfuse.com/, where it says: “Evaluation LLM-as-a-judge, heuristic functions, or human review. Run evaluators on production data or during experiments.”. How we use this
Platform for agent observability, evaluation and improvement. AX Free costs $0 with no credit card required: 25k trace spans per month, 1 GB ingestion per month, 15 days of retention, unlimited users and evals, 10 Signal issues per month, the Alyx agent and community support. AX Pro is $50 per month with 50k spans and 10 GB of ingestion per month and 30 days of retention. Arize also offers Phoenix, which is ELv2 licensed and can be self-hosted, and 2 Phoenix Cloud instances for free. When we last read the page we cite for this offer, on 2026-11-08, we refused the change we considered recording because it named no figure that had moved, and refusing a change is not a confirmation of the terms above, so we cannot confirm these terms today.
Listed in AI / ML, and on this page because we labelled it llm_evaluation. We read that on 2026-09-01 from https://arize.com/pricing, where it says: “Trace, evaluate, and improve your agents”. How we use this
Comet makes Opik, for LLM and agent observability and evaluation, and an MLOps platform for ML experiment tracking. Opik is open source and free to self-host. Opik Free Cloud allows up to 10 team members, 25k spans per month and 60-day data retention, with agent tracing and analysis, test suites and assertions, and Agent Playground. MLOps Free covers 1 platform user, with experiment tracking, dataset management and versioning, Model Registry and 100GB of data storage. Academics can apply for a free Pro plan.
Listed in AI / ML, and on this page because we labelled it llm_evaluation. We read that on 2026-09-01 from https://www.comet.com/site/pricing/, where it says: “Systematically test your AI application over an entire dataset”. How we use this
LLM observability and evaluation platform by LangChain — Developer plan is $0/seat/month with up to 5k base traces/month, then pay-as-you-go. LCU is $1.50/unit and LSU is $1.00/unit. Plus plan is $39/seat/month with up to 10k base traces/month, then pay-as-you-go.
Listed in AI / ML, and on this page because we labelled it llm_evaluation. We read that on 2026-09-01 from https://www.langchain.com/pricing, where it says: “Evaluation Score and improve agent performance”. How we use this
Rebranded to Respan; keywordsai.co redirects to respan.ai. Free plan: full platform, 100k logs, 1k scores, 5 datasets, 2 evaluators, 5 prompts. No credit card required. We could not read the page we cite for this offer when we last looked, on 2026-10-31, so we cannot confirm these terms today.
Listed in AI / ML, and on this page because we labelled it llm_evaluation. We read that on 2026-09-02 from https://www.respan.ai, where it says: “Route, observe, and evaluate every LLM call”. How we use this
Evals, prompt playground, and data management for Gen AI. Starter plan (free): 1 GB processed data/month, 10k scores/month, 14 days data retention. Unlimited users, projects, datasets, playgrounds, and experiments. Overage: $4/GB data, $2.50 per 1k scores.
Listed in AI / ML, and on this page because we labelled it llm_evaluation. We read that on 2026-09-01 from https://www.braintrust.dev/pricing, where it says: “Transparent pricing for AI evaluation, monitoring, and observability”. How we use this
A LLMOps platform helping AI teams measure, monitor, and optimize LLM applications for reliability, cost-efficiency, and performance. With a powerful DSPy component, we enable seamless collaboration between engineers and non-technical teams to fine-tune and productionize GenAI products. Free plan includes all platform features, 1k traces/month and 1 workflow DSPy optimizers. [#opensource](https://github.com/langwatch/langwatch). When we last read the page we cite for this offer, on 2026-10-24, we found a change we could not reconcile with the terms we publish, so we cannot confirm these terms today.
Listed in AI / ML, and on this page because we labelled it llm_evaluation. We read that on 2026-09-01 from https://langwatch.ai, where it says: “AI agent testing and evaluation that turns unpredictable agents into reliable production systems”. How we use this
| Vendor | Free Tier | Key Limits | Durability | Read / catalogue date |
|---|---|---|---|---|
| Langfuse | Free | Open-source LLM engineering platform for tracing, evaluating and debugging AI applications. The Langfuse Cloud Hobby pla... | stable | 2026-11-13 |
| Arize AX | Free | Platform for agent observability, evaluation and improvement. AX Free costs $0 with no credit card required: 25k trace s... | not published no difference measured · 2026-11-08 |
2026-11-08 / 2026-10-16 |
| Comet ML | Free | Comet makes Opik, for LLM and agent observability and evaluation, and an MLOps platform for ML experiment tracking. Opik... | watch effective 2026-04-14 |
2026-11-06 |
| LangSmith | Free | LLM observability and evaluation platform by LangChain — Developer plan is $0/seat/month with up to 5k base traces/month... | — | 2026-10-30 |
| Keywords AI | Free | Rebranded to Respan; keywordsai.co redirects to respan.ai. Free plan: full platform, 100k logs, 1k scores, 5 datasets, 2... | not published page unreadable · 2026-10-31 |
2026-10-06 · tried 2026-10-31, no read |
| Braintrust | Free | Evals, prompt playground, and data management for Gen AI. Starter plan (free): 1 GB processed data/month, 10k scores/mon... | stable | 2026-11-06 |
We publish no ranking of these. Nothing we record measures popularity or generosity, so naming a best or a top few would be our preference rather than our data. We hold 6 free llm evaluation offers that clear our bar, each listed with the terms we publish and the date we last read them. Our ranking rule is published at /criteria.
6 offers meet our criteria on this page. Of those, 2 carry no recorded change to the terms we publish, 1 a recorded narrowing, and we publish no durability signal for 3. The Durability column below says which offer is in which group, and links the dated record behind every change. We withhold the class where the pricing page does not resolve or states no amount, tier or rate we can read, where we refused the last read, where the listing is gated, or where a narrowing in our records cites no source or no archived copy of the vendor's page has confirmed it. We do not rank these by how much you get, because the limits are not comparable without reading each vendor's page.