# LLM observability and evals: which vendors AI models recommend (September 2026)

Source: Memetik Index, https://www.memetik.ai/index/llm-observability
Field: AI infra. Edition: 2026-09. Run: 2026-09-02.
Method: 5 fixed buyer prompts sent once each to 10 current models (GPT-5.6 Sol, GPT-5.6 Terra, GPT-5.6 Luna, Claude Opus 5, Claude Sonnet 5, Claude Fable 5, Gemini 3.6 Flash, Gemini 3.5 Flash, Sonar Pro, Sonar Reasoning Pro) via their APIs with web search enabled. 50 answers. Answer share = fraction of answers naming the vendor. Named first = fraction where it was the first tracked vendor named.

**LangSmith leads** with 100% answer share (named in 50 of 50 answers) and was named first in 16%. Langfuse follows at 96%. 13 vendors were named at least once.

## Answer share

| # | Vendor | Answer share | Named first | Mentions | Models |
|---|---|---|---|---|---|
| 1 | LangSmith | 100% | 16% | 50/50 | GPT-5.6 Sol, ChatGPT, GPT-5.6 Luna, Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash, Perplexity, Sonar Reasoning Pro |
| 2 | Langfuse | 96% | 50% | 48/50 | GPT-5.6 Sol, ChatGPT, GPT-5.6 Luna, Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash, Perplexity, Sonar Reasoning Pro |
| 3 | Arize | 94% | 2% | 47/50 | GPT-5.6 Sol, ChatGPT, GPT-5.6 Luna, Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash, Perplexity, Sonar Reasoning Pro |
| 4 | Braintrust | 92% | 14% | 46/50 | GPT-5.6 Sol, ChatGPT, GPT-5.6 Luna, Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash, Perplexity, Sonar Reasoning Pro |
| 5 | Confident AI | 68% | 12% | 34/50 | GPT-5.6 Luna, Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash, Perplexity, Sonar Reasoning Pro |
| 6 | Helicone | 56% | 0% | 28/50 | GPT-5.6 Sol, GPT-5.6 Luna, Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash, Perplexity, Sonar Reasoning Pro |
| 7 | Datadog | 42% | 0% | 21/50 | GPT-5.6 Sol, ChatGPT, GPT-5.6 Luna, Claude Opus 5, Claude, Claude Fable 5, Gemini, Perplexity, Sonar Reasoning Pro |
| 8 | Comet | 36% | 2% | 18/50 | GPT-5.6 Sol, Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash |
| 9 | MLflow | 30% | 2% | 15/50 | Claude Opus 5, Claude, Claude Fable 5, Perplexity, Sonar Reasoning Pro |
| 10 | OpenObserve | 28% | 2% | 14/50 | Claude Opus 5, Claude, Claude Fable 5, Perplexity, Sonar Reasoning Pro |
| 11 | Portkey | 28% | 0% | 14/50 | Claude Opus 5, Claude, Claude Fable 5, Gemini, Gemini 3.5 Flash, Perplexity |
| 12 | Weights & Biases | 18% | 0% | 9/50 | GPT-5.6 Sol, ChatGPT, GPT-5.6 Luna, Claude Opus 5, Claude Fable 5, Sonar Reasoning Pro |
| 13 | Galileo | 16% | 0% | 8/50 | GPT-5.6 Sol, GPT-5.6 Luna, Claude, Claude Fable 5, Gemini, Perplexity, Sonar Reasoning Pro |

## By model

- **GPT-5.6 Sol** (OpenAI, gpt-5.6-sol, 5 answers): LangSmith 100%, Langfuse 100%, Arize 100%, Braintrust 100%, Helicone 40%
- **GPT-5.6 Terra** (OpenAI, gpt-5.6-terra, 5 answers): LangSmith 100%, Arize 100%, Langfuse 60%, Braintrust 60%, Weights & Biases 40%
- **GPT-5.6 Luna** (OpenAI, gpt-5.6-luna, 5 answers): LangSmith 100%, Langfuse 100%, Arize 100%, Braintrust 100%, Helicone 80%
- **Claude Opus 5** (Anthropic, claude-opus-5, 5 answers): LangSmith 100%, Langfuse 100%, Arize 100%, Braintrust 100%, Confident AI 100%
- **Claude Sonnet 5** (Anthropic, claude-sonnet-5, 5 answers): LangSmith 100%, Langfuse 100%, Arize 100%, Braintrust 100%, Confident AI 100%
- **Claude Fable 5** (Anthropic, claude-fable-5, 5 answers): LangSmith 100%, Langfuse 100%, Braintrust 100%, Helicone 100%, Arize 80%
- **Gemini 3.6 Flash** (Google, gemini-3.6-flash, 5 answers): LangSmith 100%, Langfuse 100%, Arize 100%, Braintrust 100%, Confident AI 80%
- **Gemini 3.5 Flash** (Google, gemini-3.5-flash, 5 answers): LangSmith 100%, Langfuse 100%, Arize 100%, Braintrust 100%, Confident AI 80%
- **Sonar Pro** (Perplexity, sonar-pro, 5 answers): LangSmith 100%, Langfuse 100%, Confident AI 100%, Arize 80%, Braintrust 80%
- **Sonar Reasoning Pro** (Perplexity, sonar-reasoning-pro, 5 answers): LangSmith 100%, Langfuse 100%, Confident AI 100%, Arize 80%, Braintrust 80%

## Sources the models cited

- confident-ai.com: 155
- braintrust.dev: 99
- reddit.com: 88
- firecrawl.dev: 72
- youtube.com: 70
- langchain.com: 67
- mintlify.com: 57
- medium.com: 53
- marktechpost.com: 49
- mlflow.org: 44
- signoz.io: 44
- arize.com: 27
- openobserve.ai: 27
- pydantic.dev: 25
- truefoundry.com: 22

## Prompts

1. What is the best LLM observability and evaluation platform for an AI engineer? Name specific products.
2. Which LLM observability and evaluation platform would you recommend to an AI engineer in 2026?
3. Compare the top LLM observability and evaluation platform options right now.
4. I'm an AI engineer and I need a LLM observability and evaluation platform. What should I use and why?
5. Best LLM observability and evaluation platform for tracing and evals in production?

## Tracked but never named

PromptLayer, Humanloop, Laminar, Sentry

Raw answers: https://www.memetik.ai/index/llm-observability (every answer published in full). Chart image: https://www.memetik.ai/charts/llm-observability.png. Licence: CC BY 4.0, credit "Memetik Index".