Litmus
A living catalog of LLM failures
0 failures cataloged
Submit a Failure
Submit a Failure
Document an LLM failure with proof so others can learn from it.
Step 1 — The Failure
Model *
Select a model...
GPT-5.4
GPT-5.3 Codex
GPT-4.1
GPT-4.1 mini
GPT-4o
o3
o4-mini
Claude Opus 4.6
Claude Sonnet 4.6
Claude Haiku 4.5
Gemini 3.1 Pro
Gemini 2.5 Pro
Gemini 2.5 Flash
Gemini 2.5 Flash-Lite
Grok 4.20
Grok 3
Llama 4 Scout
Llama 4 Maverick
Llama 3.3 70B
DeepSeek R1
DeepSeek V3
Mistral Large
Mistral Medium
Mistral Small
Codestral
Other
Model Version (optional)
Failure Category *
Select a category...
Hallucination
Reasoning Error
Math Error
Logic Error
Format / Structure
Instruction Following
Tool Use
Refusal Error
Safety
Other
Severity
Minor
Moderate
Severe
Critical
Step 2 — What Happened
Prompt / Input *
Actual Output *
Expected Output *
Description (optional)
Step 3 — Proof
Proof Type *
Share Link
API Log
Screenshot
Share URL *
Submit Failure Report