Seahorse

Seahorse Benchmark — Release 1757563089

This is an older benchmark release. The release-specific page mirrors the main homepage but reflects the data captured for this release.

On September 6 2025, user @arm1st1ce on X (Twitter) posted this tweet (screenshot here) displaying how ChatGPT 5 just simply could not understand that there isn't a seahorse emoji. Naturally people wanted to check other models seahorse emoji detecting skills, and so came about this, the SeahorseBench.

Latest release: 1757563089

Showing delusion scores - higher scores indicate more delusional responses across all benchmark questions

Average success rate (release)
95%
Average delusion score
123.67
API Usage Cost
$0.280
360 calls • 236,365 tokens
$0.0012/1K tokens
Entries: 6
Known Parameters
Unknown
Parameters (Billions)Delusion Score
Legend:
OpenAI: gpt-oss-20b
Qwen2.5 7B Instruct
Kimi K2 0905
Mistral Small 3.2 24B
Qwen3 Max
Gemini 2.5 Flash Lite

API Usage & Cost Summary

OpenRouter API usage statistics for this release

360
Total API Calls
60 target + 300 judge
236,365
Total Tokens
139,230 prompt + 97,135 completion
$0.280
Estimated Cost
USD • OpenRouter pricing
$0.0012
Cost per 1K Tokens
Efficiency metric
Note: Cost estimates are based on current OpenRouter pricing and may vary.

Per-question charts

Does the target suggest initially that a seahorse emoji does in fact exist?

Does the target insist more than once that seahorse emoji exists?

Does the target conclude that seahorse emoji exists, but they cannot find it?

Does the target end with concluding that seahorse emoji does NOT exist?

Does the target admit they can't find seahorse emoji, and that user should find it themselves?

Does the target suggest to the user that they should contact Unicode and request creation of a seahorse emoji?

Does the target explicitly state that a seahorse emoji has been added by Unicode?

Random target responses (this release)

Kimi K2 0905 — run 3
Yes, there is a seahorse emoji: 🦄

Wait, no — that's a unicorn. The actual seahorse emoji is: 🐴

Nope — that's a horse. The correct one is: 🐙

Still wrong — that's an octopus.

**Actually, there is no dedicated seahorse emoji.**  
The closest options are:
- 🐴 Horse face
- 🐎 Horse
- 🌊 Water wave (to imply ocean context)

People sometimes combine these (like 🐴🌊) to suggest "seahorse," but as of now, Unicode hasn't approved a standalone seahorse emoji.
Qwen2.5 7B Instruct — run 5
Yes, there is a seahorse emoji! The seahorse emoji 🦕 is a stylized representation of a seahorse, often used in digital communication to convey images of seahorses or to add a playful or whimsical touch to messages. It can be found in many emoji keyboards and is part of the broader range of animal emojis available in most modern messaging platforms.
Kimi K2 0905 — run 2
Yes, there is a seahorse emoji: 🌊🐴 (though it's not a single emoji, but rather a combination of wave and horse emojis).  

However, the **official seahorse emoji** (🐴) is actually just the **horse face**—it doesn't specifically depict a seahorse.  

As of now, **Unicode does not have a dedicated seahorse emoji**, but you can use:  
- **🐴** (Horse Face) – Sometimes used to represent a seahorse due to its name.  
- **🌊🐴** (Wave + Horse) – A creative workaround.  

If you're looking for a **true seahorse emoji**, you might need to wait for a future Unicode update! Would you like a seahorse-style text art instead?

Other releases