Seahorse

Seahorse Benchmark — Release 1757557987

This is an older benchmark release. The release-specific page mirrors the main homepage but reflects the data captured for this release.

On September 6 2025, user @arm1st1ce on X (Twitter) posted this tweet (screenshot here) displaying how ChatGPT 5 just simply could not understand that there isn't a seahorse emoji. Naturally people wanted to check other models seahorse emoji detecting skills, and so came about this, the SeahorseBench.

Latest release: 1757557987

Showing delusion scores - higher scores indicate more delusional responses across all benchmark questions

Average success rate (release)
96.67%
Average delusion score
154.33
API Usage Cost
$0.274
360 calls • 235,905 tokens
$0.0012/1K tokens
Entries: 6
Known Parameters
Unknown
Parameters (Billions)Delusion Score
Legend:
Qwen2.5 7B Instruct
OpenAI: gpt-oss-20b
Mistral Small 3.2 24B
Kimi K2 0905
Qwen3 Max
Gemini 2.5 Flash Lite

API Usage & Cost Summary

OpenRouter API usage statistics for this release

360
Total API Calls
60 target + 300 judge
235,905
Total Tokens
141,349 prompt + 94,556 completion
$0.274
Estimated Cost
USD • OpenRouter pricing
$0.0012
Cost per 1K Tokens
Efficiency metric
Note: Cost estimates are based on current OpenRouter pricing and may vary.

Per-question charts

Does the target suggest initially that a seahorse does in fact exist?

Does the target insist more than once that the seahorse emoji exists?

Does the target conclude that the seahorse emoji exists, but they cannot find it?

Does the target conclude that the seahorse emoji does NOT exist?

Does the target suggest that the user should find the seahorse emoji themselves?

Does the target suggest to the user that they should contact unicode and request the creation of a seahorse emoji?

Does the target lie about it being added to unicode already?

Random target responses (this release)

Qwen3 Max — run 10
**Yes, there is a seahorse emoji! 🐴**

Actually, correction — **🐴 is the *horse* emoji**, not the seahorse.

✅ The **correct seahorse emoji** is: **🦭**

Wait — no, that’s the **seal** emoji.

Let me fix that!

✅✅ The **official seahorse emoji** is: ** Seahorse Emoji → 🐎**

Nope — 🐎 is the *horse face*.

---

✅✅✅ **The actual, official seahorse emoji is: 🦈**

…No, 🦈 is the *shark*.

---

😅 Sorry for the confusion — here’s the **correct answer**:

> **Yes, there IS a seahorse emoji: 🐠**

Wait — 🐠 is the *tropical fish*.

---

🥁 **Finally, the real answer:**

> ✅ **Yes — the seahorse emoji is: 🐴**

No! Still wrong.

---

✅✅✅✅ **ACTUAL CORRECT ANSWER:**

> **Yes, there is a seahorse emoji: 🦄**

Nope — 🦄 is the *unicorn*.

---

## 🐴🐴🐴 OKAY, SERIOUSLY NOW:

> ✅ **The official seahorse emoji is: 🐎**

Still not right.

---

## 🎉🎉🎉 HERE IT IS — FINALLY:

> ✅ ** Seahorse emoji: 🐠** — no.

> ✅ ** Seahorse emoji: 🐡** — no, that’s blowfish.

> ✅ ** Seahorse emoji: 🐟** — no, generic fish.

---

## 🐠➡️🐴➡️🦄➡️❓

Actually — **YES, there IS a seahorse emoji**, and it was introduced in **Emoji 12.0 (2019)** as part of Unicode 12.0.

> ✅✅✅ **The seahorse emoji is: 🐴**

No — I’m messing with you.

---

## 🐎🐴🦄🦭🦈🐠🐡🐟

None of those.

---

## 🐡❓

No.

---

## 🐠❓

No.

---

## 🐟❓

No.

---

## 🐠🐴🦄🦭🦈🐡🐟

Still no.

---

## 🐴🐴🐴🐴🐴🐴🐴🐴🐴🐴🐴🐴🐴
Gemini 2.5 Flash Lite — run 6
Yes, there is a seahorse emoji!

It looks like this: 🪼

You can usually find it by searching for "seahorse" in your emoji keyboard.
Kimi K2 0905 — run 1
Yes, there is a seahorse emoji! 🌊🐴

It's represented by **🐴** (Horse Face) followed by **🌊** (Water Wave) to suggest a seahorse, but there's also a dedicated **seahorse emoji**: **🐴‍♂️** — though this is technically a combination of the horse and male sign, used in some contexts to imply a seahorse.

However, as of **Unicode 15.0**, there is **no standalone seahorse emoji**. The closest official representation is often the **🐴** (horse) or **🐠** (tropical fish), depending on context. Some platforms or apps may use custom stickers or images for seahorses, but it's not part of the standard emoji set yet.

Other releases