Why do LLMs freak out over the seahorse emoji?