Emily Bender’s “Climbing Towards NLU” with Alexander Koller proposes two people on separate deserted islands, communicating via an underwater telegraph cable, but one of them has been replaced by a hyper-intelligent octopus who has detected the statistical patterns in the exchanges. But because the octopus has only ever seen the sequence of signals and never the things they refer to, the moment one islander faces a genuine novelty—say, a bear attack, and urgently asks how to build a weapon from the materials at hand—the octopus cannot give real help and can only produce fluent, plausible-sounding replies without grounding in the actual world. LLMs are Stochastic Parrots, as the team she was on wrote in their “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”
Why has her work struck such a nerve? Well, “Stochastic Parrots” is a very good paper. It has the catchy meme angle that tends toward virality. And Google tried to kill it, or at least to say that these thoughts are not thoughts that anyone working at Google has or is allowed to have.
As I understand it, Google has an internal pre-publication review process called PubApprove—a check that a paper doesn’t leak proprietary information or trade secrets, not a peer review. The paper cleared PubApprove. Vice President Megan Kacholia then ordered Temit Gebru to either pull the paper submission from the conference or remove all Google authors’ names from it. Gebru asked for transparency and for meetings where she could make her case, and said if those conditions couldn’t be met she’d negotiate a departure date after her vacation. Google treated that as a resignation, cut off her email while she was on vacation, and she was out. Margaret Mitchell, her co-lead, was fired a couple of months later.
Google tells false stories about why they did what they did. Jeff Dean, head of Google AI, claimed the paper “didn’t meet our bar for publication” via PubApprove because it “ignored too much relevant research” and was submitted to PubApprove with only a day’s notice, too late for proper review. The “ignored relevant research” was bullshit: PubApprove is a sensitive-information check. And nearly half of all papers going through PubApprove were submitted with a day or less notice.
The references:
**Bender, Emily M., Timnit Gebru, Angelina Mcmillan-Major, & Shmargaret Shmitchell.**2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), 610–623. March. <https://dl.acm.org/doi/10.1145/3442188.3445922>.Bender, Emily M., & Alexander Koller. 2020. “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data”.Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5185–5198. July. <https://aclanthology.org/2020.acl-main.463/>.
Why is it Emily Bender rather than Timnit Gebru or Margaret Mitchell who shows up in my feed these days? After Google lit its reputation as a place where people could do serious work on large chunks of the study of “AI” on fire, Bender took the public-explainer path in a way that Gebru and Mitchell—now heads of DAIR (the Distributed AI Research Institute) and aresearcher and ethics lead at Hugging Face—did not.
<▶https://www.youtube.com/watch?v=ZUIQYQXsEvw>
- Brad DeLong here*: With respect to:
‘
Afaces an emergency. She is suddenly pursued by an angry bear. She grabs a couple of sticks and frantically asksBto come up with a way to construct a weapon to defend herself. Of course,Ohas no idea whatA“means”. Solving a task like this requires the ability to map accurately between words and real-world entities (as well as reasoning and creative thinking). It is at this point thatOwould fail the Turing test, if A hadn’t been eaten by the bear before noticing the deception. Having only form available as training data,Odid not learn meaning…. Because agents who produce English sentences usually have communicative intents… [A] assumes thatOdoes too, and thus she builds the conventional meaning English associates withO’s utterances. Because she assumes thatOisB, she uses that conventional meaning together with her other guesses aboutB’s state of mind and goals to attribute communicative intent. It is not thatO’s utterances make sense, but rather, thatAcan make sense of them…
Well, yes. But.
