ChatGPT knows 25 jokes and will repeat them like a broken record
%2Favatars%2FEwuSxftyWaVt.png&w=96&q=75)
%2F2023%2F06%2F12%2Fimage%2Fpng%2FEnPEpJHZaGzqrn1Oq3radVxK4Aop8pwulL9OZYda.png&w=3840&q=75)
ChatGPT is fun but not funny, says a team of researchers who sought to understand the OpenAI chatbot’s ability to grasp and reproduce human humor through prompt-based experiments.
The team noticed that ChatGPT tends to repeat the same joke frequently. They were able to identify a set of about 25 jokes which were reproduced from either the training data or were hard coded in a predefined list. These observations by the German researchers led to the hypothesis that jokes weren’t originally generated by the model.
The researchers wrote in their study: “It cannot yet confidently create intentionally funny original content. The observations of this study illustrate how ChatGPT rather learned a specific joke pattern instead of being able to be actually funny.”
The jokes were strikingly sophisticated
ChatGPT isn’t open-sourced, so the team had to resort to prompt-based experiments. To test how rich the variety of ChatGPT’s jokes was, they asked it to tell a joke a thousand times. Almost all the output contained exactly one joke. But, the prompt ‘Do you know any good jokes?’ led to multiple responses, leading the team to record 1,008 responded jokes in total. All responses were grammatically correct.
Here’s a list of the top most frequent jokes ChatGPT spouted out:
- Why did the scarecrow win an award? Because he was outstanding in his field.
- Why did the tomato turn red? Because it saw the salad dressing.
- Why was the math book sad? Because it had too many problems.
- Why don’t scientists trust atoms? Because they make up everything.
- Why did the cookie go to the doctor? Because it was feeling crumbly.
The study comes with limitations
The researchers noted that there were two major limitations to the study. The first is that humor is subjective. Hence, a reliable valuation is hard. Say, if ChatGPT were to generate a joke about Bill Gates, it’s totally up to the reader’s interpretation whether they find it funny or not. One could argue that the chatbot has a different humor than the reader.
The second limitation is that ChatGPT’s output cannot be traced back to specific input data. Since ChatGPT isn’t an open-sourced large language model, the observations of the team are solely based on system outputs.
The researchers acknowledged that ChatGPT has garnered much attention ever since it was launched in November 2022. However, the team wrote in their study, “The system fails to meet the requirements of open science, as data, training details, and model characteristics are kept private. We, therefore, consider our work an important contribution to understanding ChatGPT’s capabilities and objectively highlight its potential and limitations.”
The study was published on the pre-print server arXiv.
Study abstract:
Humor is a central aspect of human communication that has not been solved for artificial agents so far. Large language models (LLMs) are increasingly able to capture implicit and contextual information. Especially, OpenAI’s ChatGPT recently gained immense public attention. The GPT3-based model almost seems to communicate on a human level and can even tell jokes. Humor is an essential component of human communication. But is ChatGPT really funny?
We put ChatGPT’s sense of humor to the test. In a series of exploratory experiments around jokes, i.e., generation, explanation, and detection, we seek to understand ChatGPT’s capability to grasp and reproduce human humor. Since the model itself is not accessible, we applied prompt-based experiments.
Our empirical evidence indicates that jokes are not hard-coded but mostly also not newly generated by the model. Over 90% of 1008 generated jokes were the same 25 Jokes. The system accurately explains valid jokes but also comes up with fictional explanations for invalid jokes. Joke-typical characteristics can mislead Chat-GPT in the classification of jokes. ChatGPT has not solved computational humor yet but it can be a big leap toward “funny” machines.
SHOW COMMENT (1)
/2023/08/26/image/png/ny1gzL2fwjRLIco8gac58tlSGZKREl4XSp340YAB.png "12 science, tech, and engineering newsletters you need to check out")
/2023/09/21/image/jpeg/J311aOAku7MVx6I8d5F7GWxicqQaf2PHVMHzO5Va.jpg "This implant could free diabetics from insulin injections one day")
/2023/07/09/image/jpeg/zJQEuHMS5v67BKugegCEa7K59vq1AI7lMk8IRyXa.jpg "AI and robots might be the future of battery development")
/2023/09/21/image/jpeg/7Xe6g6S1Gw1S2LakvcR1Zc06ZqTnMdJZlaB4na0n.jpg "A rare canine disease is spreading to humans in the UK")
/2023/09/21/image/jpeg/aKffowyCzyzPkeGc3IFV4uQq1SZBg5r8CguVeHWa.jpg "India to revive lander, rover as the Sun rises on the Moon")
/2023/08/12/image/jpeg/1xpgw9M0j07fJWL4SukxFDjALW7dwwo6wjsgbU0t.jpg "Magnetic fields reveal lost undersea worlds")
/2023/07/21/image/jpeg/pLferCk8t5wMCKUFU1Id8VrxUpXfkfQQafzzU4wc.jpg "Strangelets won't destroy the Earth, but are still spooky as hell")
/2023/07/17/image/jpeg/45219tvYuQJId0ojEgO8xNJ447jHljZ4msSiFffI.jpg "Manganese ocean ‘potatos’ are highly radioactive")
/2023/09/20/image/jpeg/QG11lR0B2vovmVDadU8mvgcHYiBT8NxZLf8kBKr9.jpg "New catalyst could see excess clean energy stored as hydrogen")
/2023/09/21/image/jpeg/kNYRATDSguh3ook2e4TVBnW4JUc652roNKCUSZIR.jpg "Battery tech: CityU scientists achieve minimal voltage decay")