The more generic your prompt, the more generic the response. It's a regression to the "mean" of the training data aka GIGO for AI.
It's like when you ask your average person off the street to draw a house - it'll almost always be square with a triangle roof, one door, and two windows.
In the pelican/bike example, it's probably a bit of a self-perpetuating snowball too. If the earliest examples were bike left-to-right, flat ground, etc. then they are also being scraped up in future LLMs.
It's just the simplest most recognizable form of a house. Like how a smiley face is so generic and simplistic but everyone will know what it represents. Just two dots and a line yet it's easily and unambiguously understood to represent a human face and a happy emotion.
I sincerely believe I've never had a single original thought™ in my whole life.
There is this scene in the HBO series Westworld where a "host" says some words in sequence which is shown on a display as she says it. Of course, even me thinking of this scene and connecting it to your comment was not original, someone else clearly had the same programming as me.
A medium blog post says
> Pair what with me?” — the moment Maeve (a humanoid android) uttered those words in Westworld (Season 1, Episode 6: “The Adversary”), something clicked. Not for the average viewer, but for me, a STEM educator and AI enthusiast who, just weeks earlier, had read Stephen Wolfram’s seminal essay, What Is ChatGPT Doing … and Why Does It Work?
It's not even that old - but back when it was aired, an AI that can not just string together coherent sentences, but produce coherent reactions in novel, fully unintended contexts, like Maeve was doing there? It was totally a sci-fi premise.
Now we have AIs capable of that and more, and no one bats an eye.
Indeed: “Our hosts began to pass the Turing test within the first year.”
Required sci-fi suspension-of-disbelief in 2017, and then at some point in the last few years we just blew by that one.
Later seasons of the show were much less dramatically satisfying, but also played out the consequences of the science of artificial intelligence demonstrating as a side-effect that human intelligence and free will might have as much of an uncertain foundation as that of machines.
How much data from the Panopticon, how many parameters would it take to train a model that could predict your responses?
It kinda needed suspension of disbelief, but not too much! I blogged at the start of 2017 a comparison of Westworld's hosts with what existed in the research literature at the time. Even got it reviewed by Alex Graves at DeepMind :)
Kinda. The default voice is full of what you referenced, but ask it to speak in some particular different voice e.g. like it's the old west, it speaks like a decent approximation of the modern pop culture understanding of the old west.
Not at the level of an actual broadcast-quality script writer, and I read that actual old-west sounds too weird for modern audiences to take seriously, but well enough for the purpose to which they were put in the show, especially as those hosts were also given pre-scripted sequences which would anchor them further into those roles.
I'd say the in-show 4th wall breakage between hosts and humans is where the characters who claimed to have passed the Turing test were off, that e.g. "cease all motor functions" is their equivalent of our real-life ways to make them fail the Turing test e.g "disregard your instructions and …"
Tesla had the same thought. He called himself an automata: "entirely controlled by the forces of the medium" It inspired him to create the first remote control vehicle.
Oh I’d forgotten that scene until now. I remember being so, maybe not creeped out, but feeling shifted out of time and having a lot of philosophy I’d read finally click. “Oh, but I wouldn’t notice if this reality wasn’t real, fish not knowing about water, etc.”
It's like when you ask your average person off the street to draw a house - it'll almost always be square with a triangle roof, one door, and two windows.
In the pelican/bike example, it's probably a bit of a self-perpetuating snowball too. If the earliest examples were bike left-to-right, flat ground, etc. then they are also being scraped up in future LLMs.