>A world where people just have their whims catered to is not a utopia.
I understand where you're coming from on that, but there are a ton of people living in slavery, or in unsafe working conditions, or with food insecurity, or dying of preventable disease.
There will always be something to strive for, even if it's made up. You think that the best football players in the world are doing something real? No, it's a made up game. We'll make up more games.
I agree with your points. My problem is that I don't see any way to "stop" AI development at a level just below human capability where you get most of the benefits but still have humans firmly in control.
Regarding entertainment jobs like sports star, musician, artist, actor, etc., the problem is those are professions where, for the most part, only the very best can make a living. Nobody is that interested in seeing the matches of the 324th best tennis player in the world, despite that person being in the top .01% of tennis players globally. Similarly, only a tiny percentage of actors "make it", everyone else does it for a few years and then leaves because they enjoy eating, or they accept poverty wages for a long time.
I get pretty scared when I hear the AI utopianists rattle off all the benefits that are coming (and when they play down the risks) because it feels clear to me that they really haven't wargamed out what society would look like with very powerful AI.
The game is made up, but the effort, and process to be one of the most skilled sports players is still real. And sports players would probably feel something missing life if AI could replace them too
Computers surpassed humans at chess nearly 30 years ago. Yet human chess is more popular than ever, and people still dedicate their life to improving at chess.
As a more silly comparison, forklifts didn't stop people from weightlifting.
Chess is inherently a leisure activity. The incentive for playing chess is just to be better at chess. But things like woodworking, typesetting, and welding, while all possible to do recreationally, are not leisure activities. They are effort used to produce something. The desire is always there to engage with the thing or idea, but if the resource cost is far too high or the knowledge required is more than one person can reasonably learn the effort will not be expended. A leisure activity will always persist so long as someone knows how to do it because it exists for itself. But many other activities exist in service to something else, and if that something else ceases to exist then the activity will as well. You lose entire domains of useful cross-disciplinary knowledge that way.
We do have entry and exit conditions. We can disrupt it and study the dynamics of the disruption. We have a good sense of the many mechanisms at play that undergird it.
What we lack is a full map of the exact process from sensation to consciousness across all modalities. And I’m afraid when it comes to interception, we can’t unless we observe every one of the 36 trillion or so cells in a human body, as well as the 36 trillion or so symbiotic and commensal microbes, continuously, all the time.
But I keep finding it astonishing that the claim that we know nothing about consciousness gets bandied about. We know a lot. We don’t have a grand unified theory. The lot we know is definitely split across many levels of evidence and hard to follow, let alone arrange. But this isn’t a black box. It’s a grey box, meeting an even more transparent box that is the LLM, where we do know what the guts are made of, and can interfere at every step in the chain of steps that constitute their dynamics.
Comparing the two, we know there’s a level of similarity in that information gets broken down via a neural network. That similarity was sought.
Since then though, neuroscientists have gone and shown that:
1. The other half of the cells in the brain, the glia, are at least as important as the neurons in cognition and consciousness
2. That interoceptive feedback and feelings are critical drivers of conscious experience
3. Evolutionarily, we know all cells can “learn”, and well before there were neurons or glia or brains, every cell evolved an internal clock that allows it to entrain to external solar and (depending on the species) lunar rhythms.
4. In the last few decades we’ve seen how synaptic activity is shaped and driven both by astrocytes and the circadian clock.
All this is showing us that the abstraction from the 1950s that current neural networks are built on were incomplete.
Whatever these components to do give rise to consciousness in biology (and we’re a long way from done solving this), we certainly wouldn’t imagine with all these modules and mechanisms missing, just maxing on one type of information flow in the brain would give you consciousness.
I’d urge you to not keep insisting consciousness is a total mystery. It’s not, and even your AI model of choice will be able to point you to all the mechanistic evidence we have that whatever it is, it isn’t just neural nets.
None of this is an explanation of what consciousness is, either physically, logically or philosophically. (Maybe some physically)
Fundamentally it comes down to an objective decision about what that is. If you say it is “feelings” based on inputs and feedback mechanisms from the brain, then we can do the philosophical discussion around that.
“If the claim is that consciousness is limited to those with a specific type of cell behaving in a given way” thats just a coping mechanism hoping to use a mechanical definition to shield you from the reality that eventually all of these inputs, outputs and feedback mechanisms can be reliably reproduced in a different form.
I’m baffled: how does the fact that each cell has an internal clock that continues to tick even when you take away all external signals not impact what consciousness is logically?
If the time order driving behavior is driven by an internal timekeeper, that is logically relevant to the behavior you’re interrogating. If, on the other hand, all component systems depended on an external clock, that logically points to a completely different dynamic process.
As for the philosophy of it all… it’s true I make no comment on it. I’d rather look at the physical substrate and see what it does, and compare it to behavior, than try to fit a philosophy that originated before we had such high resolution knowledge of the system. They are blind to these facts, just because they were written up too early.
I utterly reject this doesn’t show you anything logically about consciousness. If I can show you that the molecular dynamics of each cell organize to anticipate the dawn, and do so even in constant darkness (and all this is well established.. there was a Nobel for the field in 2017) how is that not logically distinct from a system that is outside of regular time, needs to check a clock to locate itself in time, and then do whatever dynamics it does to solve the problem at hand?
As for calling this a coping mechanism… that’s a bit rich coming from someone who seems to have very little idea of the molecular and cellular biology but seems to want to hold to the belief that we’ve solved consciousness with LLMs. We can both fling that accusation about. Seems more productive to compare the physical dynamics and see what’s different and what’s similar, no?
> I’d urge you to not keep insisting consciousness is a total mystery.
But don't want to get into the philosophy of it. I'd urge you to consider the definition of conciousness (as specified by the original question) wasn't just about biology. And it is 100% very difficult to define.
> We do have entry and exit conditions. We can disrupt it and study the dynamics of the disruption.
We've known life and death for quite some time. Even from the biological perspective, there's nothing new there in the largest sense. All we've done is refine it. Animals are aware of it.
> but seems to want to hold to the belief that we’ve solved consciousness with LLMs
Quite the contrary, much like everything else, the more we learn, the more questions we have.
> Seems more productive to compare the physical dynamics and see what’s different and what’s similar, no?
ok but....
> how is that not logically distinct from a system that is outside of regular time, needs to check a clock to locate itself in time, and then do whatever dynamics it does to solve the problem at hand?
We only check clocks because we have an internal molecular dynamic that directs us to do so, and it is tied to a similar system as everything else.
> But don't want to get into the philosophy of it. I'd urge you to consider the definition of conciousness (as specified by the original question) wasn't just about biology. And it is 100% very difficult to define.
No. I don't engage in the philosophy of it because most of the consciousness philosophy that gets bandied about and is familiar to folks is from 20-30 years ago, and rest on knowledge that is significantly less granular about details.
You don't, for instance, demand physicists need to engage in the philosophy of gravity do you? You demand theory. Neuroscience does the same of consciousness, and there are definite theories with support coming at the problem from different ends.
Insisting it's a mystery without engaging with the facts is not what I'm interested in.
> We've known life and death for quite some time. Even from the biological perspective, there's nothing new there in the largest sense. All we've done is refine it. Animals are aware of it.
Not quite sure what this means, but as with consciousness, I'd say life also isn't a complete mystery, we have made great strides understanding it as it is and it's evolution in the past few decades, and while the philosophy of life, mired in assumptions from past decades, remains hopelessly stalled, the science of life has simply moved on.
> We only check clocks because we have an internal molecular dynamic that directs us to do so, and it is tied to a similar system as everything else.
Is this tongue in cheek? Nothing about our internal molecular clock tells us to check clocks. For most of life, there were no manufactured clocks, remember? They are a relatively recent invention.
Whether human consciousness exists on neural nets or otherwise doesn't disallow an ANN in a particular configuration from being conscious. You might as well argue that human consciousness requires biological neurons, so artificial consciousness can't exist.
Odd. Why don't the details of how consciousness arises in one system inform your judgment of whether it can exist in a different system that only has partial structural overlap? Seems wildly convenient. Where else in science can you show me such a comparable situation in how you define properties?
> Why don't the details of how consciousness arises in one system inform your judgment of whether it can exist in a different system that only has partial structural overlap?
A light bulb doesn't need to do fusion to make light. An airplane doesn't need to flap its wings to fly. An ANN doesn't need to use a brain's structure to think.
You haven’t proven an LLM has thought. I can measure the spectral properties of light from both a bulb and the sun.
I can measure the dynamics that go with thought in a human.
I can measure some dynamics in an LLM. From all we know, there are huge differences, not least that you can literally turn off an LLM, whereas every process but death in biology shows continued activity even when the organism “looks” off,
What you haven’t shown me is that what the LLM does when it is on is “thinking”. Given that the dynamics are different, the substrate is different, and one can be fully turned off and the other cannot, why are these two things the same?
Give me any problem that previously required human thinking. Any. That can be solved by LLMs. All you have to do is provide a single counterexample.
> not least that you can literally turn off an LLM, whereas every process but death in biology shows continued activity even when the organism “looks” off,
There is no reasonable definition of thinking that requires that it can't be paused.
> why are these two things the same?
I said they aren't the same, just as a bulb isn't the same as a star. An LLM can think, just as a human brain can think. A bulb can produce light, just as a star can produce light.
>Give me any problem that previously required human thinking. Any. That can be solved by LLMs. All you have to do is provide a single counterexample.
Theres an itch deep in my butt, but I'm in an important meeting and can't leave the seat. Nor can I openly scratch the area.
I can tell myself these things, focus on the itching, think about how itching is felt by the body, think about the social consequences, and make it feel more bearable.
Please tell me which LLM has solved this problem so I can not outsource the issue to it. I eagerly await your answer.
> There is no reasonable definition of thinking that requires that it can't be paused.
Is there a reasonable definition of thinking where the words "paused" and "turned off" aren't treated the same? Can you do that kind of thinking and get back to me?
> I said they aren't the same, just as a bulb isn't the same as a star. An LLM can think, just as a human brain can think. A bulb can produce light, just as a star can produce light.
An LLM can produce language. You are yet to show me it produces thinking.
Your claim is like saying the bulb produces light, so it also goes through fusion.
The LLM will tell you how to solve it. If you mean the LLM should modify your thoughts, no human could do that for you either, but an LLM agent can just as easily as a human figure out how to build the brain computer interface to make this possible in the future.
> Is there a reasonable definition of thinking where the words "paused" and "turned off" aren't treated the same?
No reasonable definition of thinking requires that thinking can't be paused or turned off. Please state your point more clearly.
> You are yet to show me it produces thinking.
Every problem that previously required human thought can be solved by LLM. QED.
> Your claim is like saying the bulb produces light, so it also goes through fusion.
No. How many times do I have to explain that the mechanism of thinking is not the same, just as the mechanism of producing light is not the same? You need to think to understand this, which is something that frontier LLMs have no problem with, but which most humans struggle with.
You’ve shifted your goal posts. You asked me for a problem I can solve with thinking. Not a problem another human can solve for me by their thinking. I have given you exactly what you asked for. An LLM cannot solve it. A human can’t solve it for me but can solve it for themselves (or not be capable of it), it varies.
Whatever you thought you were going to prove with your question, I do hope you’ll dwell on what my example tells you about what thinking is. It isn’t a string of words only. It can be that, but it’s more.
> No reasonable definition of thinking requires that thinking can't be paused or turned off. Please state your point more clearly.
It does. Please define thinking however you want. I’ll show you that it’s a requirement.
> Every problem that previously required human thought can be solved by LLM. QED.
Again, this is the same superficial thinking. Firstly, as my example above showed, no. There are other examples. It requires human thought to cook. No LLM can, nor is smell digitizable in an equivalent way to humans, and yet, cooking is one place where having a human nose is determinative to making appetizing food. Your QED is logically absurd, but is also resting on an assertion that’s flatly untrue.
> No. How many times do I have to explain that the mechanism of thinking is not the same, just as the mechanism of producing light is not the same? You need to think to understand this, which is something that frontier LLMs have no problem with, but which most humans struggle with.
When your “proof” for thinking is the output, I will also compare it to the output, light. My analogy shows the idiocy of your argument, and you don’t want to sit with its implications.
LLMs replicate human language. And many tasks human language can do, LLMs can do or help with. Thinking is not just producing word strings. We think about our family. We think about our bodily aches and pains. We think about how best to do various physical tasks. All this we do sans words, and LLMs don’t do. To call their internal dynamics “thinking” is silly wishful thinking by those who don’t have the ability to go read and understand biology.
And it’s a sad statement of how little you think of your own mind that you won’t explore facts that are outside your expertise that will actually teach you how your mind works, and the wondrous complexity underlying it, and the potential it has to keep doing new things.
I’m going to stop engaging at this point. You seem convinced LLMs think. I wish you the joy of that belief, whatever it is. Religion is always absurd, and this new one is no different.
Remember the problem we're discussing is whether these LLM systems can think. Your deliberate misinterpretation of a question does not make you win that argument.
An LLM can solve problems for itself in just the same way that a human can solve problems for itself. It cannot directly rewire your thinking in the same way that another human cannot.
You have yet to show me a problem that can only be solved by human thinking.
> When your “proof” for thinking is the output
What other way can we show thinking? From the input? Obviously not.
> My analogy shows the idiocy of your argument
It shows the idiocy of your argument. The output is thinking, just as the output is light. You are the one who continues to mistake the output for the mechanism.
> Religion is always absurd, and this new one is no different.
Your assertion of magic in biology that can't be replicated artificially is indeed absurd and one that almost no scientist would agree with.
Not GP, but one of the challenges with debating whether LLMs are "conscious" is that we don't even really know what it means for a human to be "conscious", or even if consciousness is experienced by other humans the same way it is for ourselves.
What we do know: neurons carry electrical impulses across their synapses to trigger other neurons to fire, and more frequently used synapses are strengthened while infrequently used ones are pruned. This is not all that dissimilar to how a multi-layer perceptron is trained: it's floating point numbers in a big matrix rather than biological structures and electrical impulses, but there is still that element of frequently used connections being strengthened and infrequently used ones being pruned.
What we hypothesize but do not know: there is a thin brain structure of grey matter called the claustrum that has tendrils that reach into nearly every other brain structure. In many ways, this is similar to the attention mechanism of the transformer architecture. It is hypothesized that this may be the seat of consciousness, owing to experiments where electrical stimulation of the claustrum caused patients to immediately lose consciousness. However, there is no way to prove this, owing to the difficulty of otherwise removing or disabling the most connected structure in the brain and observing its effect on consciousness without permanently killing the patient.
Beyond that, we don't know much. I've got a family friend that's been a practicing therapist for 50 years, and I asked him what was the most interesting observation he made in his career. It was that "Everybody experiences the world in a different way, and yet everybody assumes that everyone else experiences the world the same way they do."
Personal note: That principle has been the bane of my autistic existence. People sometimes seem literally incapable of understanding that other people even can be different.
> What we do know: neurons carry electrical impulses across their synapses to trigger other neurons to fire, and more frequently used synapses are strengthened while infrequently used ones are pruned. This is not all that dissimilar to how a multi-layer perceptron is trained
ANNs have been inspired by biological processes, but in practice you have to squint very tightly to see the similarity. Biological neurons are multiple orders of magnitude more connected than the nodes in an ANN, plastic in terms of their connectedness and continuously learning, and their activations are also affected in complex ways by the levels of various transmitter molecules in the brain.
You are saying the AI doesn't a some property that you don't have any definition for, not even a working definition. People will disagree on whether a cat or a baby is conscious, they're not debating what a baby or cat is. They're debating this term. You might as well be debating whether an AI is a blorb or not, you have just as good a working definition of blorb as consciousness.
I believe you should look up the work of Cameron Berg before making statements like "an LLM is" or "an LLM isn't". Empirically defining all this stuff is very difficult, and making a definition that covers all beings that can exhibit conscious behavior is much more complex than a face value examination would reveal.
Well, define "feel" - in a context that could be also applied to machines. It seems to be (if we really had to employ it for the sake of discussion) "getting signals from peripherals, orienting action".
the question was: what is the endgame for the stated "second class labs" strategy of distilling their frontier competitors then undercutting them on price?
Yes yes, we all understand the game-theoretic race-to-the-bottom you're describing here. Somehow despite linux being FOSS it still powers most of the important computing in the world. Can you explain how that works despite it being free? Once you understand that case I think you'll understand the game-theory behind how large projects can exist in the absence of traditional IP protection.
the obvious difference is the massive scale of data and compute required to develop and evolve these models, and the costs they impose on those building them.
Smaller budgets, slower improvement, less risk. They're not entitled to profits if that business model isn't sustainable. They're not entitled to a change in IP laws to protect their business model. They're not entitled to growing that fast.
Parses emails or other sources, extracts numbers, correlates different transactions, web search, asks questions, stores notes (regex based, very simple).
The hard part is getting good data, I'm sure that lexus nexus or whoever can get API access to my bank account and all my credit cards, but I can't. Email turned out to be the best way for most of my providers. Managed to avoid 2factor auth so far, but it will suck when I need it.
We've got integrations with major UK banks. Curious if you'd like to use a polished product or be more interested in bank-feed-as-an-API type of use case? What banks do you use?
Credit union atlantic. The possibility of you being able to profitably support small regional credit unions is pretty much 0. Maybe with a general browser use AI and an SMS portal. Also you'd need to bypass anti-bot protections, maybe solve captchas.
In my opinion that is the thing holding back almost all of these accounting products that make consumers lives better. I've solved it because all my cards and bills happen to support email, so I can use that as my source of truth for all my credit cards.
The token is stored on a LUKS encrypted drive on kwallet. You'd have just as much luck getting my password from my firefox installs sqlite DB. This is also how email clients in general work.
Also this isn't really an agent. At least not a long lived one. Each email or transaction gets it's own session, the llm can make a few tool calls but must emit json as the final result. Very very short context lengths, very predictable results.
The bot does not have access to my passwords, there are pre-defined scripts that fetch the data that the run an LLM call for each transaction discovered.
There is a big difference between capturing value and creating value. 20 developers can build and maintain a facebook or reddit clone, so why aren't they kicking the asses of the big companies with the much higher costs?
First mover effects, network effects, brand recognition, corruption, rent-seeking over the assets created by the aforementioned
Like, yes the main point of leverage obviously isn't software quality, but also, if AI can't cause one person to be able to solve those other problems at some scale that's currently not possible with no other human help, those problems become problems someone else could also throw the AI at. If running a business relies on having some point of market leverage over anyone who pays your business for something, the reduction of the cost of independently gaining that leverage necessarily means the reduction of the value of your business
I guess another possible means to a single-person company with a billion dollar valuation is hyperinflation
I understand where you're coming from on that, but there are a ton of people living in slavery, or in unsafe working conditions, or with food insecurity, or dying of preventable disease.
There will always be something to strive for, even if it's made up. You think that the best football players in the world are doing something real? No, it's a made up game. We'll make up more games.
reply