A lot of that complexity comes from them being living cells, optimized for and functioning in a different environment than our silicon-based machines. We don't need to model it all.
(Though we do need to pay attention to evolution cheating by overfitting relative to what we'd consider a clean design. Some of the complexity may be doing double duty.)
Slime molds and single celled creatures can learn things despite having ZERO neurons. Neurons are built on top of an already incredibly complex machine evaluating hundreds of thousands of chemical and physical interactions per second that ALL effect how the cell works.
We aren't likely ever going to reduce that to a model as simple as the one used in machine learning, because it probably isn't that simple period.
Neurons are not "just" electrical signalling devices. They are complicated processors and systems in their own right.
> Some of the complexity may be doing double duty.
Since we have not succeeded in imitating even the most primitive brains, even though computationally we should have enough juice by now, it would seem that complexity can't be discarded at all, no?
Now, the goal there wasn't to get it to solve a maze, but rather to see how it can come up with a plan of action and adjust it on the fly. But I see no reason a variant of that wouldn't work with a traditional maze game - provided you remember this is a stateless model without volatile memory, so it needs to be fed its memory with every request.
No we haven't. ChatGPT is basically a sophisticated Markov chain. It is very good at pattern matching, but it has no understanding of anything, or its own will. People who think is even close to AGI are deluded, fooled by an elaborate Mechanical Turk.
This is also the reason why its output sounds convincing, but is very often factually wrong.
I disagree, but that's beside the point here. You yourself narrowed the scope to:
"imitating even the most primitive brains, even though computationally we should have enough juice by now"
Which is kind of weird to claim today. GPT-4 may be the strongest counterexample to date, but it's far from the only one.
Of course, you need to remember not to confuse the brain with attached peripherals. Just because we can't replicate a perfect worm or fly body, complete with bioelectrical and biomechanical components, doesn't mean we can't do better than their brains in silico.
I'd also say that some of it might be more computing power required, but much of it is us cracking the "puzzle" to it, we haven't figured out the exact right architecture/structure for creating say an AGI.
Just like with transformers revolutionising text generation and now things like LoRa and other fine tuning methods are helping us find a better solution to that puzzle, the same will happen for the development of AGIs.
GPT-4 does not "imitate" a "brain"; it does not function like a brain, nor is it even really analogous to a brain in any useful sense. What it imitates is human speech.
"Imitating human speech" is not a trivial thing. You can't do it by a lookup table, or by a Markov chain. Not properly, not in open-ended, unscripted situations. It requires capabilities and structures that, if they aren't a world model and basic abstract reasoning skill, then they at least start to look strikingly similar in practice. This is where we are with GPT-4. It doesn't imitate speech. It imitates reasoning.
And if it walks like a duck, and quacks like a duck, ...
GPT-4 is a good example because it's pretty clear that the model isn't merely a stochastic parrot (or, if it is in some sense, then in that sense so are we). But it's not the only game in town. Not all generative transformers deal with language. All seem to be powerful association machines, drawing their capabilities from simple algorithms in absurdly high-dimensional spaces. There are many parallels you can draw to brains here, not the least of which is that the overall architecture is simple enough and scalable, that it's exactly the kind of thing evolution could reach and then get railroaded into building on.
