@DrTomFroese Can you say what you define as "mind" and in what way an LLM is like such a mind? Personally, I think this is just one big fat category error. Convince me otherwise?

@yoginho I used the word “intelligences” precisely to avoid getting into definitional disputes. What I am interested in is the capacities, not the labels.

@DrTomFroese So, we built machines that pick up regularities in our communication - written, auditory, and visual - and reproduce them convincingly (kind of, most of the time). What do we learn about intelligence from this?

@yoginho I think it’s about more than just reproduction and we can learn a lot from the comparisons. I spell out my view in more detail in this pre-print. Curious to hear your thoughts! https://osf.io/preprints/psyar...

@DrTomFroese Nice try. But I don't understand why the "AI dilemma" is a dilemma. Isn't it obvious that you can achieve linguistic competence without making any sense at all? I don't understand why you claim that LLMs are "sense-makers." There is no meaning in the algorithmic domain.

@DrTomFroese The "blind spot of embodied cognition" is not a blind spot, but the insight that meaning or sense only come out of being a precarious self-manufacturing organism. Algorithms just copy the patterns that are present in our communication. Given the size of these networks ...

@DrTomFroese ... and the size (and human curation) of their training data, why should it surprise us that they are proficient at mimicking human communication without incorporating any meaning themselves?

@DrTomFroese The category error that is happening here is that "proficient at communication" equals "intelligence," when intelligence is mostly about *realizing relevance*. The frame problem (problem of relevance) not only remains unsolved, but basically untackled in AI.

@DrTomFroese So: we should indeed reconsider sense-making. But not by claiming senseless algorithms make sense. Instead, there is an opportunity to realize here just how much richer lived experience is that being good a language games.

@yoginho What you say here is to me a case of embodied cognition in dramatic retreat from its foundational claims: "Isn't it obvious that you can achieve linguistic competence without making any sense at all?" It is definitely a different tone from Dreyfus et al.

@DrTomFroese The insight that framing problems and solving them are fundamentally different activities goes right back to the origin of AI research. So, I'd argue that your concept of "linguistic competence" (a.k.a. "convincing mimicry") vs that of Dreyfus ("a.k.a. conveying meaning ...

@DrTomFroese ... through communication) is the basic problem here. You are falling for the magic trick that is pattern repetition based on high-dimensional correlations in human-curated data sets, and call it "linguistic competence." There is no competence there. Just statistics.

@DrTomFroese It baffles us because, as humans, we are famously ill-adapted to work with high-dimensional correlations. Any sufficiently advanced technology in this domain will naturally appear like magic to us. So we attribute all kinds of agency, meaning, or even intelligence to it.

@DrTomFroese The whole point of embodied cognition is that we urgently need to get out of our heads, if we want to reconnect to reality. What we are having here (and that includes your argument in the preprint) is classic mistaking map and territory.

@DrTomFroese I'm already tired of these claims that "AI can do X, therefore it *must* be intelligent / agentic / conscious / thinking / creative / whatever" and we'll get tons more of those over the next few months and years. And before you accuse me of making metaphysical arguments, ...

@DrTomFroese ... while expounding your "I have no metaphysics" metaphysics, please explain to me *how* an LLM is supposed to "make sense" of anything. What does that even mean? There is no self to make sense to. Just pattern mimicry? Is that what your lived experience is like? No relevance?

@yoginho Until we have a theory that can explain how meaning interacts with matter, whether in biological or more general terms, these questions will remain fundamentally open. It’s therefore advisable to keep an open mind too.

@DrTomFroese Agreed. But, then, what makes you attribute meaning to LLMs? Based on this reply, the attribution is utterly baseless.

@DrTomFroese On the other hand, we have some pretty good ideas how (a) meaning emerges in the physical world, (b) how living organization is required for such emergence, and (c) what role it plays in organismic behavior and evolution. So why not use (or at least discuss) these ideas?

@DrTomFroese This would lead to a more meaningful (yep!) comparison between an LLM's "intelligence" and organismic intelligence. And it would avoid the problem of the category error and map/territory mistake I outline above (which you elegantly sidestepped in your answer ... smooth move!).

@DrTomFroese Until you *can* explain how "sense"/"meaning" arise from an algorithmic environment, I won't take any of these arguments seriously. I have yet to see a plausible answer to this challenge. And it is the AI research community & traditional cogsci that is *not* open-minded.

@DrTomFroese I'll say it again: as long as we can't explain how meaning interacts with matter, we can't claim that LLMs make sense or have some intrinsic meaning. Any assumption to the contrary is retreating to the map from the territory. It does not get more narrow-minded than that.

@DrTomFroese In sum: attributing meaning to algorithms is pure magical thinking, for the time being, while we *do* have plausible explanations of how meaning arises in the living domain (speculative, but not magical at all). Keep an open mind, alright, but don't ignore this state of affairs.

@yoginho I don’t know of any theory that has a sufficiently compelling explanation of how meaning makes a difference in the living domain. Behavior - and especially language - is the best indicator we can go by for now, and in that domain LLMs are exceeding most expectations.