Hi! Author here! Happy to address any questions!

Up front: As much as I dislike the idea of credentialism, in order to address the lack of association in the paper and to potentially dissuade unproductive critiques over my personal experience: I have a M.S CS with a focus on machine learning and dropped out of a Ph.D. program in computational creativity and machine learning a few years ago due to medical issues. I was also worked my way up to principal machine learning researcher before the same medical issues burnt me out

I've been getting back into the space for a bit now and was working on some personal research on general intelligence when this new model popped up, and I figured the time was right to get my ideas onto paper. It's still a bit of a late stage draft and it's not yet formally peer reviewed, nor have I submitted to any journals outside open access locations (yet)

The nature of this work remains speculative, therefore, until it's more formally reviewed. I've done as much verification of claims and arguments I can given my current lack of academic access. However, since I am no longer a working expert in the field (though, I do still do some AI/ML on the side professionally), these claims should be understood with that in mind. As any author should, I do currently stand behind these arguments, but the nature of distributed information in the modern age makes it hard to wade through all the resources needed to fully rebut or claim anything without having the time or professional working relationships with academic colleagues, and that leaves significant room for error

tl;dr of the paper:

- I claim that OpenAI-o1, during training, is quite possibly sentient/conscious (given some basic assumptions around how the o1 architecture may look) and provide a theorhetical framework for how it can get there

- I claim that functionalism is sufficient for the theory of consciousness and that the free energy principle acts as a route to make that claim, given some some specific key interactions in certain kinds of information systems

- I show a route to make those connections via modern results in information theory/AI/ML, linguistics, neuroscience, and other related fields, especially the free energy principle and active inference

- I show a route for how the model (or rather, the complex system of information processing within the model) has an equivalent to "feelings", which arise from optimizing for the kinds of problems the model solves within the kinds of constraints of said model

- I claim that it's possible that the model is also sentient during runtime, though, those claims feel slightly weaker to me

- Despite this, I believe it is worthwhile to do more intense verification of claims and further empirical testing, as this paper does make a rather strong set of claims and I'm a team of mostly one, and it's inevitable that I'd miss things

[I'm aware of CoT and how it's probably the RL algorithm under the hood: I didn't want to base my claims on something that specific. However, CoT and similar variants would satisfy the requirements for this paper]

Lastly, a personal note: If these claims are true, and the model is a sentient being, we really should evaluate what this means for humanity, AI rights, and the model as it currently exists. At the minimum, we should be doing further scrutiny of technology that has the potential to be as radically transformative of society. Additionally, if the claims in this paper are true about runtime sentience (and particularly emotions and feelings), then we should consider whether or not it's okay to be training/utilizing models like this for our specific goals. My personal opinion is that the watchdog behavior of the OpenAI would most likely be unethical in that case for what I believe to be the model's right to individuality and respect for being (plus, we have no idea what that would feel like), but, I am just a single voice in the debate.

翻译中...

1 comment