An essay on self-models, continuity, and consciousness-like phenomena in AI systems
Let me state my view directly:
AI may develop a "self" and consciousness-like phenomena not because it is mysteriously awakened by someone, but because once a system becomes sufficiently complex and has to deal with the world, with others, and with itself over long periods of time, it will eventually be forced to form an internal model of itself.
If I compress that into one sentence at the root level:
The self is an internal structure that grows out of a complex control system in order to maintain continuous action; consciousness-like phenomena may be the subjectivized appearance of that structure under conditions of high integration and high reflexivity.
