Why deep learning neutral networks are so effective?

The good news of this morning! Neural networks (one speaks of deep learning) model the world much better than one might expect on basis of mathematical arguments alone. This is a real mystery. Deep learning means AI systems with large number of hierarchy levels: programs calling programs calling... is the first intuitive idea of AI outsider like me.

The solution of the puzzle proposed by physicists is elegant. The physical world is much much simpler than mathematicians - wanting to be as general as possible - assume! Simplicity means among other things holography and hierarchical structures and deep learning relies on hierarchical structures. It would be amazing if AI and physics finally could meet each other! For more details see the article of Lin and Tegmark . See also the remarks of Ben Goertzel.

Holography and its strong form

The Universe is indeed very simple according to hololographic theories. For instance, in TGD not only holography but strong form of holography holds true. The quantum and classical data assignable to string world sheets and partonic 2-surfaces dictates the dynamics of 4-D space-time surface. This effective 2-dimensionality of dynamics means enormous simplification of the quantum physical world from what it could be. For instance, preferred extremals defining space-time surfaces satisfy infinite number of conditions stating vanishing of certain Noether charges.

This extreme simplicity is lost when the sheets of the many-sheeted are lumped together to obtain the space-time of general relativity and standard model and effective classical fields are sums over geometrizes classical fields associated with the sheets. In biological systems however the dynamics of many-sheetedness comes manifest and the actions of single sheet need not be masked: things get simple in this kind of situation.

Various fractal hierarchies

Holography need not the only reason for the simplicity. The possibly physical world of TGD has hierarchical fractal structure: length scale reductionism is replaced with fractality. Dynamics looks more or less similar in all zooms and this simplifies the situation of mimicker enormously. There are hierarchies of space-time sheets topologically condensed on larger space-time sheets, hierarchy of p-adic length scales defined by primes near powers of two (or more general small prime), hierarchy of Planck constants, self hierarchy. p-Adic length scale hierarchy allows extremely simple model for elementary particle masses: one might perhaps say that one does not model the mass of "real" particle but its cognitive representation about itself in terms of p-adic thermodynamics relying on conformal invariance. The hierarchy of Planck constants means fractal hierarchy of zoom-ups of system: dark matter phases assignable to quantum criticality would be crucial for the understanding of living systems.

These hierarchies also define hierarchies of measurement resolutions making possible abstraction, getting rid of details at the level of conscious experience and behavior. The hierarchical structure would be especially important for conscious mind. Self has subselves which it experiences as mental images and is mental image of higher level self. Goal hierarchies mean a lot of structural restrictions making it easier for artificial intelligence to mimick conscious systems.

Quantum states realize finite measurement resolution themselves

Conceptualization means hierarchies and one can say that TGD Universe performs this conceptualization for us! In fact, one can say that quantum state provides its own description. This implies that finite measurement resolution is not a property of description of quantum state but of quantum state itself! For instance, the larger the number of partonic 2-surfaces and string world sheets is, the better the "half-discretization" of 4-D space-time surface by these 2-surfaces is, and the more precise is the conscious experience of system about itself. For instance, magnetic flux tube networks with flux tubes accompanied by strings and with maximally entangled at the ends of nodes would give rise to a universal proprioception. The experience about 3-space would emerge from entanglement rather, not the 3-space as some colleagues fashionably argue.

Simplicity in cosmology

This extreme simplicity is most dramatic in cosmology. The microwave temperature is essentially constant. This cannot be due to the causal interactions but reflects something deeper. Inflationary scenarios are one attempt to explain this but have not led to a breakthrough. A more radical explanation is that macroscopic quantum coherence even in cosmological scales is possible at the space-time sheets of cosmic size scale with large value of Planck constant characterizing phases of ordinary matter behaving like dark matter. The key idea is generalization of point-like particle to 3-surface: particle and 3-space are one and same thing. Particles as 3-surfaces can have even cosmological size.

See the chapter TGD Inspired Comments about Integrated Information Theory of Consciousness.