Suppose the neural nets were constructed as lisp list-based data structures rather than matrices.<p>This allows quite a few interesting ideas:<p>1) it is possible to write functions and macros that can self-modify the network<p>2) if there are subnets that have rich internal structure but sparse I/O that indicates possible "clustered concepts"<p>3) sections of the neural nets can be replaced by functions taking incoming connections and returning multiple-value outputs.<p>4) explicit programs, such as expert systems, can be embedded in the network<p>5) "long term memory" can be explicitly kept in the structure, served up by functions<p>6) "backward reasoners" can take incorrect output, walk back through the data structure and replace sections contributing to the incorrect output (aka debugging)<p>7) explicit function calls can be inserted anywhere to do things like display information, call interface functions e.g. extract sensor data and insert the result into the process. This would be useful for robot joint control, robot hearing, etc.<p>We have artificially limited ourselves to these "black box" matrix-based solutions. The list-based solutions can do the same matrix operations but with a much richer data structure.
I think AI will eventually need symbolic processing, for exactly the same reason that human intellectuals need, and engage in, symbolic processing.<p>Imagine AI evolves to human-like intelligence. It wants to do math. How does it do that? Pencil and paper? Clumsily interfacing with some computer algebra system? It would be better for it to have access to an integrated symbolic processing system in its own "head" where it can try ideas and get exact answers.<p>Sufficiently advanced AI (smarter than human) could make advances in symbolic processing.<p>AI is solving certain problems now better than we were able to do with symbolic processing, but that's not necessarily a limitation of symbolic processing, but of our abilities.<p>Things we don't know how to do symbolically, AI might be able to figure out how to do symbolically. That could be computationally cheaper than neural nets, and produce more accurate results within its niche. AI could integrate that to improve itself.<p>Symbolic processing has something to say.<p>There is a huge abstraction inversion in training hundreds of milions of neurons to do some simple symbolic processing problem that could be done with a small program using a few kilobytes of RAM.
None of the above require Lisp, as none of them are impossible with matrices and the current schemes used.<p>For most of those, doing it with "lisp list-based data structures" would be a 2-3 orders of magnitude performance hit (and of course the unability to properly use the GPUs).<p>At the level of data we're discussing, Lisp lists (aside from the performance issue) would be just as opaque as any matrix, and just as difficult for any of the operations suggested. The matrix vs lisp represenation part would be the most trivial part of the whole thing.