In Geoffrey Hinton's Cambridge discussion[0] he makes the argument that self-learning
systems could share their weights among many other copies. I refer to this
as "consensus learning".<p>I think this relies on the flawed assumption that weights can be shared.
Consider two systems, e.g. robots, that are working on a task, such as
one I'm familiar with, tightening lugnuts when changing a tire. Each robot
is not EXACTLY like any other. When learning a "tacit knowledge" task,
each robot will have different feedback due to such items as wear in
the joints. The updated weights in each system diverges, even on the
same task.<p>Even in knowledge-only tasks, if each system is trying to learn and they
are in different environments (e.g. reading books), they will develop weights
that are local. Given a dozen robots reading books in parallel it is not at all
clear how their weights could be combined.<p>Consider robots reading philosophy books or mathematics books. They
will encounter 'definitions' which have subtle or maybe not-so-subtle
differences (e.g. satire). Even such obvious ideas as "parallel" lines can
be a source of definitional argument. These new definitions change the
mapping in your Word2Vec, effectively changing the very basis of the
systems knowledge map.<p>I ran into this issue when working on Expert Systems years ago. Expert
Systems that kept extensive state which differed based on the environment
of their use would get "confused" when importing a rule developed elsewhere
by a different "expert".<p>It is my conjecture that any system that remembers and acts on prior knowledge
cannot reliably engage in consensus learning schemes. The new knowledge
has to be abstracted and taught.<p>This is the reason we need to teach. We have to abstract over our knowledge
and extract the high level goal states we wish to communicate. Students need
to self-adapt to these goal states.<p>You might disagree but methinks that proves the point. :-)<p>[0] https://www.youtube.com/watch?v=sitHS6UDMJc