I'm building a little platform to experiment with embodied / real-time multi-modal learning. Basically I have an old boom box that I gutted, stuck a USB power bank and a USB charger in there, along with a Raspberry Pi. The Pi will connect to one or more cameras, one or more microphones, and a handful of other sensors to let it sense its environment to various degrees. The main ones I want to work with are accelerometers so it can sense motion, temperature, and possibly GPS.<p>The form factor was intended to allow for portability (it has a handle and I can carry it around with ease, and the Pi can run off of the battery when then AC supply is disconnected) and to have enough space to allow for positioning multiple cameras for binocular vision and multiple microphones for stereo "hearing", as well as room for the other random sensors. Oh, and I'm re-using the existing speakers for the audio out so I can work on integrating speech synthesis into the whole kit and kaboodle.<p>For the software, I'm looking at starting with a super-simple BDI interpreter for the basic "cognitive loop", using an RDF triplestore for semantic knowledge, neural network models where appropriate (object recognition for example), and then start trying to build up from there. Also looking at systems like SOAR, ACT-R, OpenCog, etc. for inspiration.<p>Of course most of the "heavy lifting" from a computational viewpoint will need to happen on a server somewhere, since a single Raspberry Pi can only do so much. So there's a corresponding server backend piece that will work in conjunction with the remote portion. For now it'll be a fairly low-end server that's physically here at home, but if/when I start needing to scale things up I'll probably switch to using cloud resources on AWS as needed.<p>All in all, the basic idea is to have an "AI bot" that is alert, observing, and (hopefully) learning all the time and that will learn more like a child learns, compared to the way we train ML models today. That's not to say that there might not be some batch mode training as part of this but I'm hoping to experiment mainly with learning modalities than can happen in real-time. There's no fully fleshed out theory that I'm working off of, but I plan to tinker with a variety of things - Hebbian Learning, Reinforcement Learning, etc. Maybe I'll learn something interesting, maybe not. But it should be fun in either case.