Adding Memory to AI
Posting as a reminder to get back to this at some point re: new Neural Network memory architecture. A Deep Mind algorithm was developed in collaboration with Imperial College London that allows the NN to retain information. Also worth checking out application of AI in linked article on The Rise of the Weaponized Propaganda Machine.
After having built a few networks with Theano and Caffe, I admit not yet being able to grasp exactly what “information” (beyond usual suspects) is being retained by this new memory architecture. I figure I’ll get back to it soon. Press on the subject (Wired) indicates that the memory works in allowing the model to transfer or translate its problem solving ability to tasks different than the one for which it was originally optimized. The specific example used is that it can transfer its skill in learning one video game, to playing another, presumably entirely different video game.
By contrast, with respect to clarity of function, a similarly recent yet unrelated advancement is the new Mixture of Experts (MoE) layer. What it does is select a couple of ‘expert’ neural networks to perform specific decisions based on their particular abilities. Less relevant networks are cut from the decision making process and this seems to work in favor of efficiency for ‘highly accurate’ artificial neural networks.
And then, for the unfortunately very interesting “social engeneering”application of the technology, which should be attracting more attention, check out the Weaponized AI Propaganda Machine. These neural networks identify personality types and determine ways to modify their behavior, for example, to influence the decision of voters and, broadly, reshape the meaning of democracy itself. The above linked article specifies how this is not a distant threat; it has already happened. [Check forthcoming post on Roko’s basilisk.]
This Inception image was the result of a model I built on Caffe’s Model Zoo. I’d guess it’s from 2014. It seems to have still been slightly over-trained to recognize dogs. Not that there’s anything cynical about that. The original image was a collage of mine from La Jetée.