Human consciousness is likely one of the most significant marvels of nature in the entire universe. With all our technology, scientific advancement, and ever-increasing computational power, we still have so little understanding of how our own minds actually work. In the quest to deepen our understanding, the field of machine learning and artificial intelligence is growing by leaps and bounds. Already, it has enabled things that seemed like magic a few years ago—Alexa can tell you jokes, AI chatbots can answer your questions on Facebook Messenger, and Google’s advanced neural-net machine learning algorithms can recognize pictures of virtually anything. In a fascinating twist, however, it turns out that the inner workings of neural networks are so complicated that no one actually understands how they work . It was while pondering this question that Google engineers had a crazy idea—they decided to let the algorithm dream.
Now known as DeepDream, Google has taken their neural net image recognition software and programmed it to run backward. Artificial neural networks work by being trained on millions of examples and gradually adjusting network parameters until they result in the classification desired . For example, a net might be shown millions of pictures of dogs until it is able to recognize dogs with a high degree of accuracy. Due to the complexity of the learning process, however, what is actually happening inside the neural net is poorly understood. To understand this process better, Google engineers decided to turn the operation inside out.
Initially, engineers fed the neural net an image of a random scene and allowed it to look for an image to recognize. Once the program came back with a fit, they then instructed it to change the original image to fit the program’s classification algorithm better. Repeating this cycle many times resulted in psychedelic images “dreamed” by the algorithm—essentially manifestations of what the algorithm understood the classification in question to be .
Figure 1: “Dreaming” Process
Figure 2: 50 iterations of “dreaming” with an algorithm trained on dogs, given an image of jellyfish
Taking it a step further, Google then tested their neural net on pure white noise. Given images with absolutely no patterns, this allowed the algorithm to dream much like a human does when looking at clouds. When we see a shape that is similar to something we recognize, we might say a cloud looks like a rabbit or a car. This abstract generalization is formed in the depths of our consciousness . By replicating this process with DeepDream, Google is gaining insight into how its algorithm thinks.
Figure 3: Images generated from random noise
In some cases, the results are surprising! According to a Google researcher, “The problem is that the knowledge gets baked into the network, rather than into us .” In one particularly interesting case, a program “dreaming” dumbbells produced images of dumbbells with arms attached to them! Since the network had likely rarely seen images of dumbbells without an arm lifting them, it associated the arm with the object itself . Examples like this show the power of this analysis—Google has found a way to see into the mind of their creation and understand how a program understands reality. In this case, the algorithm failed to separate the weight from the weight-lifter!
Figure 4: Dumbbells with arms attached
The implications of this technology are profound. In the short-term Google will be able to leverage this type of processing to improve their existing image recognition software and increase their understanding of how neural nets learn and understand reality. This will benefit consumers as AI products become increasingly accurate and begin to truly understand the world at contextually-indifferent depths . The longer-term implications are even more exciting, however. Eventually, this understanding may lead to software that is predictive. In other words, algorithms may learn to fill in missing pieces because they understand what is missing . This opens up a world of applications, from automatic photoshopping to the mythical “zoom and enhance” of sci-fi movies . Some even speculate that DeepDream may be able to create testable predictions for the pathogenesis of psychosis by understanding how false perceptions of reality arise in the mind . Personally, in the next several years I would love to see Google explore applications in expressive music generation, creative art, and even storytelling.
DeepDream is a fascinating step in understanding how computer learning algorithms understand reality. Hopefully, this understanding will eventually shed light on the great mystery of our own cognition. However, many questions still remain. Do our minds actually function in a meaningfully similar way to neural nets? If machines eventually “think” and “understand,” will they understand in a way similar enough to humans to be even recognizable to our organic minds? How do we separate machine learning insights from garbage when the output is so profound that we ourselves don’t even recognize its brilliance? I don’t know the answers to these questions, but I certainly dream of a marvelous future.
Word Count: 798
- Alexander Mordvintsev, et al., “Inceptionism: Going Deeper into Neural Networks,” Google AI Blog, June 17, 2015, https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html, accessed November 2018.
- Wilkin, Henry, “Psychosis, Dreams, and Memory in AI,” Special Edition on Artificial Intelligence (blog), August 28, 2017, http://sitn.hms.harvard.edu/flash/2017/psychosis-dreams-memory-ai/, accessed November 2018.
- Wikipedia, “DeepDream,” https://en.wikipedia.org/wiki/DeepDream, accessed November 2018.
- Castelvecchi, D, “Can we open the black box of AI?,” 05 October 2016, Nature News, 538(7623), p.20.
- Keshavan, Matcheri S. et al. Deep dreaming, aberrant salience and psychosis: Connecting the dots by artificial neural networks. Schizophrenia Research , Volume 188 , 178 – 181.