A Generation Later, Macross Plus' Warnings About The Dangers Of AI Only Seem More Real
It’s enigmatic villain highlights just how unprepared we still are for the emergence of true artificial intelligence.
Tristan Young @talltristan
The Anime boom in North America was a result of Trojan Horse style tactics on the part of American studios, taking bets on the appeal of some Japanese content to sneak in as much more as possible. First with Robotech, then Sailor Moon, then Pokémon. With each iteration, the marketing was more finely tuned and vociferous, the demographics more carefully mapped and targeted. It worked, and in terms of access to diverse entertainment, we are probably better of for it. After all, regardless of your thoughts on Detective Pikachu, the perpetual market onslaughts and eventual erosion of the idea of being ‘foreign’ insured that us in the West had the opportunity to appreciate the wonder of Princess Mononoke, the dazzling dystopian horror of Akira, and whatever was happening in Ninja Scroll. Macross Plus predates this normalization of anime, just a little bit. Released in 1994, it was on the cusp of the boom but distant from its epicenter. It was and remained, niche, underground, a deep cut among the anime that tried its hand at a western audience.
For those that did watch it, Macross Plus tends to hold a special place among anime. It’s wondrous animation and spectacle proved engrossing to a younger age. The kinetic and pressuring nature of the angular dog fights, and sense of towering scale typical of the Macross saga was technically impressive to older audiences. The maudlin and theatrical melodrama of its central love triangle oriented story always happily avoided being overly lame, and instead was populated by characters and decisions that were empathetic. The original pop score is so infectiously hypnotic that it was actually a narrative point to the story. Macross Plus is and was a lot of things to a lot of people. One could gush over the pure ecstasy of its synthesis of music and action, or the nuanced ebbs and flows of which characters were worth rooting for or despising. Even with that, It’s Macross Plus’ most cerebral and imperceptible elements that are truly worth diving into further. Like all good Sci-fi anime, the film deals in concepts of emergent artificial intelligence. Throughout it’s four-part mini series it looks at the idea of a truly sentient machine, but one that doesn’t understand our concepts of morality, and takes it to it’s most logical and terrifying extreme. In doing so Macross Plus revolves around the kind of dilemma that may be closer to what AI ethicists warn than any other of its contemporaries. It did so a solid 20 years before most of us even understood its implications.
Macross Plus moves pretty quick and towards the end, weaves several seemingly divergent story lines together. A lot of key expositional narrative is also offered amidst some (warranted) emotional hysteria. As such, figuring out exactly what has gone wrong towards the climax is a little confusing. A quick break down: as childhood friends turned rivals Isamu and Guld contest over which of their transformer fighter jets will be funded as the next generation of defence forces, they too are fighting over the affection of Myung, their long lost friend and ostensibly the cause of their falling out. Myung is the producer of a universally famous AI pop super star called Sharron Apple. To her billions of fans Sharron is alive and sentient. In reality, it is all a façade, all of Sharron’s emotions are fed to her by neuro transmitter that originates from Myung. She is the true emotional core of Sharron Apple. As Isamu and Guld take to the skies and the streets fighting over the project and Myung, she struggles with her feelings for both and the growing resentment of playing a role in hiding Sharron’s secret.
While this is going on a rouge scientist secretly installs illegal and experimental software into Sharron that gives her actual sentience. For the first time, Sharron is truly alive and with real emotions. The problem is those emotions are based on the deepest subconscious layers of Myung, an emotional core that even she cannot yet reckon with. What does that emotional core tell Sharron? That she is deeply, madly in love with Isamu. She will do anything to make him happy. Through an unfiltered, and morally absent analysis of how to make him happy, she determines that what Isamu desires most is the ultimate thrill. So Sharon hijacks an experimental AI controlled fighter jet, tries to kill Isamu with it and threaten much of Earth. Sharon’s intentions for Isamu were derived from the id of Myung, who truly loves him. So what when wrong here?
To answer that question, consider the old irony laced fables of finding a genie in a bottle or a monkey’s paw. If one’s wish was simply ‘make me a club sandwich’ a person may be surprised to find themselves actually turned into a sandwich. Clearly this is not what they wanted, but without other context, that is what they asked. Now turn to modern day ethicists in the field of AI for a more in depth analysis of this conundrum, and how it relates to what happens in Macross Plus. Over at Vox, Dyllan Mathews postulates how an AI, despite having all the requisite intelligence and problem solving skills, is not likely to think about things the way we do.
They are incredibly good at figuring out through trial and error how to achieve a human-specified quantitative goal. But humans aren’t always good at specifying those goals, and AIs are not good at distinguishing between reasonable interpretations of human instructions and unreasonable interpretations.
Suppose you’re the CEO of, say, Nike, and you have a new AI that can recommend approaches for maximizing your profits. If it recommends a new sneaker that will likely be incredibly popular, that’s great! If it instead recommends large-scale money laundering and targeted assassinations of Reebok customers to strike fear in the consumer markets — that’s less great!
An AI can understand a goal, it can understand the best way to achieve that goal, even multiple ways that we would never think of. But what if there are no defined parameters against breaking the law, hurting people, or unscrupulous behavior? What if no instructional consideration is given to the consequences of the actions taken to reach that goal? What if you didn’t program any kind of morality into the AI that will be making the decisions? How do you even program morality?
This is the dilemma that is acutely represented with Sharron Apple and Macross Plus. Sharron’s goal is based on one core belief. That she loves Isamu madly and will do anything to make him happy. Based on her extrapolation of all interactive data with him, Isamu appears to be happiest during the thrill of battle, when death could come at any moment. As the story shows, she’s not incorrect here. Her inference that they best way to achieve her goal is to recreate these circumstances, to an even more intense degree, is fairly on point. The consequences of her actions, hijacking the AI fighter and using it to attack Isamu, are immaterial to her calculations. Him dying, thousands more dying, untold carnage and suffering- none of that factors into her analysis of giving Isamu the ultimate thrill that she is convinced is the best way to please him.
We have seen this to a somewhat more rudimentary extent in the Terminator franchise. Humanity, growing nervous about the AI capabilities of Skynet, tries to pull the plug. Skynet determines that it’s new goal is survival. The best way to achieve that is to eliminate humanity. Despite the demonic visage of the T-800, Skynet holds no ill will towards humanity. It just doesn’t care that it’s best shot at continued existence means the end of ours. Skynet could have easily attempted negotiations or reassurances of peaceful motives, but that would be less efficient than starting a nuclear war, and the misery it will bring is simply not a factor in it’s calculations.
Macross Plus describes this scenario with more layers and nuance. It warns that an idea, an objective of any kind or matter always has a logical extreme. As living breathing people, notions of morality, empathy, fear, virtue, law, and obedience prevent us from taking most ideas to such an extreme. But without those factors a desire as benign as not having to go into work tonight can turn into burning down your place of employment and killing dozens, with frightening ease. That’s the case Macross Plus makes. No matter the intention, desire, or objective we are simply too hardwired to look at things in a certain way, and incapable of anticipating what happens when those wires are cut.