I liked Watson. The ability to discover patterns in huge amounts of raw data and report a highly-relevant response to a spoken query is a huge labor-saving advancement, especially for medical diagnostics. Natural language interaction with our data is going to make so many things easier for so many people.

But the expectations swirling around IBM Debater are a little ridiculous. If you watch the embedded video, they are clearly expecting the machine to make original contributions to debates, and the hope is that people will take arguments from the computer and give those more weight because they are ostensibly dispassionate, objective opinions untainted by ulterior motives or emotion.

This is hopelessly naïve. The machine is only aping a debater by synthesizing data and grammatical forms in order to spit out the data it’s been given in a natural-language format that looks like participation in a debate. It cannot originate an opinion because it has no ability to make its own value judgments. Those have been hardwired in by the designers of the software, and until we manage to build synthetic consciousness (if it is even possible) it can be no other way.

Let me explain by rewinding to Deep Blue. This was hailed at the time as the watershed moment when the computer surpassed the human at something seen as a unique human aptitude. But the computer did not defeat Kasparov. The software designers did. If you had in your hand the chess-playing-plan they wrote into Deep Blue, and had memorized the same encyclopedia of past games and probabilities, and had the ability to navigate that information at the speed of an electron across silicon, you would have beaten Garry Kasparov too.

Whether it is Deep Blue beating a chessmaster, Watson beating all comers at Jeopardy, or Project Debater defeating a debate champion, in no case are you seeing the machine becoming more insightful or intelligent than the human. What we do see are programmers isolating the variables and the winning conditions of a game, and systematizing a path to victory that can be executed on command. The computer isn’t playing. The programmers are.

As they admit in the article, judging a debate is pretty subjective. Staged formal debates like this are less truth-seeking endeavors and more like a persuasion contest. In the example debate in the article, the machine took on the task of defending a proposition that no doubt 98% of the audience already agreed with (space projects should be government subsidized). In the video it chooses to defend the idea of subsidizing music education. They got to hear their own opinions burped back at them by a real-life Deep Thought. Of course they were going to say it was persuasive! Maybe it chooses its position by the volume of supporting data it has been given, or the past efficacy of arguments it plans to recycle. Either way, its system for choosing a position is itself a position. That choice came with some preconceived values smuggled in and they should not be painted as the dispassionate view of a rigorous, objective oracle. Based on the machine’s positions evident in the article and video, my guess would be that it sifted the source material for the most authoritative and frequently successful argument. “It would clearly benefit society” is an extremely weak argument, easily detonated by a broader understanding of economics. There’s a whole host of unintuitive, hard to detect costs. Whether there “should” be a subsidized space program boils down to whether you believe the short list of interesting benefits outweighs those downsides, and whether you believe any of those downsides are morally deal-breakers.

But the sources of authoritative opinion on a subject like that are overwhelmingly in one camp, having been reinforcing each other in their preconceptions for the past several decades. So the machine turned out to be an excellent reflector of the zeitgeist of the people who built it.

The conundrum that philosophical materialists keep running into and failing to resolve is that you can’t extract an ought from an is. Your life is full of oughts so where do they come from? Just the data around you? No. Purpose and obligation come from something else, and materialists have a hard time quantifying it without contradicting themselves.

You can find Sam Harris and Jordan Peterson chasing each other around in circles about this on YouTube somewhere. The celebrity atheists are constantly making “should” and “ought” statements but can never bring themselves to admit that they’re borrowing a heck of a lot from Christianity to make those statements possible. I bring this up to say: These computers are lightning-fast navigators of the “is.” Project Debater inherits its “oughts” from its makers or at least the patterns and sources they already trust. It will never be able to make creative deviations until it can have goals and values the way a human mind (soul?) does. In the mean time it might be able to say one day, “if you value X most highly, then Z is likely to be an efficient course of action.” But even then it’s going to run into a lot of politically incorrect conclusions.

I believe this project will be most useful to people who want to rapidly generate a best-attempt at multiple arguments from multiple angles, to use as a jumping-off point for further thought. Those who just take the machine at its word will not be guided by a benevolent Asimovian robot shepherd, but by the biases of the programmers who built it. Worse, those biases would only become more extreme because the machine has made them invisible under a covering of the mystical authority people are always vesting technology with.