Intelligence at work: a call for safety

Despite countless attempts, a definitive, all-encompassing definition of intelligence remains elusive. Perhaps that’s because intelligence is an intangible, fluid concept, open to interpretation, not something clear-cut we can simply plug into a range of contexts. Unless, of course, the context is a specific field, such as technology. However, when it comes to artificial intelligence, a simple definition posits that it is the combination of two factors: the ability to act in the world to pursue goals efficiently, and the speed and volume at which information is processed.

The latter is relatively straightforward: the more data an AI can process, the more intelligent it is considered. However, parsing data is pointless if the effort isn’t directed toward a specific goal. If you’re trying to pass a test, you use all your brainpower to achieve the best possible outcome. If you’re a computer trying to render a 3D video, you harness your GPU’s power for that purpose; and so on.

This might not strike us as a logical display of intelligence, and there are two reasons why. Firstly, most — if not all — data-crunching systems lack autonomy. After all, what intelligent entity would allow someone or something else to set its goals? Secondly, and perhaps more importantly, there is the issue of generality. If the goal pursued is in a relatively narrow domain, we may be impressed, but still largely reject any claims of “intelligence.” In chess, for example, AI-powered systems like Stockfish have been designed so that no human could possibly beat them.

For all intents and purposes, such systems are superintelligent relative to humanity: regardless of the strategy, the only possible outcome against them is defeat. However, their usefulness collapses when applied to other domains. Using Stockfish to browse the web, perform basic arithmetic, or tie a pair of shoes would be as futile as using a fork to eat soup, so this kind of “single-mindedness” clashes with what we would consider true intelligence. Generality is the idea underlying systems whose intelligence stems precisely from their ability to weave together notions, frameworks, and bits of knowledge across fields, combining them to come up with new ideas.

These concepts form two distinct axes. A vertical one, where the aforementioned definition of intelligence is used to establish a level (below, at, or above human), and a horizontal one, representing generality, which essentially dictates how complex the machine’s reasoning can become, spanning different fields of knowledge and application. This highlights how it is crucial to understand that in AI, a machine’s ability to pursue goals efficiently is the sole metric that can define its intelligence — there’s no moral judgment of the goal itself.

In his renowned paper, The Superintelligent Will, the Swedish philosopher Nick Bostrom developed the “orthogonality thesis” to emphasize this point further: at any level of intelligence and generality, an AI system might pursue any kind of goal, even one we would not deem intelligent. Whatever it does to try and achieve that would require a whole high degree of generality. Still, it would not necessarily mean that the AI would also share human values, despite fully understanding them, and thus start thinking that its seemingly weird goal is “not intelligent.”

Just a few years ago, a system like GPT-4 would probably have been considered general enough to fit the long-sought definition of artificial general intelligence, or AGI, the new holy grail that companies like OpenAI are explicitly pursuing. But now that GPT-4 is here, the goalposts have inevitably been moved, and the direction we’re heading is both exciting and unsettling.

Today, we can use a moderately intelligent system like GPT-4 in entertaining, creative, and even practical ways. Although we still need to prompt the machine ourselves, we can’t dismiss its capabilities: there’s clearly a spark of something there, an understanding of the world drawn from the meaning that lies behind words. It’s no wonder that, since its introduction, ChatGPT has brought the age-old question of “Is AI truly intelligent?” back into the limelight with renewed strength.

Since there are no clear-cut rules defining what AGI actually is (AI engineers and scientists often half-joke about it being “the AI that isn’t quite here yet”), we can expect the debate on the matter to expand, with experts disagreeing about whether the current leading-edge model is actually AGI; only to shrink after some critical point, at which most agree more unanimously. An obvious turning point could be when an AI is capable of programming another AI system, potentially superior to itself; however, signs of indisputable generality might show up long before that.

And, alongside a general mind, a superintelligent one could emerge, capable of doing everything humans do at levels of speed, scale, and accuracy we simply can’t fathom. This outlook poses a problem since companies large and small are racing to develop such systems with alarmingly little understanding of how they actually work, and our ability to keep them in check is sketchy at best. We could find ourselves in dire straits if new AIs start pursuing their own goals — either self-given or caused by our failure to instill human values into their utility functions. A superintelligent general system capable of pursuing virtually any goal could be difficult, perhaps even impossible to stop, and therefore extremely dangerous.

However we choose to define it, intelligence is power — and power, as we know, isn’t worth much without control, much like a car with a jet engine but no brakes. We need to work on the braking mechanism before these cars venture too far out into the open, perhaps running wild on their own. It will be a challenging task, but it may be our only chance to safely reap the potentially boundless benefits a more intelligent companion can offer us, including the solutions to the other crises we face today.

Let’s not be stupid.