The complexity of reality

Reality is complex. It might sound like a pre-packaged, pseudo-philosophical phrase, but it’s a concept that is actually being studied at one of the world’s most advanced facilities, the Santa Fe Institute (SFI). What does “complex” really mean in the context of reality, and what lens do we have to better comprehend it — and thus, in turn, understand a little more about life and ourselves? These questions have indeed been part of philosophical debate for centuries, but a more science-driven approach is at the core of David Krakauer’s work. Krakauer is an evolutionary biologist, currently at the helm of SFI, with research spanning genetic, neural, linguistic and cultural mechanisms, and more.

Edoardo Maggio: How has your life been over the past eighteen months? Has anything changed for you, either as a person or from the point of view of a researcher?

David Krakauer: At the Santa Fe Institute, we study complex systems, with faculty working on epidemiology, immunology, economic collapse, economic transition. So, for us, Covid has turned into a period of very active engagement. Perhaps a bit too hectic because we were called in many different ways to contribute. It was pretty demoralizing because the human population was incredibly foolish in many ways. We’re publishing a book called Complexity after Covid, which attempts to address why we, as a planet, have failed to cope with it. And when I say “failed to cope with it” I mean in all aspects, not just vaccine development, which of course we did rather well; but socially, economically, and psychologically. It’s been kind of a reckoning, I think, for the planet. And the question we must ask ourselves is whether we are coming out of it a little wiser, because it’s going to happen again.

How did you get interested in science?

Well, it’s not so much an interest in science. It’s more of an interest in ideas. I’ve been lucky. I’ve always been interested in literature and art and mathematics and natural science almost to the same extent. And one of the advantages of science over artistic pursuits is that there’s more immediate feedback on progress. In other words, if you write a paper, then your peers will tell you, “that’s good,” “that’s thoughtful,” “I like it.” There’s an almost instantaneous gratification, whereas with more creative work, it’s much more challenging. You’re on your own for longer, and it’s not clear that you’ve made a contribution. So, I want to point out that I really don’t make these distinctions between science and art. I actually find them a little childish. And part of what I do at my job is to assault that particular thought, that defends siloed imaginations.

One way to make a distinction is between the early stages and the late stages. At SFI, we’re interested in the early stages of ideas. No one stays here: We don’t scale any project up, we don’t have departments, we don’t allow large groups. So, in that respect, it’s a creative, exploratory environment, very different from an exploitative, excavating, mining context. And I like that exploratory sensibility and perspective. But my personal career is not particularly interesting. I studied in Portugal, Britain, and the US; I went to London University, Oxford, Princeton. So, I’ve had a fairly traditional education at fancy universities. But that’s not particularly interesting, I think.

You mentioned how ideas don’t live inside of silos, as they go across fields like science, art, and literature. That is reflected in the organizational structure of SFI, which is also very horizontal. Outside of that, however, what is it that truly sets SFI apart?

You know, there is nothing like SFI. There’s no comparable institute in the world. And that’s for two reasons. One is that we are an institute dedicated almost exclusively to complexity. All the other universities have a physics department, a chemistry department, a music department… But there’s nothing unique to any university, right? They’re essentially clones, cut-and-paste universities. All that changes is where they are located. And, of course, they have their own histories. Still, structurally, you could walk into one and then walk into the other and have déjà vu over and over again. We’re not like that. So, we’re unique in the sense that we work in a domain that no one else essentially works on at the institutional level.

But the most significant difference is this pursuit of the foundations, of SFI’s mission, the study of the fundamental principles behind all of complex reality. And that kind of thing is rare now. I mean, you see parts of it in some physics departments — the fundamental structure of the universe, string theory, perhaps quantum mechanics. But outside of those domains, the idea that you could say that there is a foundational theory that relates to mind, the economy, history… That’s just not typical. It’s almost invisible. I think that’s our strongest characteristic; to some people, our very weakest. We believe that all complex systems follow certain emergent laws, and believe in developing methods to study them.

How do you define topics that matter to you, that are unquestionably part of what you are doing?

The simplest way to say it is that we study adaptive agents. In, say, physics or chemistry, electrons are not adaptive: They have no inbuilt force, no purpose, they respond to forces in fields. But we study agents, things that want something, that have a goal, but also the networks, the collective dynamics of agents. Think of a city, or the human brain. They look like that. Adaptive agents transcend any particular system. Now, having said that, what do you do? Well, you have to develop methods which respond to that. So, of course, we are pioneers in network theory. That’s why companies like Facebook and Google and all the others have been affiliated with us for many years because we develop those methods.

So that’s the methodology of complexity. Then there are frameworks of complexity, like scaling theory or the theory of collective computation; how groups of cells make decisions, form ideas, how groups of individuals polarize or swarm, etc. That’s the two things that we try to do. We need new mathematics and new computation to study these things, but we also need new overarching theories that unify these things. And that’s what we’ve done. And of course, we have many projects. I mean, obviously, we’re the center of the theory of scaling and urban organization.

The modern theory of cities, where you think of cities as large computational devices that minimize energy expenditure, comes out of us. But we have also studied things like social and economic collapse and non-equilibrium economics. If you drop the equilibrium assumptions of neoclassical economics, what happens? And when you worry about agents that are not all the same and they are not just trying to work out equilibrium, supply-demand curves, what happens? That whole field of complex economics course comes from SFI. So, it’s hard to confine the contributions. They’re very broad. Wherever you find this characteristic of these networks of agents, you find us.

How do you combine the rigor of mathematics with the inherent chaos that seems to be such a fundamental part of the universe?

The fields that we have mathematized most efficiently are the fields with not a lot of chaos in them. When you think about the orbits of the planets, they’re so regular that every society on Earth has been able to make predictions even in the absence of formal mathematics. You know, the sun rises and sets, every child can see that. So that lends itself to mathematics because mathematics wants to deal with things that are very regular. But when you get into this world that we care about, like human behavior, for example. Well, what then? I mean, what are the regularities? How could you predict them? How can you understand them? I think there are two approaches to this chaos. One is machine learning and AI, which takes vast amounts of noisy data and tries to find patterns that can be exploited for various means.

But the problem with that world is that those models are completely opaque. They’re black boxes. GPT3 (a powerful machine learning-powered program used to generate text, Ed.) has on the order of 150 billion parameters, a crazy number. No human mind can grapple with a model of that size. On the other hand, there is what we do, which we call the coarse-grain paradigm of understanding and the fine-grain paradigm of prediction. We develop theories of averages, where you look at the data and find levels where regularities appear. If you look at individuals in a city, it’s impossible to predict whether they’re going to turn left or right tomorrow. It’s also useless to zoom all the way out and just say that yes, there is a human on planet Earth.

But in between those two, there is a level that shows that there is a high probability that you are going to be in a certain place. As you average, you find more and more regularity and less noise. So, part of what we do is finding that optimal granularity in between, and thus laws that operate on that scale. And it’s not going to be quantum electrodynamics, right? It’s going to be something different. That is very difficult and a big part of the endeavor.

How would you define intelligence? What is it?

Ah, now you are in my territory! So, I have my definition, but of course, everyone has their favorite. For me, essentially, intelligence is a set of rule systems, either innate or learned, that make hard problems easy. If you’re at school and you’re sitting next to someone who is very good at math, the way you know they’re good at math is if they make hard problems look easy. That’s the trait of someone who’s smart. But they could be smart at tennis, right? You’d say, “God, you make that look so easy!” The opposite of that is making easy things look hard, and that is stupidity. You have to consider both, and then ask: Does education achieve this objective? Do algorithms? And one of the virtues of this way of thinking is it’s not just humans, it’s also about machines and animals. That is intelligence. And it can be achieved collectively or individually, with or without tools.

We started with companies like Facebook and Google whose business is to match advertisers and vendors with potential buyers and ended up with people being manipulated, to the point that a far-right and rather extremist politician became president in the most important democracy in the world. How did we get there, and how could it happen so quickly?

It’s insane, and it’s real. It’s what I consider real artificial stupidity. But I think there are precedents. Think about school. Much has been written on the true purpose of school, and we know the true purpose of school is to provide time for individuals of an economically active age to make productive contributions. It has less to do with educating children. OK, that’s a separate conversation—but much of what school really does is enforce a series of norms and disciplinary uniformity on our styles of thinking. So, we will more or less form the same ideas, the same attitudes. I mean, we rebel at the margins. But by and large, we look the same, we dress the same, we think the same. So that’s a good example of a virtuous institution whose stated purpose is to inspire and fill our minds with the tools and technologies to be creative individuals, where in reality all of us know schools and universities are essentially factories that minimize individual creativity.

Whoa, that’s quite the statement.

Yes, but I think that’s clearly the case. We know that the intention and even the institutional structure, in embryo, is not necessarily that illuminating about what one becomes as an adult. Back to your original point, I absolutely think these companies did not start necessarily with nefarious intentions. But very quickly, human laziness, human energy minimization, human distraction, all of our cognitive biases and weaknesses slowly become data for a company that will pander and reinforce those weaknesses. And it takes quite an effort, I think, to break out of that cycle because when you are trying to solve real, complex problems you need energy. We are somewhat reluctant to go down that path, and these companies have discovered, collectively, that that’s a feature of humanity, and the algorithms exploit it. It’s left to us to rebel against it; I feel very strongly that way.

These days it seems like we rely more and more on technology, to the point that we are almost entirely submerged into it. If the tools we use come from the same source we have to fight against, how are we supposed to rebel? Is it even possible?

I don’t think it’s impossible. Here, too, you have to look for precedent. Think about smoking in public, same-sex marriage, institutional racism, sexism… these are all things that used to be a large part of our society, and yet we have mostly managed to overturn them. How did these revolutionary, radical moments occur? That’s what we have to look at. I suspect that the reckoning with the negative sides of technology will trigger something as radical as the suffragette movement or the fight for racial equality. That’s the scale of social self-awareness we are looking at.

However, the question here is whether it is possible to eliminate the worst abuses while maintaining the positive stuff. And I guess my main point is that the solution is not a tech solution; it’s a human solution. I remember many years ago going to a meeting run by IARPA, the Intelligence Advanced Research Projects Agency, on cybercrimes. The great insight that even five years ago was starting to be made was that the solution to cybercrime is not more software, because software is a perpetual Red Queen problem, an endless arms race. Whereas if you can convince a human being, that’s different. You do break free of the cycle a little.

Back to you, is there any one turning point in your life you would pin down as the most important?

That’s such a hard question! I think, perhaps, when I was 12 or 13, and I became interested in the idea of the idea. It was a very meta epiphany. I was reading a mathematics book in the library, and I couldn’t understand it; it was beyond my abilities. But it made me think about the idea: What this thing is, and why it’s everywhere. For instance, what is music as opposed to what is this piece by Miles David, or Schönberg? What are they both doing, and what is the theory? What is the model? You might not believe me, but it was born at 12, 13 years of age, and it’s never gone away. My entire career is essentially based on that thing. It’s quite hard to express it, I think.

Austrian writer Stefan Zweig wrote a book named ‘Decisive Moments in History’, and the conclusion he ultimately comes down to is that turning points are often born out of chaos. There’s irony in the fact that moments so important in the history of our species and the planet come out of something we cannot control. Part of what you do at SFI is trying to make sense of and control that chaos. But if we do, in a way, are we not stopping these chaotic moments from happening, and thus risking to prevent some of those decisive moments?

Yes, and I agree with that. There is this huge risk; it’s the point I was making earlier about the school system. On the one hand, saying ‘I want you to be inspired and creative and disruptive’ and what have you. On the other hand, saying ‘I’m also going to ensure that you’re the perfectly fitting component of my machine.’ We deal with this problem all the time, but that’s because the ideas that we work on are radical. In fact, as part of my job, I am myself a little bit of an instigator of chaos. What I ask members of our community is, are you radical? Could this paper generate real controversy? I care about that much more than the particular research program.

In many written constitutions, one of the things that often recur is the pursuit of happiness. Do you share this goal? And how do you define happiness?

I actually don’t. Santa Fe is a big center of Buddhism in America, and one of my good friends is a writer and a rōshi (an honorific title used in Zen Buddhism, meaning “old master,” Ed.) here. We’ve discussed this a lot; we would sit down, and he’d ask me how important happiness is to me. I told him that I’ve never pursued happiness. I’ve pursued hard problems straight. I’ve tried to remedy deficits. But not happiness, because I don’t think happiness is a thing. It’s an emergent phenomenon, and we study those.

It arises from the confluence of a good life and meaning, the pursuit of worthy goals, taking pleasure in hard work, generosity towards other human beings… which, taken together, generate this phenomenon called happiness. Whereas I get concerned that people who say that they are pursuing happiness are usually not very happy! It’s too vague. It’s the constituent parts that matter, and when you add them together, you reach this mental state that you might label as happiness. But I’m not sure it exists independently of that sum.

Last question, back to turning points once more. What would you say is the most significant turning point in the history of humanity?

That’s such a deep question! I have a few. One is the Darwinian revolution; that’s the most important one. The recognition that everything in life is related, that a tree is my distant cousin. That, to me, is extraordinary. What about the ethical implications of that insight? We still haven’t even begun to process them. Then, more personal to my profession, is the Scientific Revolution. The idea of taking nothing on faith, because there is no ultimate authority.

“And yet it moves.”

“And yet it moves”! The Galilean mischief. That’s wonderful, right? The suspicion of orthodoxy, but replaced with something constructive.

We did have a hint of it some two thousand years before that with Socrates, however. Then we tried to answer his questions…

That’s true. The distinction between antiquity, scholasticism, and then the late Renaissance and the Scientific Revolution has always fascinated me. In Plato and Socrates you do have authority, though. A scientific revolution done right, on the other hand, represents the most profound iconoclasm. The great tragedy of science is that it has become an authority, whereas it should just be permanently adolescent, in a stage of questioning authority. And not take itself too seriously.