Contra Scott on AI Races

What is a race exactly?

TLDR

1. Under slow takeoff, general AI is an indefinite race. There’s no fixed irreversible end point, and there’s no clear winner that gains a permanent advantage. But that also means that at all times countries have to be stronger than the next best power, in order to keep an advantage to win in case of war. 

2. In a model of slow takeoff, a fixed time gap still leads to a growing GDP gap! The extent of the gap depends obviously on the speed of the GDP growth. But under even modest assumptions, it leads to a very large GDP gap in favor of one country. 

3. A GDP gap in favor of one country is likely to mean that it has a very strong advantage during a war of attrition. In the most recent world war, more natural resources and better manufacturing were the determining factors of winning the war.  Given that the next large scale war will also be of that sort, it is important that countries maintain and edge here

4. In the early stages of a transformative technology it is hard to conclusively tell if something is militarily relevant or not. So, it's important not to strangle your domestic industry because it can help in a future conflict. Finally, any GDP gap due to general purpose AI is more relevant to military conflicts making it extremely crucial that countries have to stay ahead in it (see point 1)

The Actual Essay

Scott Alexander wrote an interesting essay that most technologies aren’t races. (He seems to have edited out some crucial claims in the essay. Here’s the original post. Here’s the live version). It is forceful, but wrong. I recommend you read the original essay, before you read what I’ve written below.

If I were to summarize his blog post it would go like this:

(I don’t agree or disagree with the points below, I’m just writing them down so we both have a basis for a conversation)

• For most technologies, there is no “race”. That is, there is no fixed end point, after which one country or person is said to have “won”. Did Henry Ford win the automobile “race”? Or did Toyota? Did Alan Turing win the computer “race”? There’s no answer to that question because most technologies don’t really have that sort of end point after which one person is said to have won, and then the “race” is over. No technology in the past has led to a very large shift in the balance of power, and today all developed countries have computers, automobiles, electricity etc.

• There are “races” in military technologies, like stealth bombers, tanks etc. but the focus of consumer policy on airplanes and automobiles isn’t really related to the military aspects of them. So then why should we want to speed up consumer AI to gain a military advantage?

• Scott feels that people are repeating what transhumanists call an AI race without really understanding what they mean. When transhumanists talk of an AI “race”, they’re talking of either a race to align an artificial general intelligence (if you don’t know what those terms mean: see this for alignment and this for AGI) before it’s built or a race between countries to build an AGI that leads to super-duper fast growth. 

• More on the super-fast growth part: one model of AI growth that transhumanists have is “fast takeoff”. That is, an intelligent AI is able to improve itself, and it speeds up years or decades of  progress in a short time (months, weeks or days). But here’s the issue: if you believe in this sort of “fast takeoff” model, you probably also believe in a very high chance of this super fast growing AI also ending the world. Because, if it goes so fast, it’s extremely unlikely to be controllable and aligned with human interests.  And so, it makes very little sense to have an AI race if you think it's going to kill everyone. 

• But Scott doesn’t really think that this sort of fast takeoff is possible, and he thinks it's going to be some sort of continuous process (but still fast). But if you fall behind in this race, he says it's not really fatal. The disadvantage you have exists, but he says that you can steal the technology, and that it doesn’t really matter that much. It’s important, but not very crucial.  

• So he asks: given all of this, why do you think it is so crucial to win the AI “race”? 

Now, back to my argument.

Definite and Indefinite Races

I think it’s important to define what a race is. 

Some races are definite races. 

A marathon is a definite race. There’s a clear end line, someone fires a blank and everyone races to the end. The terms of this race are very clear! You have to reach the finish line before everyone else does, and so you win. After you win it, it's irreversible. You can’t go back in time and reverse the decision. The winner stays the winner and the loser stays the loser. 

Another example of a definite race is if multiple countries had a race to colonize the Moon. The end line is clear, and more crucially, winning has an irreversible effect. There’s a clear first mover advantage to this. If you get on the Moon first and establish your colony, it’s much harder and expensive (maybe impossible) for another entity to come, dislodge you and establish themselves there. 

There’s another aspect to definite races. What happens in the meantime doesn’t matter as much as who wins. If you’re running a marathon and some other participant temporarily takes over you, but later if you take over him and win the race, the initial takeover didn’t matter. What did was who won the race in the end. 

So in the parable of the hare and the tortoise, they’re in a definite race. 

The story concerns a Hare who ridicules a slow-moving Tortoise. Tired of the Hare's arrogant behavior, the Tortoise challenges him to a race. The hare soon leaves the tortoise behind and, confident of winning, takes a nap midway through the race. When the Hare awakes, however, he finds that his competitor, crawling slowly but steadily, has arrived before him.

So we can see the characteristics of what I call a definite race here: a clear finish line, the non-importance of what happens in the meantime and irreversible and permanent effects of winning

So by this measure, building AGI if there’s going to be a fast takeoff is a definite race. 

But there is another kind of race too: an indefinite race.

Take for example two companies trying to build a better smartphone. Do Apple and Samsung have a clear picture of what The Final Smartphone looks like? No. They have some idea of what the next iteration looks like, but there’s no finish line in mind. There’s no estimate that by some year there will be a clear victory for Apple and Samsung, and then the race is over. They’re just trying to beat their competition for that specific product cycle, and improving research so they don’t fall behind.

There's a joke about two hikers and a bear that goes like this:

Two friends are in the woods, having a picnic.  They spot a bear running at them.  One friend gets up and starts running away from the bear.  The other friend opens his backpack, takes out his running shoes, changes out of his hiking boots, and starts stretching.

“Are you crazy?” the first friend shouts, looking over his shoulder as the bear closes in on his friend.  “You can’t outrun a bear!”

“I don’t have to outrun the bear,” said the second friend.  “I only have to outrun you.”

In a competition like this, the main indicator is whether you’re behind your competitor or not. There’s no clear end line, and the point is to just keep going ahead of the other guy. 

One example of an indefinite race is the naval arms race between Germany and the UK before World War I. Somewhere in the 1890s the Germans realized that as their economy grew and as they were becoming the number 2 power in Europe they needed a navy to match it. More specifically, they came to believe that naval power was going to be the decisive factor that allowed nations to impose their will against others. So, in order to ensure that they weren’t beaten by the British, who had a strong navy, they decided to start building a navy that could beat them.

They passed five laws from 1898 to 1912 that expanded the size of the German naval fleet. The British, who wanted to keep their superiority on the seas, expanded their naval arms production. 

In 1906 Britain unveiled its ship the HMS Dreadnought. The Germans were terrified. Not only did it have far more guns and a bigger displacement, it had a faster speed than any of its competitors. The race to build more dreadnoughts was on. Germany incorporated building four dreadnoughts in its naval plans which were then completed in 1908. By 1908, they had four dreadnoughts and Britain had only two! 

This sent leaders of both countries mad. They wanted to build more and more dreadnoughts, apparently, at any cost. One Conservative MP quipped: “We want eight and we won’t wait!”. Winston Churchill joked that “The Admiralty had demanded six ships; the economists offered four; and we finally compromised on eight.”

HMS Dreadnought via Wikipedia

The Germans for their part were no better. At a meeting between Wilhelm II and British officials, when the Kaiser was asked if they could  build less to calm tensions, he responded that “Then we shall fight, for it is a question of national honour and dignity.”

The arms race ended on the German side when Germany stopped in 1912 after it realized that it cost too much money and they had to spend money on their army to deter Russian aggression. Britain for its part continued to build, having 22 at the start of WW1.  

This race is very different from the race to the moon. It doesn’t have a definite end. There’s no point where Britain or Germany can say they have “won” the race. That’s because there’s no point when there’s no point where one player gets an irreversible edge over the other one. If Britain built the dreadnought, Germany could build it in the future. If Britain built a bigger cruiser, Germany could make it up in the future. So the race had no definite end.

And here a reasonable person might ask: why bother getting into the race if you can catch up in the future? After all, if Britain built its first dreadnought in 1906, and Germany was able to catch up in 1908, why was there so much fuss about it?  If tensions got too bad, Germany would always catch up in the future right? 

Wrong.

The German fear was that two years was two years too long. If they weren’t on par with Britain at all times, Britain would attack them at any possible time and they would lose a war to them. So they had to be ahead of Britain at all possible times during the arms race to both deter Britain from attacking and to ensure their eventual victory if the war ever happened.

The difficult part of being in an indefinite race is that it is always on. You can’t be the tortoise in Aesop’s fable of the hare and the tortoise and take a break. Because if you do, the hare will come with a dreadnought, bombard your shores, blockade your ports, force you to starve to death and beat you during the war.

Applying my race model to his essay

Now Scott is correct in saying that if there’s going to be a fast takeoff, it is definitely a race. And his essay somewhat talks of the idea of indefinite races, with the stealth bomber example. But I don’t think he understands the nature of indefinite races, and why it’s important to stay forward in one.  

GDP Gaps will grow under a lag

One main argument in the essay seems to be that the costs of falling behind in a race for general artificial intelligence isn’t that bad. Maybe they get a GDP advantage for some time, and there’s a chance it tips the conflict. But it’s not fatal or crucial.

I don’t really have strong views on whether future AI systems will be general in nature (which means if they can do a lot of things) or whether they will be narrow (specialized systems doing specific things). Neither do I have a strong view on how much such general systems are going to accelerate economic growth in the countries that have them. 

But it does seem that Scott thinks that there’s going to be general AI so strong that it can lead to very large economic growth. He’s talked about artificial general intelligence quite a bit (here, here etc.) and seems to think from the essay that AI will cause large increases in economic growth.  

Now given these assumptions, the conclusion that getting slow AI takeoff later than the others isn’t crucial isn’t very strong. If there are two countries with the same GDP, and one has AI takeoff, and the other has the same AI takeoff two years later, (as his model says), then the gap between them is growing!  

Intuitively, if your opponent grows at 18.9% a year (doubling every ~4 years), and you have your takeoff moment two years after that, then clearly because it's an exponential, the gap increases exponentially. 

Here’s a desmos calculation showing that (base GDP is 100). 

In a case where the doubling period by itself also reduces (it first doubles in 4 years, then 2 and then 1 and then continues doubling), the gap between the two countries increases exponentially even faster! Here’s a model showing the gap between the two countries under this model. 

And so, in either of these models the GDP gap increases over time. In the case where GDP growth doubles “only” every 4 years, in 10 years the faster country has a $160 advantage over the slower one. In 20 years it’s a $933 advantage. 

(Note that this is an absolute advantage not a relative one because that’s what matters here. If you can mobilize for example 900 tanks and they can mobilize 1200, the absolute gap of 300 is what matters, not that it’s “just” 30% more. )

But if you expect the speed of GDP growth to increase (as I presume Paul Christiano does), this gap also increases even faster over time! In the model where GDP first doubles every 4 years, then every 2 years and then every year, the gap increases even faster!

An indefinite race with slow takeoff

And in the model of an indefinite race, slow takeoff is crucial too. You might be just two years behind, and at any point of time there might be a war and you’ll be left behind. That’s what he doesn’t get. It’s that if you’re in an indefinite race, you’re on your toes all the time! And if you’re losing in the indefinite race, you might lose the whole thing altogether. 

But you might ask: so what if your GDP is lower? Does it really matter?

But so what?

In his original piece Scott also makes that claim that this really isn’t a big deal. 

China is currently about 2 years behind the US in AI. If they’re still two years behind when a slow takeoff happens, the US would get a ~40% GDP advantage. That’s not enough to automatically win any conflict (Russia has a 10x GDP advantage over Ukraine; India has a 10x GDP advantage over Pakistan). It’s a big deal, but it probably still results in a multipolar world. Slow-takeoff worlds have races, but not crucial ones

My first concern is covered by the above section - taking his example to its logical conclusion, the GDP gap goes up really quick. And if the GDP doubling time is faster than expected, the gap goes up faster than that. But, so what? So what if our opponents have 40% more GDP than us. So what if they have 100% more GDP than us? Doesn’t the experience of Ukraine and Pakistan prove that a smaller country can defeat a larger one in war?

I don’t think what he is saying is true. First, the example of Ukraine is misleading because he presents it as if Ukraine is defeating Russia (the war’s not over yet, although Ukraine has done much much better than expected) by itself. But it has got massive amounts of foreign support not only during the war in terms of material and funding but also before the war in terms of training and aid (remember the Trump Ukraine scandal?). So, it really doesn’t make the point he is making that smaller countries can much beat larger ones.

Pakistan as an example makes the exact opposite point. Pakistan has fought between four and five wars with India (depending on how you count the Siachen conflict). India has won every single one of them, except for 1948 which was a partial Pakistani victory because they had the first mover advantage. In 1965 although it was a ceasefire, Indian forces went all the way to Lahore and captured 500 sq. km of Pakistani territory, but did not capture the city because a ceasefire was being negotiated. 1971 and 1999 were clear Indian victories, as was the Siachen conflict. 

But both of these don’t ask the fundamental question: what happens if one country has a large GDP advantage over another country? It is true that in guerilla wars, larger countries can lose because of the asymmetric nature of the war. We’ve seen this with Sun Tzu, Shivaji, the Anglo-Irish War, Vietnam etc. But when you consider large scale wars that involve multiple countries, it is the economic and technological aspect that gives the winner a heavy advantage to begin with. 

WW2 as an example of economic dominance

Oil in WW2

Let’s take World War II as an example because it is the only thing as close to what a future war between superpowers might look like. WWII became a war of attrition where opposing forces began to run each other down and the main objective was to last longer than the other participant. The economic and technological potential of the country began to matter more. The losers of the war - Germany and Japan - were hit by oil shortages mainly which led to their downfall in crucial battles. For example Erwin Rommel lost the war in North Africa in large part due to the fact that his oil supplies were captured and Germany didn’t have much more to give him. And so he said:

“The bravest men can do nothing without guns, the guns nothing without plenty of ammunition, and neither guns nor ammunition are of much use in mobile warfare unless there are vehicles with sufficient petrol to haul them around.”

Or also to quote George Patton:

“My men can eat their belts, but my tanks have gotta have gas.”

Hitler too invaded Sweden to get its minerals and tried to invade the USSR to get Azerbaijani oil. Japan also invaded South East Asia to get oil which was why they hit Pearl Harbour and in the end also why they could not keep up with the Americans. This economic constraint was one of the main reasons why the Axis powers lost WW2. 

And if you model in any future World War as going to be a war of attrition much like WW2 became, then your economy matters very much! What does matter is the volume of resources you can amass to defeat the enemy. Japan and Germany were defeated because of this. 

To quote from Mark Harrison’s The Economics of World War II (emphasis mine):

In the second period of the war, which began in 1942, economic fundamentals reasserted themselves. The early advantages of the Axis were dissipated in a transition period of stalemate. A war of attrition developed in which the opposing forces ground each other down, with rising force levels and rising losses. Superior military qualities came to count for less than superior GDP and population numbers. The greater Allied capacity for taking risks, absorbing the cost of mistakes, replacing losses, and accumulating overwhelming quantitative superiority now turned the balance against the Axis. Ultimately, economics determined the outcome

Airplane and tank manufacturing in WW2

Another important claim made in the essay is that consumer policy for automobiles isn’t really relevant to military policy. And so, we shouldn't really design AI consumer policy too because of that

Maybe in some sense the British won a “race” for radar, although it wasn’t a “race” in the sense that the Axis knew about it and was competing to get it first. Maybe in some sense countries “race” to get better fighter jets, tanks, satellites, etc than their rivals. But ordinary mortals don’t concern themselves with such things. No part of US automobile policy is based on “winning the car race” against China, in some sense where consumer car R&D will affect tanks and our military risks being left behind.

And I’ll grant that to him that in this day and age we have a very clear distinction between consumer and military automobiles. They’re funded, produced, designed etc. in complete independence of each other. But historically, that wasn’t the case, and today if there would be a large war that requires requisition of civilian assets, having a large automobile sector would definitely help in producing more tanks and planes. 

And in WW2 when automobiles were in their childhood, exactly that happened. Nobody consciously designed automobile policy to produce more tanks, planes and ships during the war. But that was a fortunate byproduct of the large industrial capacity. To quote from a NYT review of Freedom’s Forge:

He [Bill Knudsen] had mastered Detroit’s discovery: the enormous power of using precisely tooled, interchangeable parts to break down complex manufacturing into far simpler steps. He also knew everyone who was anyone in industry. When he urgently needed a thousand tanks a month, his way was to call up K. T. Keller, Chrysler’s president. “Can you do it?” The answer was, “Sure.” For the engines, he phoned Jack Reese at Continental Motors, who welcomed the business. For welding heavy steel plates, never before done with tanks, it was Bill Smith at A. O. Smith.

…

The pace of change all across America was staggering. By the end of 1942, the author says, three million women were working in a war industry, up from barely 80,000 six months after Pearl Harbor. In due course, America’s arsenal turned out two-thirds of the Allies’ total war needs, an astonishing outpouring of aircraft carriers, battleships, destroyers, submarines, bombers, tanks, artillery pieces, trucks, jeeps, machine guns and 41 billion rounds of ammunition.

And it was so crucial that the first person to run it was Bill Knudsen, formerly of GM and Ford. Clearly, while they did not make a large auto industry just so they could have had more tank manufacturing, that was the correct policy in hindsight!

And it is extremely common for domestic manufacturing capacity to be taken for national use during emergencies like wars or pandemics. For example GM (yes, the same company) was requisitioned to build ventilators during COVID-19.  (I’m not sure how many were actually built versus just being an announcement). 

So, to conclude this section, natural economic resources like oil and metals are extremely important. And in industries with strong spillovers to military technology, it is also extremely important. But the other lesson you should draw from this is that in the early stages of a large technology, it is hard to say what can be used for the military and what can’t be. So, while we might not be able to predict what AI tools can be used for military applications now because it is a new industry, getting a bigger AI industry is extremely helpful just like getting a big car industry was pre WW2. 

Economic wars and indefinite races

If a war is short and limited (say like India and Pakistan in 1999), then your economic strengths don’t really matter as much. What does matter is the level of military mobilization in that specific time period. But if you expect to fight a war of attrition like WW2 was, then you are in indefinite race mode. Keeping an economic lead over your opponents matters much more because losing the economic race means you might lose the war. Much like the Germans in the naval arms race, falling behind means that you’ll have a serious disadvantage. 

You can’t relax and wait for X years because that means that during a war, you’ll be X years behind. And if you’re X years behind, depending on how long you were behind for, you’re probably at a large economic disadvantage. 

An AI-specific argument

Beyond all of this, there is an interesting AI specific argument too. 

It is that AI is more important to winning a war than other sources of economic growth are. If one country becomes rich by selling services (say call centers) then that doesn’t help very much during a war. It’s hard to convert an advantage in call centers to a military one. But if another country gets rich by having better manufacturing, then it's easy to see how it might contribute to a military advantage. And if a country gets rich by having general artificial intelligence, this converts into a military advantage quite well!

It could mitigate or even solve many of the resource constraints countries have by doing better R&D (for example the Germans might have sped up their synthetic oil or nuclear bomb project if they had AI that did research), it might plan better than the humans can, and it might run manufacturing better than humans can etc. etc.

It would just give a large advantage to whoever had the AI, even if the GDP gap wasn’t particularly large. Or let me rephrase it: if a given gap in GDP is produced by general purpose AI, then it will cause a much larger gap in military capabilities than any other reason. 

Yes, losing a world war is pretty bad

I somehow get the sense that Scott thinks that losing a world war isn’t really that bad. He writes in his original piece

The only case where there’s a single critical point - where you either have the transformative AI or nothing - is in the hard-takeoff scenario where at a certain threshold AI recursively self-improves to infinity. If someone reaches this threshold before you do, then you’ve lost a race

He’s never directly said it, and I don’t know if he means it but it's hard to escape that conclusion from the tenor of the post. 

I’m going to state the obvious that losing a war is very very bad and is worth spending large amounts of national resources avoiding. The Germans and Japanese who lived in poverty for the next few years after WW2 were lucky that the American postwar strategy was to ensure that they were rich and aligned with American interests. Other people say those who were colonized or made into client states like the Poles, Hungarians, East Germans etc. weren’t so lucky. 

Conclusion: what about alignment?

And you might ask, how do we trade this race dynamic off with the risk of building a misaligned general intelligence? Honestly, I don’t have the insight to answer that. But his argument means “there is no race, don't use a race as an excuse to not care about getting AI to be aligned with humans”. My argument is that there is a race, so the first point itself doesn’t hold true