@Maxwell Tabarrok, I suppose the question comes down to the degree of excludability. Some technologies/ideas are more excludable than others. Atomic weapons were both extremely powerful and the technology behind them very excludable, thus is made sense to do so. Many ideas, like electricity, we not as easily excluded.
The US is already attempting to keep the knowledge and equipment used for AI out of the hands of its adversaries.
Very interesting piece! Still I’m not sure I agree with your conclusion, even granting your premises.
Let’s say that AI is “intelligence on tap” that still is just a raw input into much additional engineering work to make it useful for military purposes. I don’t think that implies that it could not be worthwhile for a state to try to withhold that input from an adversary.
Both electricity and nuclear science are likewise only inputs into much more engineering work necessary to make weapons. But the anecdote with Fermi shows that Szilard wanted to lock down even that basic science, not just the final engineering diagrams for nuclear bombs. One might suppose that if electricity were being invented in that time, Szilard might indeed have wanted to lock that knowledge up too, and indeed one would imagine that this would be an incredible advantage in a war.
Now one practical difference might be that nuclear science is easier to withhold than electricity: an adversary could reverse engineer electrical science from captured devices, since the electricity is used as part of the durable outputs, whereas in nuclear science where there isn’t much to glean from an exploded bomb.
But AI might be more like nuclear here. The outputs (intelligent decisions) don’t necessarily contain very much information about the process used to generate them, assuming the AI is in a secure data center and not on-device.
To put it another way, you can break down the problem into two parts: is it possible to lock up the knowledge, and would it confer an advantage on a state to have locked up the knowledge. I would argue that for all three, it would indeed have conferred an advantage to be the only state with this knowledge, even if that knowledge is only an input.
Conceivably for electricity it was not possible. Indeed in the long run it didn’t prove possible to guard nuclear secrets against the Soviet Union’s espionage either. But it was possible for long enough to plausibly make a huge difference to the fate of the world, and AI could plausibly be the same.
I think you've identified a great empirical crux. With this piece and with Leopold's, I'm curious what cybersecurity/InfoSec experts in national security and tech are actually seeing with these models. They're not going to write "here are all the vulnerabilities" blog posts, but do you have any recommendations for who to read to understand whether the weights actually are the security bottleneck, or the scaffolding as you suggest might be possible, or if there are vulnerabilities across any Azure data center, etc? I see these as fast moving empirical questions and wonder whether the intelligence community is retaining the people equipped to answer them.
I also wonder whether Leopold drew a distinction between nationalizing the labs to lock down the weights/secrets, and nationalizing the supply chain because industry can't scale a trillion dollar or a ten trillion dollar cluster. I'm personally skeptical that we have anything near the state capacity to do that (no ASI lest a NEPA review finds a spotted owl), but they might be two different arguments.
@Maxwell Tabarrok, I suppose the question comes down to the degree of excludability. Some technologies/ideas are more excludable than others. Atomic weapons were both extremely powerful and the technology behind them very excludable, thus is made sense to do so. Many ideas, like electricity, we not as easily excluded.
The US is already attempting to keep the knowledge and equipment used for AI out of the hands of its adversaries.
Very interesting piece! Still I’m not sure I agree with your conclusion, even granting your premises.
Let’s say that AI is “intelligence on tap” that still is just a raw input into much additional engineering work to make it useful for military purposes. I don’t think that implies that it could not be worthwhile for a state to try to withhold that input from an adversary.
Both electricity and nuclear science are likewise only inputs into much more engineering work necessary to make weapons. But the anecdote with Fermi shows that Szilard wanted to lock down even that basic science, not just the final engineering diagrams for nuclear bombs. One might suppose that if electricity were being invented in that time, Szilard might indeed have wanted to lock that knowledge up too, and indeed one would imagine that this would be an incredible advantage in a war.
Now one practical difference might be that nuclear science is easier to withhold than electricity: an adversary could reverse engineer electrical science from captured devices, since the electricity is used as part of the durable outputs, whereas in nuclear science where there isn’t much to glean from an exploded bomb.
But AI might be more like nuclear here. The outputs (intelligent decisions) don’t necessarily contain very much information about the process used to generate them, assuming the AI is in a secure data center and not on-device.
To put it another way, you can break down the problem into two parts: is it possible to lock up the knowledge, and would it confer an advantage on a state to have locked up the knowledge. I would argue that for all three, it would indeed have conferred an advantage to be the only state with this knowledge, even if that knowledge is only an input.
Conceivably for electricity it was not possible. Indeed in the long run it didn’t prove possible to guard nuclear secrets against the Soviet Union’s espionage either. But it was possible for long enough to plausibly make a huge difference to the fate of the world, and AI could plausibly be the same.
I think you've identified a great empirical crux. With this piece and with Leopold's, I'm curious what cybersecurity/InfoSec experts in national security and tech are actually seeing with these models. They're not going to write "here are all the vulnerabilities" blog posts, but do you have any recommendations for who to read to understand whether the weights actually are the security bottleneck, or the scaffolding as you suggest might be possible, or if there are vulnerabilities across any Azure data center, etc? I see these as fast moving empirical questions and wonder whether the intelligence community is retaining the people equipped to answer them.
I also wonder whether Leopold drew a distinction between nationalizing the labs to lock down the weights/secrets, and nationalizing the supply chain because industry can't scale a trillion dollar or a ten trillion dollar cluster. I'm personally skeptical that we have anything near the state capacity to do that (no ASI lest a NEPA review finds a spotted owl), but they might be two different arguments.