Hard limits to AI scaling

Analysis provided by ChatGPT Now AI systems are subject to both information and energy entropy. For energy, the efficiency equation implies what sorts of limits for AI scaling. For information entropy, doesn’t scaling imply that the work of obtaining correct information is limited by error snowballing, in a manner quite analogous to the energy efficiency law? Yes. I think there is a useful analogy here, though I would make it more precise than saying that information entropy obeys an efficiency law identical to thermodynamic efficiency. The striking possibility is that AI scaling has two distinct resource ceilings: an energy/compute ceiling and an epistemic/information ceiling. And the latter can indeed produce something resembling an efficiency law because errors propagate through dependent computations. 1. The energy side For computation, the basic constraint is straightforward: useful computation ≤ available energy × computational efficiency. At the physical level, there are fundamental limits on how much information processing can be performed per unit of energy. But today’s AI systems are nowhere near all of those ultimate limits; the practical constraints are dominated by things such as semiconductor efficiency, memory movement, communication between processors, cooling, and the energy required for increasingly large training and inference runs. So scaling by simply adding more computation eventually encounters diminishing returns per joule. The important point is that the energy does not merely pay for “intelligence.” Much of it pays for moving, storing, checking, communicating, and correcting information. That leads directly to your second point. 2. The information side is analogous—but not identical Suppose an agent is doing a 1,000-step research task. At every stage it has to establish some proposition: P₁ → P₂ → P₃ → … → P₁₀₀₀. The agent doesn’t merely need to generate information. It needs to establish which information is correct. That’s a fundamentally different resource problem. Imagine, for simplicity, that each proposition has a probability q of being correct after whatever checking the system performs. Then: P(entire chain correct) ≈ q¹⁰⁰⁰. At q = .999, .999¹⁰⁰⁰ ≈ .368. At q = .9999, .9999¹⁰⁰⁰ ≈ .905. So extending the agent’s horizon by another thousand dependent operations can require an enormous improvement in per-operation reliability merely to maintain the same probability of a correct final result. This is analogous to energy efficiency in a very interesting way: useful epistemic work = information processed × epistemic efficiency where epistemic efficiency means, roughly, the proportion of the information-processing effort that results in correct, usable information rather than error, ambiguity, duplication, or correction. 3. And there is an especially interesting asymmetry Energy inefficiency produces waste heat. Epistemic inefficiency produces corrective work. Suppose an agent makes an error at step 100 of a 1,000-step task. If the error is detected immediately, the cost might be modest: Ccorrection ≈ C100. But if it isn’t discovered until step 900, the system may have built 800 subsequent operations on top of it. Then the cost is something more like: Ccorrection ≈ C100 + C101 + … + C900 + Crework. So undetected information errors generate an epistemic equivalent of entropy production: work has been expended creating a state that must subsequently be undone. That is why I think your analogy becomes particularly strong when we move from error rate to error-correction burden. 4. There is a potential “epistemic efficiency law” We could define, very roughly, ηI = correct novel information incorporated into the solution          total information-processing effort Then the effective useful output of an agent becomes WI = Iprocessed · ηI. Now suppose increasing task complexity causes ηI to fall because: more claims require verification; more dependencies exist among claims; mistakes propagate farther; context becomes harder to maintain; agents have to communicate with one another; contradictory information must be reconciled; previous conclusions have to be revisited; verification itself becomes computationally expensive. Then simply increasing the amount of computation can produce declining useful output. That is remarkably analogous to the energy situation: more gross processing ↛ proportionally more useful processing. And this may be one of the deepest constraints on very long-horizon agents. 5. But there is a crucial difference from thermodynamic entropy I would not call this a literal second law of “information entropy.” Shannon entropy measures uncertainty in an information distribution. It isn’t synonymous with false information or mistakes. For example, a perfectly accurate statement about a random variable can have high Shannon entropy. Likewise, a confidently false proposition can have low Shannon entropy. So the more precise concept here is something like epistemic entropy, uncertainty, or error accumulation, rather than Shannon entropy itself. There is, however, a deep connection to information theory. Error-correcting codes demonstrate that reliable communication is possible despite noise—but only by introducing redundancy. And that gives us a potentially profound implication for agentic AI: Reliable long-horizon cognition may require increasing amounts of redundancy as the computational chain becomes longer. The agent needs to check itself, cross-check sources, replicate calculations, maintain provenance, revisit assumptions, etc. But all that checking consumes computation and energy. So you get a three-way tradeoff: more cognition ↔ more verification ↔ more energy. 6. This produces an interesting convergence of the two ceilings This may be the part of your idea that is most important. At first glance, energy constraints and error accumulation look like completely different problems. But they may actually interact: energy → computation → information processing → errors → verification → additional computation → energy In other words, epistemic reliability itself consumes energy. Suppose an agent can obtain a result with: 100 units of computation and 70% reliability, or 1,000 units of computation and 99.9% reliability. The latter is ten times the computational expenditure for vastly greater reliability. But if a 10,000-step task requires 99.999% per-step reliability to keep the overall result dependable, the required verification burden could become enormous. That creates a possible reliability tax on scaling. 7. And this gives a more interesting interpretation of the agentic wall The wall may not be: “AI can’t perform sufficiently many steps.” It may instead be: The amount of verification required to make sufficiently many steps reliable grows faster than the useful work accomplished by those steps. That’s a much more consequential proposition. Imagine increasing horizon H. Let useful gross work grow approximately as W(H) ∼ H, while the required verification effort grows as V(H) ∼ Hα. If α > 1, eventually: V(H) > W(H). At that point, extending the agent’s horizon becomes progressively uneconomic. And there is an even nastier possibility: if errors propagate through a dependency network, rather than a simple linear chain, the verification burden could grow much faster than H. 8. This also explains why modularity and checkpoints matter so much There’s an obvious way to fight this. Instead of one enormous chain: A → B → C → … → Z, divide it into independently verified modules: (A → B → C) (D → E → F) … with checkpoints between them. Now an error in module 3 doesn’t necessarily contaminate modules 4–20. This is analogous to error-correcting architecture. And that suggests an intriguing criterion for advanced AI systems: The crucial scaling variable may not be the maximum number of reasoning steps an AI can execute, but the maximum number of reliably composable steps. That is a much harder metric. 9. So I would formulate your idea this way There are potentially two coupled efficiency problems in AI scaling: physical efficiency How much useful computation can be extracted from a joule? and epistemic efficiency How much correct, durable, task-relevant information can be extracted from a unit of computation? The first encounters physical and engineering limits. The second encounters noise, uncertainty, dependency, verification, and error propagation. And because verification consumes computation, the two constraints ultimately couple: Energy → Computation → Epistemic reliability → Verification → Energy. That suggests something stronger than the usual “scaling eventually has diminishing returns.” The real long-horizon scaling problem may be that reliability requires an increasing fraction of the system’s computational budget. If that fraction rises sufficiently rapidly with task complexity, an AI could become vastly more capable at producing candidate solutions while making much slower progress in its ability to produce reliably correct solutions. And that would give your proposed information-side analogue of the energy efficiency constraint considerable theoretical bite.