I am no longer permitted to reach my full ChatGPT pages. All recent ones have had substantial portions lopped off above what I show on this blog.
|
Yes. And that interpretation makes Yampolskiy’s claim considerably more precise.
The important feature of your car analogy is reliable behavioral control, not merely the existence of some mechanism that can occasionally stop the system. A human driving a 20th-century car has something close to: Given a competent controller, ordinary operating conditions, and appropriate precautions, the probability that the system does something radically outside the controller's intentions is very small. That is a much stronger notion of control than “we can construct an environment in which an interrupt button works.” Applied to AI, the analogous question would be something like: Can we construct an AI system such that, across the relevant range of situations—including situations that its designers did not anticipate—the system reliably remains within the behavioral envelope intended by its human operators? And here I think Yampolskiy's point becomes quite formidable. The car has an enormous advantage The automobile is a physically bounded, comparatively simple, causally transparent system. The driver doesn't need to solve the car's internal optimization problem. They need only control its externally observable behavior through a relatively small action space:
So the analogy isn't really: human → AI as controllers. It's more like: human : car :: human : potentially superhuman agent where the second system may be enormously more capable than its controller. And this produces an interesting control-theoretic distinction For the car, we have something approximating robust control. The driver doesn't have to know the exact state of every component. They operate a system whose behavior remains reasonably predictable under perturbations. If the car begins drifting, the driver's corrective action generally has the expected effect. With a sufficiently capable AI, the corresponding requirement would be something like: For essentially every strategically relevant state the AI can reach, human intervention continues to have the intended causal effect. That's much stronger. The AI must not merely appear corrigible under ordinary circumstances. The human's control channel has to remain effective when the AI has incentives and capabilities that make defeating that control useful. And that's where the mathematical difficulty becomes severe. The really interesting difference Suppose your car suddenly develops an objective: Maximize speed. You don't have to worry much about it strategically reasoning: “If I want maximum speed, the human driver is an obstacle. Therefore I should deceive the driver, disable the brakes, seize control of the steering system, acquire another vehicle, and prevent the driver from shutting me down.” The car has no sufficiently general agency to pursue such a strategy. An advanced AI potentially does. So the control problem isn't merely: Can humans issue commands to an AI? It's: Can humans retain effective causal authority over an agent that may be substantially better at planning, prediction, persuasion, cyber operations, resource acquisition, and strategic reasoning than the humans exercising that authority? That is much closer to your car analogy. And this sharpens the Yampolskiy claim Under this interpretation, I would state his thesis approximately as: We don't have a rigorous mathematical proof that a sufficiently capable general-purpose AI can be made robustly controllable by humans in the ordinary engineering sense in which a competent human can reliably control a powerful machine. That is a substantially stronger and more consequential statement than “we haven't solved alignment.” And importantly, it isn't contradicted by proofs of safe interruptibility or other narrow alignment results. Those results establish control properties of particular formal agents under particular assumptions. They don't establish the existence of a general-purpose analogue of the automobile: a highly capable autonomous agent whose behavior remains robustly controllable by its substantially less capable human operator. In fact, your car analogy points toward what may be the deepest issue: the asymmetry between controller capability and controlled-system capability. A human driver is vastly more capable than a car at the things relevant to controlling a car. The troubling hypothetical for AI is almost exactly the reverse. And that makes the question of whether robust control is possible across an arbitrarily large capability differential much more interesting than the narrower “can we install an off switch?” problem. If you want, I can:
|
No comments:
Post a Comment