Musk Says AI Struggles with High-Performance Programming
Elon Musk, founder of xAI, stated on social media that the C/C++ tech stack used to train AI models is still handwritten by human engineers. He pointed out that artificial intelligence has not yet reached the level of independently writing "extremely high performance software," which typically has stringent requirements for execution efficiency, memory management, and low-level optimization. However, Musk also indicated that this situation is only "current" and he expects AI will eventually be capable of writing such high-performance code, characterizing this limitation as a temporary rather than a permanent technical boundary. This statement directly relates to the AI large model training infrastructure itself—the underlying frameworks and computing kernels relied upon to train large-scale neural networks, which usually need to be written in system-level languages like C/C++ to maximize the computational efficiency of GPU/TPU hardware. The difficulty and optimization requirements for this are much higher than for general application-level software development. Mechanically, this statement reveals that despite the widespread promotion of AI code generation tools (such as GitHub Copilot, Cursor, and various large model-assisted programming products) as significantly enhancing software engineering efficiency, in the niche area of extreme performance low-level code required for training AI models, human engineers remain an irreplaceable core productivity force. The scarcity and bargaining power of these high-end system-level engineers are unlikely to be depressed in the short term by AI, and may actually increase due to the ongoing expansion of demand for AI training infrastructure.
ABAB AI Insight
xAI was founded by Musk in 2023 and quickly built a large-scale GPU cluster "Colossus" for training the Grok series of models. Its infrastructure team is known for pursuing extreme hardware utilization and training efficiency—Colossus went from construction to 100,000 GPUs online in about 122 days, supported by a large amount of handwritten, highly optimized low-level training code. xAI continues to invest heavily in purchasing NVIDIA GPUs and building supporting data centers and power infrastructure, while also attracting top system engineers and compiler experts with high salaries to maintain and optimize its C/C++ training tech stack. Musk's statement somewhat justifies the company's ongoing investment in a costly human engineering team—indicating that replacing top system-level engineers with AI at this stage is unrealistic, and capital should still prioritize these scarce human resources rather than hoping to directly replace them with AI programming tools. This is similar to the semiconductor industry's long-standing situation where "chip design automation tools (EDA) are widespread, but top chip architecture design still heavily relies on the experience of senior engineers"—tools can significantly enhance efficiency, but at the lowest levels closest to physical hardware limits and with the highest performance sensitivity, human expert experience is still difficult to fully replace with algorithms in the short term. The current AI industry is at a stage where "application layer code generation capabilities are rapidly improving, but low-level system performance engineering remains a core barrier for humans." This essentially reflects a structural layering in the process of technological substitution: the pressure of AI replacing software engineering positions is gradually penetrating from the "application layer" to the "low-level system layer," but has not yet fully crossed over. The mechanism behind this is that extremely high-performance software's optimization for hardware characteristics (such as cache hit rates, instruction pipelines, and memory alignment) heavily relies on a deep understanding of specific chip architectures and repeated empirical tuning, knowledge that currently resides mainly within the experience of senior engineers rather than being fully learnable and generalizable from existing large model training datasets. This gives low-level system engineering positions a relatively stronger moat amid the wave of AI impacting the software industry for the time being.
ABAB News · Cognitive Laws
- The application layer falls first, the low-level is always the last bastion.
- AI can write code, but cannot express reverence for hardware.
- Scarcity does not depend on the level of the position, but on whether AI can learn.