Y Combinator Discusses Non-GPU Computing Routes This Week

This week, Y Combinator's Paper Club shifted its focus from conventional model training to alternative computing. The core judgment is that current artificial intelligence is tied to the same path as Transformers, backpropagation, and GPUs, and as computing efficiency begins to peak, the next generation of machines may not necessarily use traditional digital hardware.

The discussion unfolded along three experimental lines. Francois Chaubard first explained why to move away from backpropagation and introduced the zero-order optimization method SOMA. The public paper defines SOMA as a model combination for piecewise optimization: it estimates updates using two forward passes and random perturbations without calculating gradients, allowing training even with non-differentiable losses. He previously trained a 1 billion parameter LSTM for next-word prediction using methods from zero-order textbooks and found that the SPSA proposed by James Spall performed the best, but the gradient estimation error increases linearly with model size, meaning the perturbation calculation required for a 10 billion parameter model would exceed that of a conventional forward and backward pass.

Optical computing was explained by Stanford postdoc Ilker Oguz. He implemented diffusion models using light propagation in diffraction layers and pointed out that photons are about four orders of magnitude higher than electrons in terms of loss and bandwidth, with approximately 99% of intercontinental data transmission already using light. The real reason for the lack of widespread adoption is not that light itself is slow, but that the conversion between light and electrons still consumes energy. He stated that the next milestone is to achieve an end-to-end optical system at the scale of 1 billion parameters, rather than just creating another small demonstration.

Brain-like chips were summarized by Alok Vasudev, a Stanford electrical engineering PhD and partner at a $1.5 billion venture capital fund. He identified the most likely direction for recent developments as integrated storage and computing, acknowledging that this only qualifies as neuromorphic under a broad definition. Closer to the brain, sparse pulse and event-driven chips still lack a large-scale training stack and customers. He also noted that strong inhibition between cortical columns allows different assemblies to learn independently, a structure that chips have yet to replicate.

Biocomputing was explained by Sean Cole, CEO of Parasma, in collaboration with Cortical Labs. About 200,000 human neurons reprogrammed from adult donor cells are laid out on a multi-electrode array, receiving game states through electrical stimulation and mapping outputs to movement and shooting, navigating and finding enemies in the 1993 game Doom. Cortical Labs' CL1 is priced at approximately $35,000, with single-module life support capable of maintaining cells for up to six months, accommodating around 800,000 neurons. In 2022, the company published DishBrain, using human and rodent neuron cultures to play Pong, which took over 18 months to develop at that time.

This is not an order for an immediate replacement of GPUs. Buyers remain cloud vendors, model laboratories, and sovereign computing projects, with funds primarily flowing to Nvidia GPUs, high-bandwidth memory, and data center power. The driving force is research funding and early venture capital seeking a second curve: optical interconnect companies sell bandwidth, integrated storage and computing chips sell inference efficiency, and biocomputing currently sells experimental platforms and drug screening interfaces. Beneficiaries are those who can connect to existing GPU clusters' peripherals for interconnect and memory solutions, while those under pressure are new foundations that remain at the demonstration stage without training stacks and replicable output.

The human brain operates at about 20 watts, and the CL1 reduces latency from milliseconds to sub-milliseconds, but Sean Cole lists distributed inference as the most challenging open problem in biocomputing. Optical computing has yet to deliver a 1 billion parameter end-to-end system, and zero-order methods have not yet proven stable reproducibility on the same random seed. All three lines currently explain the bottlenecks of the GPU path, but none have yet produced capacity that can replace training clusters.

Source: Public Information