Jaya Gupta: The Inference Layer Becomes the Second Largest Financing Track

The inference layer is becoming the second largest financing direction outside of laboratories, with teams justifying large funding rounds by pointing to data center and computing power demands, continuously attracting capital.

More inference companies are receiving funding, even though the switching costs for underlying products are extremely low, allowing users to migrate almost seamlessly; new laboratories will rent out their own computing power, further injecting additional supply into the same market.

Companies that previously focused on vertical integration around batch processing workloads and post-training are now shifting to pure inference businesses, joining a continuous price-cutting cycle that creates a permanent reverse auction. Providers are repeatedly undercutting each other, with offers such as half price for Fireworks, half price for Baseten, and an additional 10% off for GLM or Kimi; every new entrant or new laboratory releasing capacity accelerates the downward pressure on prices.

Application companies currently have a plethora of choices, allowing for fierce price comparisons and driving prices to extremely low levels; once a significantly cheaper option appears, they can shift millions of dollars in annual expenditures overnight, as the cost of switching technology is essentially just a line of code.

Easy access to capital, additional computing power from new laboratories, low stickiness, and ongoing price pressure combine to make the inference layer one of the weaker defensive layers in the tech stack, even though it still attracts high valuations; savvy infrastructure teams can easily raise $100 million in seed rounds after new laboratories.

Capital continues to flow into equity of inference companies, while the increased computing power supply from new laboratories and inference providers creates selling pressure; the application layer, as a buyer, utilizes standardized interfaces and near-zero switching costs to drive prices down; inference providers face profit pressure, while application companies and end users benefit from lower costs.

Fireworks processes over 40 trillion tokens daily and has annual revenues exceeding $1 billion, while Baseten is valued at $13 billion and Together AI at $8.3 billion, indicating that large financing and valuations coexist amid intense price competition.

Source: Public Information

Fireworks was founded in 2022 by the core team of PyTorch from Meta, initially focusing on open-source model inference and post-training; Baseten started with data science tools and later shifted to multi-cloud orchestration and dedicated deployment. Both companies completed several rounds of large financing between 2025-2026, with valuations reaching $17.5 billion and $13 billion respectively. A similar path is seen with Together AI, which began as a research-oriented cloud platform and quickly transitioned to production-grade inference with support from Saudi and U.S. capital. These companies previously relied on model openness and API compatibility for expansion but have historically faced customer concentration risks, with some providers heavily dependent on a single application client for over half of their revenue.

Capital is mainly flowing into computing power procurement, data center expansion, and engineering team growth, motivated by capturing the growth of inference demand outside laboratories, while new laboratories rent out idle or excess computing power to dilute their own costs. Venture capital firms and strategic investors (including chip manufacturers) shift funds to directly quantifiable token processing volumes and GPU utilization after laboratories complete large-scale financing, forming a resource mobilization sequence of "first laboratories, then inference." Some providers are also locking in enterprise clients through dedicated deployments, compliance isolation, and multi-cloud scheduling, attempting to establish differentiation outside of the price war.

This pattern is reminiscent of the cloud infrastructure price wars of the 2010s, where AWS, Azure, and Google Cloud competed for developers through continuous price cuts and standardized APIs, ultimately commoditizing the basic computing layer; it is also similar to the early ISP bandwidth market, where pricing power quickly shifted to buyers after oversupply. The current inference layer is transitioning from a late expansion phase to a power struggle, with leading companies attempting to solidify their positions through scale and dedicated models, but new entrants and laboratories' own computing power continuously dilute barriers.

Essentially, this represents a transfer of pricing power. The combination of standardized OpenAI-compatible interfaces, near-zero switching costs, and excess computing power supply allows buyers to compare prices in real-time among multiple providers and quickly migrate expenditures; the influx of capital further increases supply, reinforcing the reverse auction mechanism, leading to profit pressure on the inference layer, while the application layer gains lower costs and greater choice. The additional computing power released by laboratories accelerates this transfer, turning inference from a potentially differentiated layer into a replaceable commodity layer.

ABAB News · Cognitive Laws

In layers with low switching costs, the more capital-intensive, the harder it is to form barriers.
In oversupplied markets, prices only trend downward.
The easiest tracks to finance are often the most easily replaceable tracks.

ABAB AI Insight