Perplexity has open-sourced the local inference engine Lily, specifically designed for the Qwen3.6-35B-A3B on Apple chips, to facilitate hybrid computing for its Computer products, preventing edge computing power from bottlenecking tasks.

Lily operates as a single-process runtime: Rust loads weights and drives the generation loop, with a handwritten Metal kernel executing the model, and the execution path does not include PyTorch or MLX. It provides a minimal OpenAI-compatible chat completion interface, performing only greedy decoding. The code is available in the pplx-garden repository, licensed under Apache-2.0.

The target weights are MLX affine 4bit, with a group size of 64 for Qwen3.6-35B-A3B, approximately 19.4 billion bytes; during loading, it verifies the precise architecture and quantization layout. Dense versions, smaller Qwen, BF16, GGUF, AWQ, GPTQ, int8, and fp8 are not supported. The official benchmark locks the fixed revision number of the mlx-community checkpoint. Hardware requirements include Apple GPU family 10 and above (M5 and newer), with Metal tensor operations requiring a newer macOS, and the Rust version is pinned by the toolchain file.

The company measured an average pre-fill of about 1.23 times and decoding of about 1.35 times relative to MLX-LM on an M5 Max with 40 cores and 128GB of unified memory. With a 4K prompt and 4K decoding context, the pre-fill is about 5749.9 tokens/second and decoding about 186.6 tokens/second, compared to MLX-LM's approximately 4737.5 and 140.9. The maximum pre-fill gain comes from GPU-resident expert routing at about 89% and dequantization integrated into grouped GEMM at about 77.4%; speculative decoding, however, slows down by about 18%. In quality comparisons, perplexity is only 0.04% higher, with the first token consistency rate at about 96.35%.

Qwen3.6-35B-A3B is a sparse mixture of experts: total parameters are about 35 billion, with approximately 3 billion activated per token, and the router selects 8 out of 256 experts at each step, plus shared experts; the structure mixes full attention layers with Gated DeltaNet. Hybrid Compute offloads heavy inference to the cloud, while the local model handles private files and applications. The product side of hybrid computing lists macOS 15+, with a minimum memory of 24GB and 32GB for more stability.

Mechanically, the buyers are Computer users and developers running private tasks on Mac, while the seller is Perplexity, which open-sources the edge engine while keeping cloud inference within its own products. The event is driven by edge-cloud division of labor: weights come from the Alibaba-affiliated Qwen open-source line, and the runtime is specialized by a U.S. company using Metal; the beneficiaries are those needing to reduce local ignition latency on M5-level high-memory machines, while the burden falls on the general framework that handles multiple models, yielding throughput at this checkpoint.

Public information does not provide a complete comparison for non-M5 models, nor does it commit to supporting a second model in the future.

Source: Public Information