Alloy Compute says it has built an AI inference system that splits language model work between AMD graphics chips and programmable accelerators to cut response times. Alloy Compute Splits Inf
Alloy Compute says it has built an AI inference system that splits language model work between AMD graphics chips and programmable accelerators to cut response times.
The company described the design on Jul. 31 as a disaggregated architecture, one that sends each phase of a model's execution to the processor best suited for it. Most inference systems run every stage on a single chip type. AMD graphics processors take prompt prefill and attention, while FPGA accelerators handle token decoding and mixture-of-experts layers.
Alloy Compute has not published benchmark results and said it is still integrating the system while measuring latency, throughput, energy use and traffic between the two chips. Customer evaluation programs have not opened.
Also Read:Bitcoin ETFs Absorb $233.1M As A Single Fund Supplies Most Of It
Founder and Chief Executive Nour de Vos said prefill, attention and token generation make different demands on hardware, so running a whole model on one architecture wastes capacity. The company said it is prepared to guarantee first-token responses under 200 milliseconds, though only for deployments that stay inside set limits on models, input length and concurrency. Early work covers quantized Qwen models.
De Vos came to AI infrastructure from large-scale cryptocurrency mining, where profits hinged on keeping machines busy and power bills low. He has said that same system-level thinking, treating chips, memory, networking and software as one unit, shaped this design.
Read Next:Polymarket Traders Give Spider-Man A 91% Shot At A Historic Debut
