d-Matrix and Gimlet Labs Partner to Boost Agentic AI Inference Performance

The partnership aims to integrate d-Matrix Corsair accelerators into Gimlet Cloud alongside traditional GPUs. "Under the partnership, Gimlet plans to integrate d-Matrix Corsair accelerators into Gimlet Cloud alongside traditional GPUs." The collaboration expects up to 10x improvements in latency and throughput per watt compared to GPU-only deployments. "The companies say this split can deliver up to 10x improvements in latency and throughput per watt compared with GPU-only deployments." In the hybrid architecture, GPUs handle compute-heavy stages of inference, while memory- and latency-sensitive operations are routed to Corsair. "In the hybrid architecture, GPUs handle compute-heavy stages of inference, while memory- and latency-sensitive operations are routed to Corsair." Corsair ships as a standard PCIe card with air cooling. "Unlike many high-end accelerators, Corsair ships as a standard PCIe card with air cooling, enabling deployment inside existing GPU servers without specialized chassis or unconventional plumbing." d-Matrix's Jetstream NICs move data between Corsair and GPUs over standard Ethernet. "d-Matrix's Jetstream NICs move data between Corsair and GPUs over standard Ethernet, simplifying integration at scale and improving utilization." Initial targets for the joint solution include frontier AI labs building high-capability models, interactive coding, and autonomous agents. "Sheth said that initial targets include frontier AI labs building high-capability models. “Use cases include interactive coding and autonomous agents, where speed matters – both for human users and for machine-driven workflows,” Sheth said."