Global Infrastructure SherpaHuawei and a Chinese carrier jointly develop an intelligent computing service platform
Huawei and a Chinese carrier jointly developed an intelligent computing service platform to address challenges in AI adoption, including inference failure, costs, and speed. The platform uses KV cache technology and algorithm optimization to improve throughput, reduce inference costs, and shorten response times.
Counterparts (2)
Deal Analysis
- Joint development of an intelligent computing service platform.
- Addresses critical AI adoption challenges (inference failure, costs, speed).
- Partnership between global technology company Huawei and a Chinese carrier.
- Utilizes KV cache technology and algorithm optimization for performance improvements.
- Focuses on the data center sector in China.
Source Intelligence
The platform uses the KV cache technology to improve storage resource utilization and supports inference applications of different large models like DeepSeek and Qwen. It optimizes cost-effectiveness by innovatively eliminating repeated computing via querying. Through the collaboration of on-chip memory, DRAM, and AI storage, the platform enables PB-scale KV cache storage. This improves the overall throughput by more than ten times, reduces inference costs by about 50 percent, and shortens response time to less than one second. In addition, algorithm optimization addresses challenges like low KV cache hit ratios and inference failure due to long-sequence inputs in research report analysis. Serving as the foundation for AI, the platform has been deployed at scale at the group to enable multidimensional innovation across services, including internal IT systems, B2C services, B2B services, and B2H services.
"Yuan presented an intelligent computing service platform, jointly developed with a Chinese carrier, that tackles these challenges."
"Yuan presented an intelligent computing service platform, jointly developed with a Chinese carrier, that tackles these challenges."