inference

On November 12, at the opening ceremony of Baidu World 2026, Kunlun Chip, a subsidiary of Baidu, introduced its next product plans. The M100 has been launched as scheduled, and the M300 will be released next year.Among them, the M100 focuses on extreme cost performance; the flagship M300 has made comprehensive capability leaps in multiple dimensions such as computing power, storage, and connectivity. In addition, by 2028, multiple chips based on the new N architecture will also be launched to match differentiated application scenarios.Baidu founder Robin Li stated at the conference that the demand for AI computing power has shifted from being primarily training-focused to balancing both training and inference. The scaling of intelligent applications has driven a massive explosion in inference computing demand, "inference will become the main driving force for the long-term sustained growth of AI chips."

According to a Torus announcement, this energy storage company headquartered in Salt Lake City, USA, has completed a $500 million Series C financing round, led by Magnetar, with participation from ICONIQ, Origin Ventures, Ridgeline Partners, and Trinity Capital.Torus also launched a modular AI inference computing power product called Foundry, which integrates power supply, energy storage, heat dissipation, networking, and computing power into prefabricated equipment, with each site ranging from 1 to 20 megawatts, requiring approximately 12 to 16 weeks from contract signing to operation. The first batch of customers includes Hydra Host, and the company's goal is to deploy over 47,000 GPUs by the end of 2027.

NEAR AI tweeted that it has closed unauthorized access and continues to complete the investigation. NEAR AI Cloud will remain suspended until it can be safely restored. NEAR Protocol and other services are not affected, and the team stated that further updates will be provided within two hours.Previously, NEAR AI tweeted that it recently discovered unauthorized access to its internal systems and has suspended NEAR AI Cloud inference services as a precautionary measure to conduct an investigation. NEAR AI Cloud inference operates in hardware-isolated TEE to protect user prompt privacy.

According to Mira's official tweet, Mira launched the AI inference validation routing product Sterling on November 5, aimed at addressing the issue where the models users pay for in the current API market do not match the actual execution models (such as providers secretly downgrading models or sabotaging tool calls).Sterling conducts round-the-clock testing and public scoring of various API providers, evaluating dimensions including model identity, tool calls, parameter compliance, silent failures, and statement context. Users only need to change the Base URL to route requests to validated providers. A disclosed study conducted by Mira over a period of 4 months, covering 189 endpoints, found that 29 endpoints had 37 undisclosed changes.Sterling is currently in the Beta phase. The product supports pay-as-you-go, with settlement supporting USDC (via the x402 protocol) or $MIRA tokens. At launch, the inference prices for Claude, GPT, and Gemini can be up to 50% lower than the official listed prices.

According to Tech in Asia, the Singapore-based semiconductor startup Symatics released the Morphic Processing Unit (MPU) architecture at the 2026 Business and Charity Forum, aiming to improve AI response speed by optimizing data organization and processing methods.Symatics stated that the architecture employs row-column parallel processing and compiler planning execution mechanisms to reduce data transfer between memory and computing units, primarily targeting interactive AI and inference tasks. The company plans to collaborate with Singapore's research institutions and industry partners through an open architecture and licensing model. Currently, the MPU is still in the R&D stage, with technical validation work ongoing, and no progress has been disclosed regarding financing, prototype chips, or tape-out.

According to a report by the Science and Technology Innovation Board Daily, AI inference chip company MoXin Artificial Intelligence Technology (Shenzhen) Co., Ltd. recently announced the completion of nearly 1 billion RMB in C+ round financing, with investors including Renai Capital, Haitang Fund, Dongheng Petroleum, Biqiu Capital, Pala Capital, and Zhongcai Capital. The funds will be used for software and hardware upgrades of sparse dedicated inference chips, research and development of the next generation of chips, and collaborative research between industry and academia in sparse computing. The company disclosed that this financing round occurred about 5 months after the previous round of nearly 1 billion RMB, with a total of nearly 2 billion RMB raised across both rounds.In the C round financing disclosed in May this year, institutions such as Shenzhen Capital Group, Greater Bay Area Common Home, Liding Capital, and Yunsheng Capital participated, while existing shareholders like Kaixuan Venture Capital, Shengjing Jiacheng, and Yanshan Technology continued to invest. MoXin was founded in 2018 and is headquartered in Shenzhen, primarily developing AI inference chips, computing cards, and supporting software for data centers and industry clients. Founder and CEO Wang Wei has a master's background in Electronic and Computer Engineering from Carnegie Mellon University and has previously worked at Qualcomm and Intel.The company has launched computing cards such as the S4, S30, and S40, equipped with one, three, and four self-developed Antoum chips, respectively. The company plans to launch a new generation of computing card SP200 by the end of this year and release an upgraded software stack SECA. According to the company, its inference clusters have been deployed in regions such as Northwest, Southwest, East China, and North China, and partnerships have been established with leading telecommunications operators; revenue is expected to grow several times by the third quarter of 2026, but the revenue amount and growth comparison base period have not yet been disclosed.

DeepSeek announced the full open-source of its internal inference framework DeepInfer, and simultaneously open-sourced the production-level token encoding and decoding component deepseek-recipe. DeepInfer integrates high-performance operators and a deeply optimized inference engine, supporting distributed inference, speculative decoding, SWA Bounded Replay, and structured output, compatible with NVIDIA and Huawei Ascend platforms.DeepSeek stated that it has implemented high throughput and low latency large-scale inference services for DeepSeek V4.1 Flash based on DeepInfer, and future inference code and optimizations for new models will continue to be updated here. deepseek-recipe is fully aligned with the processing logic of the DeepSeek official API, aiming to ensure that the community receives an experience consistent with the official deployment. The open-source repository and technical report have been released simultaneously.

Micron Technology Vice President Li Zhe stated at the Korea Semiconductor Exhibition SEDEX 2026 held at COEX in Seoul that the company is developing Edge Inference Memory (EIM) for physical AI. The industry expects this product to enter mass production in the second half of 2028. Micron is researching the use of EIM in LPDDR6 to meet the demand for low-power DRAM in applications such as autonomous driving and humanoid robots, and is in discussions with major customers regarding related products.According to a report by the Science and Technology Innovation Board Daily, EIM is positioned between AI chips and memory, utilizing a method of stacking LPDDR on logic chips to process inference data at high speed, although the computing architecture has not been fully disclosed. Li Zhe mentioned that current robotic AI models have about 1 billion to 2 billion parameters, with low bandwidth requirements; however, as more advanced visual language models emerge, LPDDR alone will not suffice, and HBM will eventually be used for humanoid robots. Micron anticipates that future high-performance humanoid robots will require approximately 200 GB of DRAM and 4 TB of storage, while current service robots require about 12 GB of DRAM and 256 GB of storage.Micron CEO Sanjay Mehrotra stated that the storage capacity of a single humanoid robot is about 10 times that of an ordinary L2+ passenger vehicle. The storage industry may enter a period of sustained demand growth for several decades around 2030, driven by the need to synchronize data from hundreds of sensors and run large models locally for millisecond-level control. Counterpoint Research predicts that global DRAM demand for humanoid robots will increase from approximately 8,300 Tbit this year to about 172,000 Tbit by 2030.

Ethereum co-founder Vitalik Buterin shared that the cost of Bitcoin mining is roughly half derived from capital expenditures on computing power equipment and the other half from electricity consumption; in contrast, the cost of AI inference is almost entirely dominated by capital expenditures on computing power. This difference reflects the distinct cost structures of the two computing businesses. Bitcoin miners need to continuously pay for electricity and determine their profit margins based on mining machine efficiency, coin prices, and network difficulty changes; AI inference, on the other hand, relies more on investments in expensive hardware like GPUs, as well as equipment utilization and computing efficiency.The industrial boundaries of both are also changing. As the demand for AI computing power grows, some Bitcoin mining companies are beginning to shift their existing electricity resources and data center facilities towards AI and high-performance computing businesses, hoping to transition from pure mining to computing infrastructure services. Some mining companies are leveraging previously allocated electricity capacity for mining to enter the AI data center market. However, transformation does not mean that cost pressures disappear: mining operations are more directly exposed to fluctuations in electricity prices and Bitcoin prices, while AI computing businesses must bear risks related to GPU procurement, depreciation, and underutilization. Both types of businesses compete for electricity and computing resources, but ultimately, profitability depends on how these resources are converted into revenue.

Blockworks released a status report on cryptocurrency inference service providers, stating that global computing power investment is substantial and demand still exceeds supply; the market capitalization of liquidity crypto AI is approximately $24.9 billion, lower than Crusoe's $30.9 billion, and no decentralized computing network has disclosed verifiable revenue exceeding tens of millions of dollars.The report compares seven crypto-native inference providers on OpenRouter: Venice, Phala, Darkbloom, io.net, AkashML, NEAR AI, and Chutes. Over the past month, the seven have processed an average of about 211 billion tokens daily, accounting for about 1% of the platform's daily average of 21.9 trillion tokens. Venice processed about 87 billion tokens daily, with the team stating in July that its annual recurring revenue reached $100 million, with OpenRouter contributing less than 10%; Phala averaged about 54.6 billion tokens daily over the past week, up from about 5 billion in July.Chutes is the Bittensor inference subnet SN64, operating approximately 4,350 GPUs, with the team claiming to process 160 billion tokens daily, with OpenRouter channels averaging 5.7 billion. Akash has 283 active GPUs, down from a peak of about 897, with an annual leasing fee of about $3.7 million, and GPU incentives have ended. io.net's demand has nearly doubled over the past month to an average of 14.8 billion tokens daily.

According to official news, the P2P AI inference network Antseed has officially been released, while also disclosing that the nonprofit foundation Antseed Foundation has completed a $2.4 million token round of financing. Spark Capital led the investment, with participation from Collider, DCG, North Island Ventures, Reciprocal Ventures, Relay Capital, and Venice.ai.Antseed aims to apply BitTorrent's peer-to-peer model to the AI inference field, allowing different providers to sell models and Agent services directly. After running Antseed's local router on their computers, users can connect tools like Claude Code, Codex, OpenCode, etc., to the same address. Each call is selected by the router based on price, speed, reputation, and privacy conditions, with fees settled directly in USDC on Base. Providers can run their own local models, fine-tune models, or offer other AI services and set their own prices.The biggest difference between Antseed and OpenRouter is that there is no unified platform responsible for forwarding in between. Buyers connect directly to providers, while Antseed is responsible for discovery, routing, reputation, and payment protocols. Ordinary providers can still see the requests they handle; users can also choose providers verified through TEE, allowing requests to run in a trusted execution environment.Antseed is also continuing to improve verification mechanisms such as model fingerprints, signature responses, and audits to prevent providers from misrepresenting cheaper models as more expensive ones. The code has been open-sourced, and the network is operational, although GitHub is currently addressing compatibility issues with Codex, Claude Desktop, and others.

ARK Invest founder Cathie Wood stated that the cost reductions brought about by innovative platforms such as AI will gradually transmit to broader economic sectors, enhancing productivity and corporate profitability, and driving inflation rates down to levels below most investors' expectations, a phenomenon referred to as "benign deflation."Cathie Wood believes that the current AI inference costs are decreasing by 99.99% annually while maintaining performance levels. Meanwhile, OpenAI's annual revenue run rate has increased from $20 billion to $70 billion, indicating that the decline in costs is driving rapid growth in demand.ARK Invest expects that real GDP growth rates could reach high single digits, although this prediction seems "crazy" to most people; in a stronger real growth environment, interest rates may actually rise.

According to The Information, Apple (AAPL.O) is considering a return to the server market and has been in talks with Nvidia (NVDA.O) about using networking technology.The servers that Apple plans to launch are expected to hit the market in 2029, targeting the artificial intelligence inference sector.

Salesforce and NVIDIA jointly launched KOA, which is the first CRM inference model built on NVIDIA NeMoTron.

Zhipu released the lightweight flagship model GLM-5.3 Flash, aimed at competing with DeepSeek V4 Flash, and utilizes over 100,000 domestic chip clusters to provide inference services. The model has a parameter scale of approximately 300B, about 40% of GLM-5.3, and is roughly on par with DeepSeek V4 Flash; it adopts the MoE architecture, with actual activated parameters around 18B, possesses native multimodal capabilities, supports image and video input, and is the first new model with multimodal capabilities following Zhipu's focus on coding strategy.The comprehensive evaluation score exceeds that of the previous flagship GLM-5.2, is on par with Claude Opus 4.8, and is higher than the relevant scores of DeepSeek V4 Pro official version. The pricing is 0.8 yuan per million tokens for input, 2.8 yuan for output, and 0.23 yuan for cache hits, approximately one-tenth of GLM-5.3, with a half-price discount offered for the first two weeks after launch. The model is fully compatible with domestic computing power, online traffic is handled by domestic chips, and Zhipu claims that the cluster hardware efficiency and unit token cost have reached levels comparable to mainstream NVIDIA GPUs; this model was previously tested anonymously on overseas platforms under the code name Ox Alpha.

Sources from relevant personnel at Zhongke Shuguang reveal that the company will launch a new generation of token acceleration solutions aimed at large model inference, focusing on addressing issues of redundant computation and data scheduling during the inference process.Currently, Zhongke Shuguang's ParaStor distributed storage has been applied to the country's first fully domestically produced 100,000-card AI supercluster Shuguang 8000 (Dengfeng), providing data throughput and elastic scaling support for large-scale computing tasks. The latest report from CCID Consulting shows that Zhongke Shuguang ranks first in four sub-industries: AI, education, embodied intelligence, and autonomous driving.

Cerebras released its fourth-generation AI inference system CS-4 this week, based on the same 5nm WSE-3 wafer, achieving double the performance by doubling the clock frequency and power consumption. A single CS-4 cabinet accommodates 3 wafers (CS-3 has 2), featuring a modular "backpack" design that simplifies manufacturing and deployment, with a TDP of approximately 125 to 135kW. The CS-4 can provide an inference speed of nearly 4000 tokens/second/user, about twice that of the CS-3, and supports decomposed inference with heterogeneous systems such as AMD and AWS Trainium.Cerebras claims that the CS-4 offers about 2000 times the on-chip memory bandwidth of NVIDIA's Rubin (43PB/s), but the 44GB SRAM capacity remains unchanged, and long-context inference still requires multi-wafer stacking. For example, with the DeepSeek V4 Pro (1.6T parameters), approximately 20 systems are needed for a 1M context window, and about 40 systems are required for 256 concurrent users, corresponding to a CAPEX exceeding 20 million USD. Cerebras is collaborating with clients such as OpenAI and plans to achieve approximately double performance improvements each year, aiming for a 20-fold throughput increase by 2027. The "backpack" cabinet design of the CS-4 will continue into the next-generation "Nexus" platform.