“Chips or Not, Chinese AI Pushes Ahead: A Host of Chinese AI Startups Are Attempting to Write More Efficient Code for Large Language Models”, 2024-08-23 (; similar):
[chip starvation driving Chinese DL flight to cheap/specialized/edge uses, away from SOTA leading-edge LLMs like GPT-5] Chinese technology companies cut off from the world’s most advanced chips for artificial-intelligence computing are rallying around an appealing message from industry pioneers: to make money, they might not necessarily need them.
…01.AI, a unicorn backed by Alibaba and Xiaomi, employs a lower-precision training format that reduces the energy and time needed to train machine-learning models. [FP16?] The format, used in the U.S. by companies including Google, can accelerate a model’s output, according to Nvidia researchers. In China, “we don’t have a lot of [graphics processing units], and that forces us to develop very efficient AI infrastructure and inference engines”, 01.AI founder Kai-Fu Lee said, citing a lack of funds as a reason for low chip supplies. The company has said its chip-cluster failure rate, a measure of how often groups of connected chips fail to work together, is lower than the industry average.
…In the quest for consumer buy-in, some Chinese companies seek to develop specialized applications rather than focus on creating the biggest and best models, analysts say. A recent KPMG report said AI investors in China in the second quarter “focused on AI-enablement rather than on LLM offerings”, including in areas like robotics and improving workplace efficiencies.
…Some industry experts expect that smaller-size models that can power AI features on smartphones and laptops, or “edge AI models”, will be the next game-changer. “This year is about small models”, said Winston Ma, an adjunct professor at New York University School of Law and an adviser at Dragon Global, an AI-focused family office. Smaller models using less training data are faster, benefiting real-time applications with specific functions, he said. AI unicorn Baichuan is working with Qualcomm to integrate a smaller LLM in its AI PC in China, people familiar with the matter said. Samsung has used models from Baidu and ByteDance in its smartphones in China. “Focusing on edge doesn’t mean lowering the tech requirement, just shifting to an area where higher computing power isn’t the most critical requirement”, said Boris Van, an analyst at Bernstein Research.
…“It is important to figure out how we can achieve better results through engineering instead of blindly investing in computing power”, 01.AI’s Lee said at the AI industry conference in China… Companies are also researching how to combine different types of chips to avoid relying on any one kind of hardware. “We shouldn’t think that not having the most advanced AI chips means we won’t be able to lead in AI”, Zhang Ping’an, a Huawei senior executive in charge of its cloud-computing business, said at the July AI conference. “We should abandon this viewpoint in China.” [previously]