Alibaba Employee Provides An Overview Of The Chinese LLM Program

(CTN News) – Across the Pacific Ocean, Chinese Alibaba tech companies are gathering resources and talent to narrow their gap with OpenAI. It’s a rare glimpse into how Alibaba develops large language models, one of a range of Chinese internet giants trying to match ChatGPT’s capabilities.

X user Binyuan Hui shared his daily schedule on X, mirroring a post by OpenAI researcher Jason Wei that went viral.

It’s amazing how similar their typical days are, with wake-up times at 9 a.m. Until 1 a.m. Meetings follow, followed by coding, model training, and brainstorming. After they get home, they keep running experiments and pondering ways to improve their models well into the night.

Their leisure time is characterized differently. According to Hui, the Alibaba employee, he reads research papers and browses X to keep up with “what’s happening around the world.” And unlike Wei, Hui doesn’t have a glass of wine after he gets home.

Chinese tech talent with top university degrees is joining tech companies in droves to build competitive AI models, so this intense work regime is not uncommon.

I think Hui’s busy schedule is partly a reflection of a desire to keep up with, if not outpace, Silicon Valley companies in the AI space. This seems different from involuntary “996” hours at more “traditional” Chinese internet businesses that have heavy operations, like video games.

In fact, even renowned AI investor Kai-Fu Lee works hard. When I interviewed Lee about his newly minted LLM unicorn 01.AI in November, he admitted late hours were the norm, but people were willing to work hard. His staff messaged him at 2:15 a.m. with excitement about 01.AI.

Intense work ethics indicate the urgency of tech firms’ remits and the speed at which they’re rolling out LLMs.

Qwen has open sourced foundation models trained on English and Chinese data. In the largest of these, there are 72 billion parameters, which describe how much knowledge the model gains from historical training data. (GPT3 from OpenAI is believed to have 175 billion parameters; GPT4, its latest LLM, has 1.7 trillion. But it could be argued that the aim of the particular LLM will be more important.) The team has also been quick to introduce commercial apps.

The online retailer Tmall and Alibaba’s enterprise communication platform Dingtalk started integrating Qwen last April.

Venture capital firms and corporate investors are betting across multiple contenders in China’s LLM space so far. Alibaba isn’t just building its own LLM; it’s also investing in startups like Moonshot AI, Zhipu AI, Baichuan and 01.AI.

In the face of competition, Alibaba’s multilingual move could be a selling point. Several Southeast Asian languages were added to the LLM in December. SeaLLM can handle Vietnamese, Indonesian, Thai, Malay, Khmer, Lao, Tagalog and Burmese.

SeaLLM can potentially tap into these services down the road through Alibaba’s cloud computing business and acquisition of ecommerce platform Lazada.

SEE ALSO:

Thailand’s National Credit Bureau Warns Over US$16 Billion in Mortgage Defaults