The field of open source language models is rapidly evolving, challenging market-leading models like GPT-4. While the best open source models have largely come from outside the US, a US-based company called 01.AI is positioning itself to benefit from emerging innovations in the field. This article delves into the implications of open source AI models and discusses the potential impact of 01.AI’s model, Yi-34B, in comparison to Meta’s Llama 2 and other major tech rivals.
US vs. International Models
In recent times, international open source language models have exhibited significant advancements, often surpassing their US counterparts in specialized tasks. The restricted nature of US companies and their diminished transparency have hindered their ability to keep up with the rapidly evolving landscape of AI. Conversely, an interesting dynamic has emerged, indicating that companies that release open source projects contribute to the growth of the AI ecosystem and consequently gain strength in building AI.
Meta’s Llama 2, a prominent open source AI model developed by a US company, seeks to challenge industry leaders such as OpenAI, Microsoft, and Google. To compete with these major players, Meta chose to release its AI language model under a license allowing commercial reuse, albeit with certain restrictions. Curiously, after the release of the Chinese model, Yi-34B, developers noted that 01.AI’s code had initially included references to Meta’s Llama 2, which were subsequently removed. Richard Lin, 01.AI’s head of open source, acknowledged the oversight and rectified it by reinstating the changes. Furthermore, 01.AI has acknowledged Llama 2 as the basis for part of Yi-34B’s architecture. Notably, both models are built upon the “transformer” architecture introduced by Google in 2017, with 01.AI deriving that component from Llama 2. Although a legal expert consulted by 01.AI has determined that Yi-34B is not subject to Llama 2’s license, Meta has not provided any comment on the matter.
Distinctiveness in Functionality
Despite the similarities in architecture, Yi-34B and Llama 2 exhibit stark differences in functionality due to the disparate training data they have been exposed to. Notably, Eric Hartford, an AI researcher at Abacus.AI, argues that Yi-34B’s training is distinct from Llama 2’s and significantly outperforms it. This distinction demonstrates the potential of open source models to diverge and develop unique characteristics, even when stemming from a shared foundation.
China’s Engagement with Generative AI
Although China’s AI expertise has been touted by industry experts, recent developments indicate that China is currently following America’s lead in the field of generative AI. While Chinese researchers have released numerous large language models, the industry as a whole lags behind the US. Western companies have gained a significant advantage by leveraging public releases to test models, gather user feedback, and generate interest. Chinese AI companies face stronger regulatory and economic headwinds, placing them at a slight disadvantage compared to their US counterparts. This situation raises questions about the full potential of China’s AI capabilities and its ability to compete on a global scale.
Speaking at the World Economic Forum, Kai-Fu Lee, a prominent AI expert, emphasized the significance of an open approach in fully harnessing the power of AI. Lee highlighted the inequality that arises when a few companies dominate the market, limiting access for individuals from different socioeconomic backgrounds, researchers, students, entrepreneurs, and hobbyists. Open source AI models provide a platform for individuals to learn and develop their potential as creators, inventors, and developers of future applications. This assertion underscores the importance of open source projects in fostering innovation and inclusivity within the tech industry.
As the AI landscape continues to evolve, open source language models play a pivotal role in driving innovation and facilitating the development of the next generation of AI applications. 01.AI’s Yi-34B, although grounded in Meta’s Llama 2, demonstrates the potential for unique advancements and distinct functionality. The global competition in AI development pits countries like the US and China against each other, with each facing its own set of challenges and advantages. Embracing open source models and maintaining transparency will be crucial for any country to fully capitalize on the power of AI and ensure a more inclusive, equitable future for the tech industry.
Leave a Reply