With little fanfare, researchers from Apple and Columbia University released an open source multimodal LLM, called Ferret, in October 2023. At the time, the release — which included the code and weights, but for research use only, not a commercial license — did not receive much attention. But now that may be changing: With open source models from Mistral making recent headlines and Google’s Gemini model is coming to the Pixel Pro and eventually to Android, there has been increased chatter about the potential for local LLMs to power small devices.
This chatter has only grown louder with Apple’s recent announcement of a key breakthrough in deploying LLMs on iPhones. The company released two new research papers introducing new techniques for 3D avatars and efficient language model inference. These advancements are being hailed as a game-changer, potentially enabling more immersive visual experiences and allowing complex AI systems to run on consumer devices such as the iPhone and iPad.
The AI community, which initially overlooked the Ferret release, is now celebrating Apple’s unexpected entry into the open source LLM landscape. Apple has traditionally been known as a “walled garden,” so their participation in open source AI is seen as a significant step forward. Bart de Witte, who runs a European non-profit focused on open source AI in medicine, expressed his excitement: “Ferret’s introduction is a testament to Apple’s commitment to impactful AI research, solidifying its place as a leader in the multimodal AI space.” Tristan Behrens, a German AI music artist and advisor, also chimed in, saying, “Apple (yes Apple!) recently released a Multimodal Large Language Model? Including code and weights?”
“What is the AI development in 2023 that you expected the least? For me, it was Apple releasing open source LLMs (albeit with a non-commercial license).” – Ben Dickson
The announcement surprised many, as Apple has long been known for its closed systems and secretive approach to development. However, in retrospect, it seems like a strategic move for Apple to compete in the LLM market and avoid dependence on cloud providers like Microsoft or Google. Apple’s infrastructure may not be built to serve LLMs at scale, but by releasing open source models, they can enter the market and challenge models like ChatGPT.
