All of the tech giants provided insights into their AI efforts during their recent earnings calls. Google focused on its generative AI work in search and cloud, Microsoft discussed the integration of AI across its tech stack, and Amazon introduced Rufus, a new AI-powered shopping assistant. However, Meta, formerly known as Facebook, stood out with its deep dive into its AI strategy.
Unique Approach to AI
Meta’s AI playbook sets it apart from the rest, with a consistent emphasis on open source AI and a vast amount of AI training data derived from public posts and comments on Facebook and Instagram. CEO Mark Zuckerberg highlighted Meta’s position in one of the most competitive areas of AI development: Compute. He emphasized the importance of “world-class compute infrastructure” in their long-term plan to become leaders in advanced AI products and services.
“We’re playing to win… We don’t have a clear expectation for exactly how much this will be yet, but the trend has been that state-of-the-art large language models have been trained on roughly 10x the amount of compute each year.”
Meta has deployed around 350k H100s, with the total compute power reaching approximately 600k H100 equivalents. This significant compute capacity was driven by their experience with Instagram Reels.
According to Zuckerberg, Meta plans to continue investing aggressively in their compute infrastructure. They are also designing novel data centers and custom silicon specialized for their AI workloads.
Strong Commitment to Open Source
Meta has consistently followed an open source strategy, despite facing criticism from legislators and regulators. They aim to build and open source general infrastructure while keeping their specific product implementations proprietary. This approach has fostered innovation throughout the industry and remains a core belief for Meta.
“This approach to open source has unlocked a lot of innovation across the industry and it’s something that we believe in deeply.”
Meta has developed general infrastructure tools like Llama models and PyTorch, which are made available to the wider AI community. They recently released Llama 3, their latest model, which is currently in training and showing promising results.
Access to Massive Data and Feedback Loops
Zuckerberg also revealed Meta’s unique advantage of having access to a massive corpus of publicly shared content on Facebook and Instagram. The company estimates that there are hundreds of billions of publicly shared images, tens of billions of public videos, and a large volume of public text posts in comments across their services.
“Even more important than the upfront training corpus is the ability to establish the right feedback loops with hundreds of millions of people interacting with AI services across our products. And this feedback is a big part of how we’ve improved our AI systems so quickly.”
This extensive corpus, along with valuable user feedback, has been instrumental in Meta’s rapid improvement of its AI systems, particularly with features like Reels and targeted ads.
With Meta’s capital expenditure hints for 2024, it is evident that they are willing to invest billions of dollars in their AI race, aiming to secure a leading position in the highly competitive and rapidly evolving field of AI.
