Captions, the innovative AI video startup founded by Gaurav Misra, former head of design engineering at Snap and a former software development engineer (II) at Microsoft, continues to make strides in the industry. With a recent $25 million funding round from top VCs and the introduction of their new dedicated AI dubbing app called Lipdub, Captions is proving to be a force to be reckoned with.
Lipdub is an iOS app that utilizes artificial intelligence to automatically translate and dub any prerecorded video with spoken audio into 28 different languages. One of the standout features of Lipdub is its ability to match the speaker’s lip movements with the translated language, creating a seamless and natural viewing experience.
Unlike some other AI dubbing and audio translation services, Lipdub goes beyond the basics. It offers the option to translate using specific dialects and even vocabulary, including options like Gen Z and Texas slang. This level of customization sets Lipdub apart from its competitors.
“AI Dubbing’s success inspired us to push this technology to the next level and introduce synced lip movement to the mix,” said Gaurav Misra, founder of Captions.
With over 100,000 daily users and more than 5 million creators who have tried their products, Captions has established itself as a leader in the industry. Their iOS app offers a range of features, including an AI Trim that automatically removes filler words and an AI Enhance Speech that removes background noise. Additionally, creators can upload and compress video on their website and utilize the AI Eye Contact feature to correct videos where the speaker wasn’t looking at the camera. Captions also automatically adds AI-generated subtitles and captions to videos, as the name suggests.
Notable users of the Captions app include ESPN commentator Omar Raja, “Mr. Wonderful” from Shark Tank, Justin Kan (founder of Twitch), and the influencer Unnecessary Inventions. Captions’ popularity and success within the industry are undeniable.
“We’ve been focused on making this technology widely available, leading us to create and launch Lipdub as its own separate app,” said Gaurav Misra.
Rival company ElevenLabs, founded by former Google and Palantir employees, also recently announced their own prerecorded video AI Dubbing feature. However, Lipdub’s extensive language options, lip syncing capabilities, and user-friendly interface give Captions a competitive edge.
While there are other startups in the AI dubbing and video/audio editing space, such as MURF.AI, Play.ht, and WellSaid Labs, they do not offer the same breadth of video editing tools as Captions. Meanwhile, Meta Platforms has launched SeamlessM4T, an open-source multilingual foundational model that can understand nearly 100 languages and generate real-time translations.
Captions’ dedication to innovation and user experience have solidified their position as a leading AI video startup, and the introduction of Lipdub further showcases their commitment to revolutionizing the industry.
