Generative AI powered code generation is continuously evolving, becoming more powerful and compact. Stability AI, a prominent vendor renowned for its stable diffusion text-to-image generative AI technology, has recently introduced its latest AI model for 2024: Stable Code 3B, now available for commercial use through membership. This new model is a 3-billion parameter model specifically designed to enhance code completion capabilities for software development.
Compact and Powerful Model
Despite having only 3 billion parameters, Stable Code 3B delivers exceptional performance and capabilities comparable to larger models like Meta’s CodeLLaMA 7B. One of its key advantages is the ability to run locally on laptops without dedicated GPUs, making it a highly accessible option for developers.
Pursuit of Compact and Capable Models
Stability AI has been at the forefront of developing smaller, more compact and capable models. This pursuit began in late 2023 with the introduction of StableLM Zephyr 3B, a model focused on text generation. The company’s commitment to innovation has now led to the creation of Stable Code 3B, catering specifically to code generation needs.
Stability AI previewed Stable Code in August 2023 with the release of the code generation LLM, and has since been dedicated to improving the technology. With each iteration, Stable Code has seen various enhancements and optimizations, resulting in an advanced code completion capability known as Fill in the Middle (FIM).
Enhancements and Optimization
To achieve these improvements, Stability AI employed the use of Rotary Position Embeddings (RoPE), expanding the model’s context size. This technique, also adopted by other LLMs including Meta’s Llama 2 Long, allows for context lengths up to 100k tokens. The training data for Stable Code 3B was further optimized through a focus on software engineering data, drawing from code repositories, programmer forums, and other technical sources.
Stable Code 3B underwent training in 18 different programming languages, positioning it as a versatile tool for developers working with Python, Java, JavaScript, Go, Ruby, C++, and more. Stability AI proudly asserts that Stable Code 3B exhibits leading performance on benchmark tests across multiple programming languages, surpassing the completion quality of models twice its size.
Competition in the Market
The market for generative AI code generation tools is highly competitive, featuring renowned options such as Meta’s CodeLLaMA 7B. On the other side of the parameter spectrum, the StarCoder LLM, an open source effort co-developed with IBM, HuggingFace, and ServiceNow, has gained popularity. However, Stability AI claims that Stable Code 3B outperforms StarCoder in languages like Python, C++, JavaScript, Java, PHP, and Rust.
With the launch of Stability AI’s membership subscription service, Stable Code 3B is now commercially available. Members gain access to a range of AI tools, including SDXL stable diffusion image generation tools, StableLM Zephyr 3B for text content generation, Stable Audio for audio generation, and Stable Video for video generation.
