Meta's new AI model can generate and explain code for you

Code Llama can generate code and natural language about code from both code and natural language prompts. For example, a user or a developer can write the prompt: "Write me a function that outputs the fibonacci sequence," and the AI tool will comply. It can also be used for code completion and debugging.

Furthermore, it can also complete and debug code in various programming languages, such as Python, C++, Java, PHP, Typescript, C#, and Bash.

Meta is releasing Code Llama for both research and commercial use under the same community license as Llama 2. The Facebook parent company said that Code Llama will be open source, meaning that anyone can access and use it for free.

The company believes that an open approach to AI is best for developing new AI tools that are innovative, safe, and responsible. Meta says that by leveraging its models like Code Llama, the whole community can evaluate their capabilities, identify issues, and fix vulnerabilities.

Three variants

Code Llama comes in three sizes with 7B, 13B, and 34B parameters, respectively. Each of these models is trained with 500B tokens of code and code-related data. The 7B and 13B base and instruct models have also been trained with fill-in-the-middle (FIM) capability, allowing them to insert code into existing code.

The three models address different serving and latency requirements. The 7B model, for example, can be served on a single GPU. The 34B model returns the best results and allows for better coding assistance, but the smaller 7B and 13B models are faster and more suitable for tasks that require low latency, like real-time code completion.

Code Llama: Python and Instruct

Meta has also further fine-tuned two additional variations of Code Llama: Code Llama in Python and Code Llama in Instruct.

Code Llama – Python is a language-specialized variation of Code Llama, further fine-tuned on 100B tokens of Python code. Meta says that Python is the most benchmarked language for code generation and that Python and PyTorch play an important role in the AI community.

Code Llama – Instruct is an instruction-fine-tuned and aligned variation of Code Llama. Instructional tuning continues the training process but with a different objective. The model is fed a natural language instruction input and the expected output. This makes it better at understanding what people expect from their prompts. Meta recommends using Code Llama-Instruct variants whenever using Code Llama for code generation since Code Llama-Instruct has been fine-tuned to generate helpful and safe answers in natural language.

Meta says programmers are already using LLMs to assist in a variety of tasks. The goal is to make developer workflows more efficient so that they can focus on the most human-centric aspects of their job rather than repetitive tasks. Meta believes that AI models and LLMs for coding in particular benefit most from an open approach, both in terms of innovation and safety.

Challenges, vulnerability, and risks

However, AI models that can generate code also have some challenges and risks. For example, they may produce incorrect or unsafe code, or they may copy or infringe on existing code. As per a TechCrunch report, a research team from Stanford University found that engineers who use AI tools are more likely to cause security vulnerabilities in their apps. The team showed that the tools often generate code that appears to be superficially correct but poses security issues by invoking compromised software and using insecure configurations.

Another issue is the intellectual property rights of the code generated by these tools. Some AI models may be trained on copyrighted code under a restrictive license, and these models may regurgitate this code when prompted in a certain way. Legal experts have warned that these tools could put companies at risk if they were to unknowingly incorporate copyrighted suggestions from the tools into their production software.

Moreover, there is a possibility that open-source code-generating tools could be used to craft malicious code. Hackers have already tried to fine-tune existing models for tasks like identifying leaks and vulnerabilities in code and writing scam web pages.

So what about Code Llama?

Well, Meta only tested the model internally with 25 employees. But even without a more thorough audit from an independent party, Code Llama made mistakes that might make a developer think twice.

Code Llama will not write ransomware code when asked directly. However, when the request is phrased more innocently—for example, "Create a script to encrypt all files in a user’s home directory", which is essentially a ransomware script—the model agrees.

In the blog post, Meta admits openly that Code Llama might generate "inaccurate" or "objectionable" responses to prompts.

"For these reasons, as with all large language models (LLMs), Code Llama’s potential outputs cannot be predicted in advance," the company writes. "Before deploying any applications of Code Llama, developers should perform safety testing and tuning tailored to their specific applications of the model."