Meta unveils ChatGPT rival AI LLaMA, but no guarantee on hallucination

Unlike Google and OpenAI, Meta will share its LLaMA language model with AI researchers, claims the social media giant.
Baba Tamim
Illustration: Meta unveils ChatGPT rival AI language model.
Illustration: Meta unveils ChatGPT rival AI language model.

Artapixel/Pixabay | Uladzimir Zuyeu/iStock 

The Fundamental AI Research (FAIR) team at Meta, Facebook's parent company, has introduced a new "state-of-the-art" artificial intelligence (AI) language model called Large Language Model Meta AI (LLaMA).

The model will be made accessible to researchers, and is anticipated to aid scientists and engineers as they investigate new uses of AI, CEO Mark Zuckerberg announced on Friday.

"We're releasing a new state-of-the-art AI large language model called LLaMA designed to help researchers advance their work," Zuckerberg wrote in a Facebook post. 

"LLMs have shown a lot of promise in generating text, having conversations, summarizing written material, and more complicated tasks like solving math theorems or predicting protein structures." 

AI advancements have become a focus for both large tech companies and startups, with large language models such as Microsoft's Bing AI, OpenAI's ChatGPT, and Google's unreleased Bard AI helping to underpin applications.

However, there are a number of ways that Meta's LLM differs from these models, notably its size and openness to researchers, claims the social media giant. 

LLaMA will have 7-65 billion parameters

According to Meta, the sizes of LLaMA will range from 7 billion parameters to 65 billion parameters. 

Despite the fact that larger models have been successful in extending the technology's capabilities, they can be more expensive to operate, a stage known as "inference." The Chat-GPT 3 from OpenAI, for instance, includes 175 billion parameters.

"Smaller models trained on more tokens — which are pieces of words — are easier to retrain and fine-tune for specific potential product use cases," Meta AI wrote in a blog post on Friday. 

"We trained LLaMA 65B and LLaMA 33B on 1.4 trillion tokens. Our smallest model, LLaMA 7B, is trained on one trillion tokens."

Contrary to Google's LaMDA and OpenAI's ChatGPT, whose underlying models are private, Meta has also declared that their LLM will be made available to the AI research community.

"Unlike Chinchilla, PaLM, or GPT-3, we only use datasets publicly available, making our work compatible with open-sourcing and reproducible, while most existing models rely on data which is either not publicly available or undocumented," Guillaume Lample, a research scientist at Facebook AI tweeted on Friday.

AI hallucinations - no guarantee

Like previous large language models, LLaMA generates text recursively by using a sequence of words as input and predicting the following term.

Meta claims to have used literature from the 20 languages with the greatest number of speakers to train the model, concentrating on the Latin and Cyrillic alphabet-based languages.

However, Meta hasn't given any assurances that its language model won't experience hallucinations like others.

"There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges," Meta said in the blog.

Add Interesting Engineering to your Google News feed.
Add Interesting Engineering to your Google News feed.
message circleSHOW COMMENT (1)chevron
Job Board