Meta: Our LLaMA-13B outperforms OpenAI’s GPT-3 despite being 10x smaller – NewsEverything Technology

Last Friday, Meta announced its new AI-based large language model (LLM) called LLaMA-13B. The company says it can outperform GPT-3 model from its rival OpenAI “on most benchmarks”.

A Facebook logo on the wall. Meta is the parent company of Facebook. Image credit: Anthony Quintano via Flickr, CC BY 2.0

GPT-3 is the basis of the famous ChatGPT artificial intelligence chatbot. If these claims are true, a reduced-size model could work in stand-alone environments, such as individual laptops or even smartphones.

The LLaMA comes in several varieties differing in size. The smallest language model from this family contains 7 billion parameters, while the most complex variant contains 65 billion parameters.

For comparison, OpenAI’s GPT-3 (the one that is used in ChatGPT) is built using 175 billion parameters.

Meta also announced on Twitter that its LLaMA models were trained using publicly available datasets, including Common Crawl, Wikipedia, and C4. Therefore, they also released the model weights for all of LLaMA-13B versions as open source.

Today we release LLaMA, 4 foundation models ranging from 7B to 65B parameters.
LLaMA-13B outperforms OPT and GPT-3 175B on most benchmarks. LLaMA-65B is competitive with Chinchilla 70B and PaLM 540B.
The weights for all models are open and available at https://t.co/q51f2oPZlE
1/n pic.twitter.com/DPyJFBfWEq

— Guillaume Lample (@GuillaumeLample) February 24, 2023

“Unlike Chinchilla, PaLM, or GPT-3, we only use datasets publicly available, making our work compatible with open-sourcing and reproducible, while most existing models rely on data which is either not publicly available or undocumented,” said Guillaume Lample, a member of the LLaMA-13B project at Meta.

Some industry experts already reacted to this news by saying that AI language models could be run on mobile phones and laptops, giving them a large chunk of capabilities native to a substantially larger ChatGPT.