A new Meta/Facebook language model

© iStock.

Meta has announced LLaMA-13B, a new AI-powered large language model (LLM) that the company says can outperform OpenAI’s GPT-3 model despite being “10x smaller.” Smaller AI models could allow ChatGPT-like assistants to run locally on devices such as PCs and smartphones. LLaMA-13B is part of a new family of language models called “Large Language Model Meta AI”, or LLaMA for short. Meta has trained its LLaMA models using publicly available datasets such as Common Crawl, Wikipedia, and C4, making them open-source compatible. The company is designing these foundational models as the technological basis for more evolved AI models, similar to how OpenAI built ChatGPT from a GPT-3 foundation. A simplified version of LLaMA is available on GitHub.

LLaMA paves the way for ChatGPT-like performance on consumer hardware in the near future. The number of parameters in a language model is a key factor in its performance, with larger models generally able to handle more complex tasks and produce more logical results. However, a larger number of parameters takes up more space and requires more computational power. The size of the language models in the LLaMA collection ranges from 7 to 65 billion parameters. In comparison, OpenAI's GPT-3 model — ChatGPT’s base model — has 175 billion parameters. Meta has not yet announced plans for a larger version of the model.

⇨ Ars Technica, Benj Edwards, “Meta unveils a new large language model that can run on a single GPU.”

2023-02-24