With LIMA, Meta’s AI researchers introduce a new language model that achieves GPT-4 and Bard level performance in test scenarios, albeit fine-tuned with relatively few examples.

LIMA stands for “Less is More for Alignment,” and the name hints at the model’s function: It is intended to show that with an extensively pre-trained AI model, a few examples are sufficient to achieve high-quality results.

Few examples in this case means that Meta manually selected 1,000 diverse prompts and their output from sources such as other research papers, WikiHow, StackExchange, and Reddit.

The team then used these examples to refine its own 65-billion-parameter LLaMA model, the leaked language model that sparked the open-source language model movement. Meta avoided the expensive RLHF, which OpenAI uses to tune its models and sees as an important part of the future of AI.

Source: https://the-decoder.com/metas-new-lima-language-model-reaches-gpt-4-level/