Stanford Alpaca
A small but capable 7B model based on LLaMA that often behaves like OpenAI’s text-davinci-003
About Stanford Alpaca
Instruction-following models such as GPT-3.5 (text-davinci-003), ChatGPT, Claude, and Bing Chat have become increasingly powerful. Many users now interact with these models regularly and even use them for work. However, despite their widespread deployment, instruction-following models still have many deficiencies: they can generate false information, propagate social stereotypes, and produce toxic language.
To make maximum progress on addressing these pressing problems, it is important for the academic community to engage. Alpaca 7B is a model fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations. Alpaca behaves similarly to OpenAI’s text-davinci-003, while being surprisingly small and easy/cheap to reproduce (<$600).
