Meta, formerly Facebook, has been developing a powerful language model behind closed doors and it has now been leaked to the public. It was first posted to 4chan and has spread to all corners of the internet. This specific model was previously only given to government organisations, approved researchers and trusted civil servants.
This large language model, which was previously exclusive, is now available for anyone to download online after being leaked on 4chan, now just about anyone can use the torrent file to download LLaMa; an abbreviation for (Large Language Model Meta AI). This is the first time that the organisation’s properietary AI model has been ever leaked to the public.
To date, the largest tech firms including Google, Microsoft, and OpenAI have kept the door shut on their language models, only allowing users to access it through a filtered API and not giving them a look at their code. The implications of this LLaMa leak are not yet known but it’s sure to cause chaos since the AI is not filtered.
Meta didn’t bother denying their mistake of sharing the model between researchers as it said this in a statement to Motherboard:
“It’s Meta’s goal to share state-of-the-art AI models with members of the research community to help us evaluate and improve those models. LLaMA was shared for research purposes, consistent with how we have shared previous large language models. While the model is not accessible to all, and some have tried to circumvent the approval process, we believe the current release strategy allows us to balance responsibility and openness,” the email read.
Like other AI models including OpenAI’s GPT-3, LLaMa is built on a massive collection of pieces of words, or “tokens.” From here, LLaMa can then take an input of words, and predict the next word to recursively generate more text, Meta explains in a blog post from February. LLaMa has multiple versions of different sizes, with LLaMa 65B and LLaMa 33B being trained on 1.4 trillion tokens. According to the LLaMA model card, the model was trained on datasets scraped from Wikipedia, books, academic papers from ArXiv, GitHub, Stack Exchange, and other sites.
In that same February blog post, Meta says that it is releasing LLaMa under a noncommercial license focused on research use cases to “maintain integrity and prevent misuse.”
“Access to the model will be granted on a case-by-case basis to academic researchers; those affiliated with organizations in government, civil society, and academia; and industry research laboratories around the world,” the post says. These security concerns are now null and void as anyone can access the language model.
“We believe that the entire AI community—academic researchers, civil society, policymakers, and industry—must work together to develop clear guidelines around responsible AI in general and responsible large language models in particular. We look forward to seeing what the community can learn—and eventually build—using LLaMA,” The blog post adds.
The CEO of Hugging Face, Clement Delangue, posted a staff update to GitHub regarding a user’s LLaMa repository. He had this to say about his employees breach:
“Company Meta Platforms, Inc has requested a takedown of this published model characterizing it as an unauthorized distribution of Meta Properties that constitutes a copyright infringement or improper/unauthorized use,” the notice said.
Delangue cautioned users against uploading LLaMa weights to the internet. The flagged user’s GitHub repository is currently offline. He has warned the public that uploading LLaMa weights to the internet is a bad idea as it breaches copyright. The flagged user’s GitHub repository has been taken offline.
