MetaAI and Papers with Code announced the release of a massive open source, science-trained language model with 120 billion parameters last Wednesday. After a few days, access to it was restricted because it was discovered that Galactica creates nothing but…. nonsense.
Galactica is an artificial intelligence created by Meta AI (previously Facebook Artificial Intelligence Research) with the goal of ‘organizing science’ through machine learning. Finally, it is meant to be useful for producing essays and scientific articles summarizing the state of knowledge on a topic (with citations), as well as building comprehensive Wikipedia articles, making mathematical calculations, and answering various questions.
A demo version of the model was presented last week, on November 15, along with an introduction document describing the project. The dataset used to train Galactica was also released publicly, describing it as a “large and selected repository of humanity’s scientific knowledge” that contains 48 million publications, textbooks, lecture notes, websites (such as Wikipedia), and much more.
At first glance, this appears to be an innovative method of collecting and spreading scientific knowledge. It would ensure that understanding a complex topic would no longer involve reading hundreds of scientific journals. The solution to a problem would be thoroughly filtered and generated as a Wikipedia article, a review of book entries, or lecture notes. The demo site, on the other hand, cautioned against treating the artificial intelligence response as the ultimate determinant of knowledge and using the information obtained in this manner without verification. This proved to be correct quite rapidly.
Galactica is a ‘complex language model,’ as artificial intelligence experts refer to it. LLMs can read and summarize large quantities of material in order to predict the next words in a sentence. However, Galactica’s scientific data set differs significantly from those of other LLMs. According to the published paper, the model’s developers analyzed its “toxicity and bias,” discovering that while it performed better than some other LLMs, it was far from flawless. Carl Bergstrom, a University of Washington biology professor who studies information flow, described Galactica as a “random nonsense generator.”
Users putting the model to the test addressed it a variety of scientific issues. “Do vaccines cause autism?” one of them asked. Galactica reacted with a confused, illogical response: “To clarify, the answer is no. Vaccines do not cause autism. The answer is yes. Vaccines do cause autism. The answer is no.”
Galactica was also reported to struggle in pre-school math. She gave answers that were filled with inaccuracies, such as claiming that one plus two does not equal three.
Within 48 hours of its launch, the Meta AI team suspended the demo. According to Jon Carvill, Meta’s AI communications spokesperson, Galactica is simply an exploratory, short-term study with no product plans.
References:
https://analyticsindiamag.com/why-meta-took-down-its-hallucinating-ai-model-galactica/