What can AI learn about the universe?

Artificial intelligence and machine learning have become ubiquitous, with applications ranging from data analytics, cybersecurity, pharmaceutical development, music composition and artistic renderings.

In recent years, large language models (LLMs) have also emerged, adding human interaction and writing to the long list of applications. This also applies to ChatGPT, an LLM that has had a major impact since its introduction less than two years ago. This application has sparked much discussion (and controversy) about the potential applications and implications of AI.

Astronomy has also benefited enormously, using machine learning to sift through vast amounts of data to look for signs of planetary transits, correct atmospheric interference and find patterns in the noise. According to an international team of astrophysicists, this may be just the beginning of what AI can do for astronomy.

In a recent study, the team refined a Generative Pre-trained Transformer (GPT) model using observations of astronomical objects. In doing so, they have successfully demonstrated that GPT models can effectively assist in scientific research.

The study was conducted by the International Center for Relativistic Astrophysics Network (ICRANet), an international consortium consisting of researchers from the International Center for Relativistic Astrophysics (ICRA), the National Institute for Astrophysics (INAF), the University of Science and Technology of China , the Chinese Academy of Sciences Institute of High Energy Physics (CAS-IHEP), the University of Padua, the Isfahan University of Technology and the University of Ferrera.

Their paper, “Can AI Understand Our Universe? Test of Fine-Tuning GPT by Astrophysical Data,” was recently published at arXiv preprint server.

As mentioned, astronomers rely heavily on machine learning algorithms to sift through the volumes of data acquired by modern telescopes and instruments. This practice started about a decade ago and has since grown by leaps and bounds to the point where AI has been integrated into the entire research process. As ICRA president and lead author of the study, Yu Wang, told Universe Today via email:

“Astronomy has always been data-driven, and astronomers are among the first scientists to adopt and use machine learning. Now machine learning is integrated into the entire astronomical research process, from the production and control of ground- and space-based telescopes (e.g. optimizing the performance of adaptive optics systems, improving the initiation of specific actions (triggers) of satellites under certain conditions, etc.), to data analysis (e.g. denoising, data imputation, classification, simulation, etc.) and the establishment and validation of theoretical models (e.g. testing modified gravity, limiting the equation of state of neutron stars, etc.).”

Data analytics remains the most common of these applications, as it is the easiest area into which machine learning can be integrated. Traditionally, dozens of researchers and hundreds of citizen scientists analyzed the volumes of data produced by an observation campaign.

However, this is not practical in an age when modern telescopes collect terabytes of data every day. This includes all-sky surveys such as the Very Large Array Sky Survey (VLASS) and the many phases conducted by the Sloan Digital Sky Survey (SDSS).

To date, LLMs have only been sporadically applied to astronomical research, as they are a relatively recent creation. But according to proponents like Wang, it has had a huge social impact and has a lower-bound potential equivalent to an “industrial revolution.”

As for the upper limit, Wang predicts that it could vary considerably and perhaps result in the “enlightenment or destruction” of humanity. However, unlike the Industrial Revolution, the pace of change and integration is much faster for AI, raising questions about how far its adoption will go.

To determine its potential for the field of astronomy, Wang said, he and his colleagues took a pre-trained GPT model and refined it to identify astronomical phenomena:

“OpenAI provides pre-trained models, and what we’ve done is fine-tune it, which means changing some parameters based on the original model, allowing it to recognize astronomical data and calculate results based on this data. This seems like a a bit on the fact that OpenAI gave us a bachelor’s student, whom we then trained to become a master’s student in astronomy.

“We provided limited data with modest resolution and trained the GPT less frequently compared to normal models. Nevertheless, the results are impressive and we achieve an accuracy of around 90%. This high level of accuracy is due to the robust foundation of the GPT , who already understands data processing and has logical deductions, as well as communication skills.”

To refine their model, the team introduced observations of various astronomical phenomena, derived from different catalogs. This included 2,000 samples of quasars, galaxies, stars and broad absorption line (BAL) quasars from the SDSS (500 each). They also integrated observations of short and long gamma-ray bursts (GRBs), galaxies, stars and black hole simulations. When tested, their model successfully classified various phenomena, distinguished between types of quasars, inferred distance based on redshift, and measured the rotation and inclination of black holes.

“If anything, this work shows that LLMs can process astronomical data,” says Wang. “In addition, the ability of a model to process different types of astronomical data is a capability that other specialized models do not possess. We hope that LLMs can integrate different types of data and then identify common underlying principles to help us understand the world. Of course This is a challenging task and not one that astronomers can accomplish alone.”

Of course, the team acknowledges that the dataset they experimented with was very small compared to the data output of modern observatories. This is especially true for next-generation facilities such as the Vera C. Rubin Observatory, which recently received its LSST camera, the largest digital camera in the world!

Once operational, Rubin will conduct the 10-year Legacy Survey of Space and Time (LSST), which is expected to produce 15 terabytes of data per night! To meet the demands of future campaigns, Wang says, improvements and cooperation between observatories and professional AI companies are needed.

Nevertheless, it is a foregone conclusion that there will be more LLM applications for astronomy in the near future. This is not only a likely development, but also necessary given the enormous amount of data that astronomical studies generate today. And since this is likely to increase exponentially in the near future, AI is likely to become indispensable in the field.