Today's news is frequently dominated by coverage of environmental crises such as extreme weather events, changing climates, diminishing crop yields, rising sea levels, deforestation and forest fires. Scientists all over the world are constantly trying to develop tools and solutions that can help address these daunting environmental challenges and mitigate their impact on our planet. An organisation that is fully committed to exploring and developing innovative solutions to make our planet more sustainable in the future is the European Space Agency, and more specifically the Phi-lab Innovation Lab. One of the main goals of researchers in the Phi-lab is to explore the immense potential of cutting edge technologies such as artificial intelligence in analysing Earth Observation data, in order to understand the deterioration of the earth’s systems and provide tools that can help improve the future of our life on Earth.
Earth Observation and AI
But what is the link between Earth Observation and Artificial Intelligence and how can they be used to address the Earth’s environmental challenges? Let us start with definitions:
● Earth observation data is the information about our Earth’s physical, chemical and biological systems obtained from satellites carrying remote sensing devices such as high resolution cameras and atmospheric sensors. ESA has been investing in Earth Observation from space since the 1970s with its launch of the first EO satellite Meteosat in 1977. Since then ESA has developed a new family of satellites which have been designed to provide detailed information about weather conditions, high resolution images of land and oceans, and radars to monitor sea levels and atmospheric conditions. These satellites collect vast amounts of data that can be used for weather forecasting, climate change monitoring, land & coastal monitoring, deforestation & wildfire detection, agriculture & food security applications, disaster monitoring and air-pollution tracking. Earth observation data is therefore crucial to understanding our planet, advancing scientific research and providing us with the tools to support a more sustainable and greener future.
● AI and more specifically Machine Learning (ML) is the practice of using algorithms to create “smart” models that are capable of predicting outcomes and classifying information. Much like humans these models learn through repetition and mistakes. As an ML model is fed data, it is asked to predict information. The training algorithm will then inform the model whether its prediction is correct or incorrect. This process is repeated until the model “learns” to pick out relevant information within the data that will allow it to correctly predict outcomes and classify information. AI technologies have become pervasive in everyday life, from voice commands and face recognition in smartphones, to medical imaging applications and in chatbots like ChatGPT.
One major challenge of working with Earth Observation data is simply the vast amounts of data and information that is constantly being generated by the multitude of EO satellites in orbit, with a single satellite capable of generating 1.6TB of data daily. There is no way in which human beings can sift through all of this data productively and in short periods of time. Researchers at the Phi-lab are therefore trying to harness the potential of Artificial intelligence to help analyse the large amounts of data that is captured by the remote sensing devices of the EO satellites and help develop AI powered tools. These tools can extract crucial information from EO data quickly, efficiently and automatically (without the need of human experts to interpret the data). They can provide up to date information about key earth systems, allowing policy makers to more efficiently tackle pressing issues such as food security, climate change and rising sea levels. Moreover, these tools can also help provide invaluable information to first responders during extreme weather events, floods and forest fires helping save lives.
My time at ESA
During my time at ESA as a National Trainee (Graduate Program funded by MCST) I was located at the European Space Research Institute (ESRIN), Italy. ESRIN is the primary source for the acquisition, distribution, and use of data from EO satellites. While there I was part of the Phi-lab Innovation Lab. My background in Machine Learning allowed me to contribute to the Lab’s ongoing A.I. research, focusing on Geo-spatial Foundation Models.
Geo-spatial Foundation Models
Having labelled data is a key component in applying A.I. algorithms successfully to any domain. Data labelling is the process of manually adding one or more meaningful and informative labels to the data, providing context so that a machine learning model can learn from it. For example, labels might indicate whether a photo contains a bird or car. However, for most real-world applications, labelling large datasets is laborious, expensive, and time-consuming. This holds especially true for EO, as many satellites orbit the Earth, and produce a lot of data. Labelling them would require a massive amount of human interactions and frequent updates, making it necessary to train ML models on small EO datasets. However, this does not mean that we should discard the massive amounts of unlabelled EO data available to us. Instead we can use a technique called Self-Supervised Learning (SSL) to create a Foundation model that is pre-trained on unlabelled satellite data, using the data itself to generate intrinsic labels, rather than relying on external labels provided by humans. This allows the model to learn “foundational” and generic information about the data. The model is then trained on smaller labelled datasets to solve specific tasks like detecting forest fires. Models that are pre-trained on unlabelled data before being applied to specific tasks tend to have better performance than models trained from scratch on just the specific task.
Like most things in life this concept is best explained in terms of food! Let's say that I challenge two people to bake a tasty croissant (the specific task). Both people have never baked a croissant before. However, one of the contestants has some experience in the kitchen and has some foundational knowledge about how to use basic kitchen equipment (pre-trained model). While the other contestant has never set foot in a kitchen before and doesn’t even know how to switch on an oven (from scratch model). Who do you think will manage to deliver a tasty croissant first, the contestant that just needs to learn how to bake a croissant? Or the contestant that needs to learn how things in a kitchen work before even attempting to bake a croissant? The answer is obvious!
The nice thing about EO satellite data is that geo-spatial information about the data such as coordinate locations and data capture times is readily available. One of my main contributions during my time at the Phi-lab was exploring the use of geo-spatial information to create novel SSL techniques unique to EO data.
I’d like to thank MCST and ESA for giving me the opportunity to work in an interdisciplinary, innovative and European organisation. The skills leant, experience gained and connections made are priceless. With Malta poised to become an associate member state of ESA in the next couple of years, it is an exciting time for the burgeoning Maltese space industry. I’m looking forward to seeing it take off!
Luke Camilleri was a Maltese National Trainee for a year at the European Space Agency