PhiSat-1Nanosatellite Mission

PhiSat-1 Nanosatellite Mission

PhiSat-1 (Φ-Sat-1) is the first European satellite to demonstrate how onboard artificial intelligence can improve the efficiency of sending Earth observation data back to Earth. This revolutionary artificial intelligence technology will fly on one of the two CubeSats that make up the FSSCat (Federated Satellite System) mission – a Copernicus Masters winning idea. ¹⁾

As the overall 2017 Copernicus Masters winner, FSSCat, was proposed by Spain’s UPC (Universitat Politècnica de Catalunya) and developed by a consortium of European companies and institutes.

FSSCat is an innovative mission concept consisting of two federated 6U CubeSats in support of the Copernicus Land and Marine Environment services. They carry a dual microwave payload (a GNSS-Reflectometer and a L-band radiometer with interference detection/mitigation), and a multispectral optical payload to measure soil moisture, ice extent, and ice thickness, and to detect melting ponds over ice. It also includes a radio/optical inter-satellite link and an Iridium intersatellite link to test some of the techniques and technologies for upcoming satellite federations. FSSCat will be the precursor of a constellation of federated small satellites for Earth observation achieving high temporal resolution and moderate spatial resolution in a cost-effective manner. ²⁾

«The Federated Satellite System 6U tandem mission for sea ice and soil moisture monitoring captured the interest of the challenge experts immediately. Not only because the mission concept shows a high degree of well thought through technical novelties, but also because it will provide data that is complementary to the Sentinel fleet. This is especially true for the soil moisture monitoring component, which is not part of the current Sentinel portfolio. The FSSCat mission development is good to go and due to its disruptive approach, we are confident that it will be seen as a breakthrough in procuring future small missions at ESA.»

The two CubeSats will collect data, which will be made available through the Copernicus Land and Marine Environment services, using state-of-the-art dual microwave and hyperspectral optical instruments. They also carry a set of intersatellite communication technology experiments.

During Φ-week (9-13 September 2019 at ESA/ESRIN), ESA’s Director of Earth Observation Programs, Josef Aschbacher, said, “We see that there is huge interest in Φ-Sat and thanks to our partners, it is ready to be launched. We live in exciting times, the pace at which digital technology is developing coupled with the wealth of satellite information being delivered and, indeed, the growing demand for such data, means there are many opportunities to make a step change for the future of Earth observation. And, with Φ-Sat – Europe’s first artificial intelligence in space – we are going to do just this.”

The hyperspectral camera on one of the CubeSats will collect an enormous number of images of Earth, some of which will not be suitable for use because of cloud cover. To avoid downlinking these less than perfect images back to Earth, the Φ-Sat artificial intelligence chip will filter them out so that only usable data are returned.

Marco Esposito, from cosine Remote Sensing, the company that led the development of the artificial intelligence algorithm, explained, “While compact, the instrument – which covers the visible and near infrared with hyperspectral capability, enhanced with bands in the thermal infrared – is very powerful and will acquire terabytes of data that can be used to monitor vegetation changes and to assess water quality, for example.

“However, generating this amount of data actually poses a problem, as the data have to be handled efficiently so that they can reach the users in a timely manner. With Φ-Sat we have effectively given the instrument its own brain, which processes the data onboard to detect clouds in the images. This not only ensures better quality data, but makes the delivery much more efficient.”

ESA’s Massimiliano Pastena, noted, “Indeed, this will be the first satellite to demonstrate the use of artificial intelligence in orbit and we are very much looking forward to it being launched in the coming months.”

Φ-Sat-1 is the first scientific and technology initiative of the Earth Observation directorate of the European Space Agency (ESA) in the NewSpace field. ³⁾ Φ-Sat-1 is the result of an enhancement of the FSSCAT mission using AI (Artificial Intelligence) and improved imaging capabilities of the hyperspectral camera HyperScout-1 VNIR imager. cosine Remote Sensing is leading the development and execution of the demonstration, including the highly integrated spectral imaging in the VNIR and TIR, as well as the integration of state-of-art AI accelerators including the first AI algorithms for cloud screening. The HyperScout-2 spectral camera is the successor of HyperScout-1, a miniaturized reflective optical instrument equipped with enhanced processing capabilities, in orbit since February 2018 as part ESA GOMX-4B technology demonstration mission.

Φ-Sat-1 will demonstrate the capabilities of small instruments for scientific applications, and will validate in-orbit innovative technologies. ⁴⁾ The technology demonstration includes the miniaturization and highly integration of visible, near infrared and thermal spectral channels, as well as state-of-art processors and machine learning algorithms. The hardware co-registration of the VNIR and TIR channels will enable a variety of applications for terrain classification and change detection. Φ-Sat-1 will demonstrate an AI-based inference engine for cloud detection and will be launched as enhancement of the FSSCat mission, ⁵⁾ during the VEGA SSMS (Small Satellite Mission Service) Proof Of Concept flight.

Sensor Complement

HyperScout-2

The HyperScout product series consists of very compact spectral imagers, in the order of 1-2 kg. The instruments are rather inexpensive, which are not meant to replace or compete with larger institutional satellites like Sentinel or Landsat, but more to complement large observation spacecrafts, being exploited for early detection of anomalies, as well as other applications. HyperScout is the one of the first hyperspectral imagers with such a compact envelope, therefore enabling applications which rely on the potential of very high temporal resolution (i.e. order of hours) in order to detect anomalies as early as possible and to refer to other space or ground assets for more detailed investigations.

A spectral imager with such a small envelope becomes an attractive solution for deployment in constellations, and therefore offers to the user the possibility of achieving high revisit times over regions of interest. The HyperScout concept is a combination of state of the art optics packed in an extremely compact format, in combination with very powerful processing abilities. It was first designed to be installed on CubeSat and small satellites, however it can be easily incorporated into larger platforms given the small engineering budgets and the compact volume.

HyperScout-2 is based on the building blocks of HyperScout-1 that was successfully demonstrated in flight by cosine Remote Sensing and ESA onboard the ESA GOMX-4B satellite and it is operational since February 2018. ⁶⁾ The HyperScout-2 specifications are reported in Table 1. The optical front-end is based on a monolithically and reflective TMA configuration, based on freeform optical design and manufactured using diamond turning machining technique. The dual channel implementation envisages a beam splitter at the end of the front end optics, diverting the shorter wavelengths towards the VNIR sensor, and allowing the longer wavelengths to impinge on the TIR sensor.

This configuration was envisaged during other ESA activities related to navigation and planetary science, with goal of designing a multispectral camera for relative navigation. ⁷⁾ A very compact relay has been conceived in order to enhance the numerical aperture of the TIR channel. The VNIR Focal Plane Array (FPA) is a CMOS sensor with a linearly variable hyperspectral filtering element that separates the different wavelengths from 400 to 1000 nm, while the TIR FPA is based on a microbolometer and a set of four spectral filters from 8 to 14 µm. Both FPAs are 2D sensors operated in push broom mode. The TIR sensor is actively thermally controlled via a thermoelectric cooler (TEC) to limit the dependency of the sensor readout on the sensor temperature fluctuations.

HyperScout-2 has the following subsystems:

• Telescope assembly;

• VNIR Focal Plane Array (VNIR FPA);

• TIR Focal Plane Array (TIR FPA);

• Back-End Electronics unit (BEE);

• On-Board Data Handling (OBDH);

• Mass Memory Units (MMUs): operated in hot and cold spare configuration;

• Instrument Control Unit (ICU): main point of contact from the S/C bus for data interface and main instrument control;

• Eyes of Things board (EOT): data processing board for on-board artificial intelligence algorithms.

The architectural diagram of HyperScout-2 is depicted in Figure 2, while the subsystems locations in the payload is indicated in Figure 3. The back-end electronics (BEE) is a compact and versatile FPGA based single board image acquisition system. The On-Board Data Handling (OBDH) is physically installed on the BEE. The Instrument Control Unit (ICU) controls and monitors the operational status of the payload subsystems. The Vision Processing Unit (VPU) consists of a Myriad 2 processor located on the Eyes of Things (EoT) board. The ICU can independently power the BEE, OBDH, the EoT board and each MMU. The FPAs are independently powered via the BEE. All subsystems have independent latch-up and overcurrent protection.

The majority of the HyperScout subsystems can be operated independently from each other, offering an additional degree of freedom to optimize the power budget by powering up only the components that play a role within the specific operating mode. The ICU is always powered to serve as the contact point for communication with the S/C bus and to monitor the status of the payload throughout the orbit. During acquisition the FPAs, BEE and MMU are mainly used. In this phase for example, the OBDH will consume minimal power transferring the image data to disk. During processing both FPAs and BEE are powered off completely. The OBDH will normally consume the most power in the processing mode. The dependency of the subsystems per operating mode is reported in Figure 4.

The frequency of the observation mode can vary according to the mission planning. The acquired data is cached in the instrument MMUs for subsequent processing or transmission to ground, as requested by the S/C bus. The processing operating mode can be run using different processing subsystems. For this scope HyperScout-2 offers a powerful computational set of processors, including a FPGA, CPU, GPU and a dedicated board for AI inference represented by an Eyes Of Things (EOT) board consisting of the Intel® Movidius™ Myriad™ vision processing unit (VPU) designed for accelerating machine vision tasks. The Intel's Movidius™ Myriad™ 2 VPU (Ref. 5) (second generation VPU from Movidius™, an Intel® company) is specialized for AI, vision and imaging applications where both performance and low power consumption are important.

HyperScout-2 is therefore capable of performing a vast range of processing operations, that can be continuously updated during flight, even with completely new applications.

HyperScout-2 calibration and applications

In-flight Calibration approach: HyperScout-2 has been characterized in the laboratory before integration into the spacecraft. For what concerns in-flight calibration, typically on-board stimuli are used to calibrate the full optical chain for the long waves infrared channel, such as blackbodies. For a miniaturized instrument like HyperScout this is currently not an option. A lean system approach is adopted that relies on vicarious calibration in combination with a thermally stable instrument and detector.

For the absolute calibration of the HyperScout VNIR spectral channel, instrumented test sites of RadCalNet are considered. Test sites consist of large, homogeneous, cloud-free areas used as radiance or reflectance reference targets. The method relies on the comparison of satellite date with a simultaneous ground truth measurements. Two methods are distinguished:

• Reflectance based: the reflectance of the earth surface is measured on-ground, together with extinction depth and other meteorological parameters. Subsequently a radiative transfer model is used to convert these values to a TOA (Top of Atmosphere) radiance, which can be compared to the sensor data. These methods allow absolute radiometric calibration with a accuracy <5%.

• Radiance based: the radiance of the earth scene is measured at an altitude much above aerosol scattering. After correction for residual scattering and absorption, the radiance can be compared directly to the sensor for absolute radiometric calibration within 2.8% accuracy.

Pseudo invariant characterization sites (PICS), are sites with a high temporal or spatial stability. Desert sites are the most promising candidates as they are characterized by large, homogeneous areas and constant atmospheric conditions (no clouds). An extensive catalogue with site information is available on the websites of the USGS and CEOS. The most common application is to monitor the relative radiometric response of the sensor, without the need of an on-ground reference measurement.

Data will be crossed calibrated on the long term using Sentinel-2A/-2B, acquired over the same region of interest (ROI). For the purposes of cross-calibration of HyperScout, the Sentinel-2 satellites are considered thanks to the good match between the spectral bands of the Sentinel Multispectral Instrument (MSI) and HyperScout and the in-orbit HyperScout data collection opportunity provided by the FSSCat spacecraft with limited costs for cosine.

For the TIR channel in-flight calibration, a spacecraft maneuver with the instrument pointed at deep space is used to measure the instrument thermal background signal. As there is no black body calibrator on-board the payload for in-flight calibration, two options are considered: 1) intercalibration or cross-calibration, 2) vicarious calibration. Intercalibration calibrates a satellite instrument by relating its measurements to those of a well calibrated reference satellite instrument. This technique relates measurements in similar spectral bands. Well-calibrated reference satellite instruments that match each TIR band of HyperScout-2 are for example Sentinel-3, MODIS or Landsat-8. Vicarious calibration uses natural or artificial sites on the surface of the earth to calibrate satellite instruments. Within this technique, well-calibrated ground-based or airborne radiometers take measurements of a spectrally and spatially homogeneous test sites at the time of the satellite instrument overpass.

Two methods can be used to perform vicarious calibration in the thermal infrared:

• Radiance-based method;

• Temperature-based method.

The radiance-based method requires that the spectral response of the well-calibrated reference radiometer matches the spectral response of the satellite instrument. In practice, this condition is most likely not going to be fulfilled. Consequently, this method is also not considered to be a suitable option for inflight monitoring and calibration. In contrast to the radiance-based method, the temperature-based method in principle does not impose restrictions on the spectral response of the instrument. Hence, this method is considered a suitable option for in-flight monitoring and calibration. The following calibration sites could be envisaged for the in-flight calibration of HyperScout:

• Lake Tahoe on the California/Nevada border

- ideal thermal calibration target due to thin atmosphere above the lake thanks to the high altitude

- does not freeze in winter as it is very deep

- annual temperature ranges from about 4 to 20°C

- excluding instrument noise, uncertainty in the predicted at-sensor spectral radiance for TIR bands and skin surface temperature of 300 K is about 0.43 K - expressed in equivalent apparent temperature

- freely available data

• Salton Sea in Southern California

- less ideal target than Lake Tahoe due to thicker atmosphere (lower altitude)

- annual temperature variation ranges from about 4 to 35°C (extends temperature range compared to lake Tahoe)

- excluding instrument noise, uncertainty in the predicted at-sensor spectral radiance for TIR bands and skin surface temperature of 300 K is about 0.43 K - expressed in equivalent apparent temperature

- freely available data

• NOAA Ocean and Great Lakes

- larger surface area than Lake Tahoe and Salton Sea

- temperature variation ranges from about 3 to 30°C depending on the season and location

- excluding instrument noise, uncertainty in the predicted at-sensor spectral radiance for TIR bands and skin surface temperature of 300 K is about 0.48 K - expressed in equivalent apparent temperature

- freely available data.

cosine is also leading a Dutch consortium for the development and commercialization of a miniaturized spectroradiometric calibration stimuli, expected to be integrated and tested in orbit in late 2020 / early 2021. This miniaturized calibration source will be considered for future flights of HyperScout related products as well as third parties compact spectrometers.

HyperScout-2 data products

A number of data products have been selected for HyperScout-1 and this is being extended for the HyperScout-2 mission adding the potential applications enabled by the coexistence of reflectance and thermal information of the target. The target applications are: agriculture, thermal inertia and soil moisture, urban heat island, fire hazard and monitoring, and water quality.

In-Orbit Test Bed: A Platform for Artificial Intelligence Experiments

Applying Artificial Intelligence processing algorithms on the data acquired by HyperScout-2 directly onboard and in real time may represent a big leap forward in inferring and delivering Earth Observation derived information at a very high temporal resolution and with a very short timeliness. Alternatively, an interesting use case is the screening of the data before being downloaded. This is the case for the first demonstration that will be performed in orbit as part of the Φ-Sat-1 mission, where AI is leveraged to select the data to be downloaded based on the cloud coverage of the acquired scene. The application has been selected by ESA as the presence of clouds is one of the fundamental problems of optical remote sensing and it becomes particularly relevant considering the large data volume generated by Earth Observation missions based on hyperspectral imaging.

However, it should be noted that the HyperScout-2 processing chain has been designed in order to accept third party AI based software that can be run on the either on the VPU, or CPU/GPU, for testing purposes before being employed in operational scenario. The processing chain is depicted in Figure 7.

The first step prepares the data and corrects it for known aberrations. After the raw data is corrected, the spectral cube can be computed in order to produce spectral bands. The second pre-processing step is mandatory if the VPU is used. In this case the spectral cube is prepared and loaded into the VPU, ready to be analyzed by the AI algorithm. The inference step is use-case based, and as discussed can be performed on the VPU if high time efficiency is required.

Launch

Launch: The Φ-Sat-1 CubeSat mission (6U), as enhancement of the FSSCat mission, was launched as a passenger payload at 01:51 UTC, 03:51 CEST on 3 September, 2020 (22:51 local time on 2 September in Kourou).The launch vehicle was the VEGA SSMS (Small Spacecraft Mission Service) Proof Of Concept flight. ⁸⁾

Orbit: Sun-synchronous orbit; target orbit for the 7 microsatellites: altitude of 515 km, inclination of 97.45º; target orbit for the 46 nanosatellites: altitude of 530 km, inclination = 97.51º. The nominal mission duration (from liftoff to separation of the 53 satellites) is: 1 hour, 44 minutes and 56 seconds.

Passenger payloads (53) of the Vega rideshare mission

Arianespace has realized the first European “rideshare” mission for small satellites, with 53 satellites onboard the Vega launcher for 21 customers from 13 different countries. With this new SSMS (Small Spacecraft Mission Service) shared launch concept, Arianespace demonstrates its ability to respond – in an innovative and competitive manner – to institutional and commercial requirements of the growing market for small satellites. The total satellite launch mass was 1,327 kg. ⁹⁾

With the demonstration of its new SSMS service, Arianespace is strengthening its position in the growing market for small satellites. This service will soon be supplemented by the MLS (Multi Launch Service) – a similar offer available on Ariane 6, allowing Arianespace to increase the number of affordable launch opportunities for small satellites and constellations.

• ESAIL is a maritime microsatellite with a mass of 112 kg for AIS (Automatic Identification System) ship tracking operated by exactEarth. Is was built by a European manufacturing team led by the satellite prime contractor Luxspace. ESAIL features an enhanced multiple antenna-receiver configuration for global detection of AIS messages and high-resolution spectrum capture, which will enable the demonstration of advanced future services such as VDES (VHF Data Exchange System) message reception. ¹⁰⁾

• Lemur-2, eight 3U CubeSats built by Spire Global Inc., San Francisco, CA . These satellites carry two payloads for meteorology and ship traffic tracking. The payloads are: STRATOS GPS radio occultation payload and the SENSE AIS payload.

• TriSat is a 3U CubeSat (5 kg) imaging mission led by the University of Maribor, Slovenia. The mission is focused on remote sensing by incorporating a miniaturized multispectral optical payload as the primary instrument, providing affordable multispectral Earth observation in up to 20 non-overlapping bands in NIR-SWIR (Near to Short Wave Infrared) spectrum.

• The launch integrator company Spaceflight Inc. of Seattle WA is providing its services for four different customers with a total of 28 satellites. These are:

a) NewSat-6 (also written as ÑuSat-6), is a low Earth orbit commercial remote sensing microsatellite (43.5 kg) designed and manufactured by Satellogic S.A. with HQs in Argentina, a vertically integrated geospatial analytics company that is building the first Earth observation platform with the ability to remap the entire planet at both high-frequency and high-resolution. This is Satellogic’s 11th spacecraft in orbit, equipped with multispectral and hyperspectral imaging capabilities and it will be added to the company’s growing satellite constellation.

The spacecraft is named "Hypatia" after the philosopher, astronomer, and mathematician (350-415 A.D.) who lived in Alexandria, Egypt, and was a symbol of learning and science. She was renowned in her own lifetime as a great teacher and a wise counselor and became seen as an icon for women's rights and a precursor to the feminist movement. In line with Satellogic's NewSats already in orbit, Hypatia is equipped with sub-meter multispectral and 30 m hyperspectral cameras. This NewSat Mark IV is also equipped with new technologies in service of Satellogic's research and development of Earth-observation capabilities. Upon successful commissioning, these new capabilities will be available to existing Satellogic customers.

b) 14 Flock-4v, 3U CubeSats, next-generation SuperDove satellites of Planet Inc., San Francisco, they will join its constellation of 150 Earth-imaging spacecraft.

c) SpaceBEE, 12 (.25U) picosatellites of Swarm Technology which provide affordable global connectivity.

d) Tyvak-0171, an undisclosed minisatellite of Tyvak, developed by Maxar with a mass of 138 kg.

• Planet Inc. of San Francisco launches a total of 26 Flock 4v SuperDoves on this mission. They will be split into two batches on the same launch: 14 of them will be housed inside and deployed from ISL’s QuadPack deployers and the remaining 12 will be deployed from D-Orbit’s InOrbit Now (ION) freeflying deployment platform. ¹¹⁾

• Athena, a communications minisatellite mission (138 kg) of PointView Tech LLC, a subsidiary of Facebook. The objective is to provide broadband access (internet connectivity) to unserved and underserved areas throughout the world.

• AMICalSat, a 2U CubeSat, an educational mission, developed by CSUG (University of Grenoble Alpes, France) and MSU-SINP (Lomonosov Moscow State University-Skobeltsyn Institute of Nuclear Physics, Russia). The objective is to take pictures of the Northern light in order to reconstruct the particle precipitation into the polar atmosphere. The payload is a very compact, ultra-sensitive wide filed imager (f=23mm, aperture f/1.4). Firstly, AMICal Sat will observe auroras using nadir pointing, i.e. by determining the center of the Earth to map and link the geographical position of the auroral oval and its internal structures with solar activity. Secondly, the CubeSat will perform image capture ‘in limbo’ through tangential orientation with the Earth to capture the vertical profile of the auroras and match an altitude to their various emissions.

• PICASSO, a 3U CubeSat mission (mass of 3.8 kg) developed for ESA ( European Space Agency) led by BISA (Belgian Institute for Space Aeronomy), in collaboration with VTT Technical Research Center of Finland Ltd, Clyde Space Ltd. (UK) and the CSL (Centre Spatial de Liège), Belgium. The goal is to develop and operate a scientific 3U CubeSat.

• GHGSat-C1 of GHGSat Inc., Montreal, Canada, is the first of two nanosatellites (~16 kg) as the commercial follow-on to the GHGSat-D (CLAIRE) demonstration satellite developed and launched by UTIAS/SFL of Toronto in 2016. GHGSat monitors industries greenhouse gas (GHG) and air quality gas (AQG) emissions, including: oil & gas, power generation, mining, pulp & paper, pipelines (natural gas), landfill, chemicals, metals & aluminum, cement, agriculture, and transportation.

• NEMO-HD of SPACE-SI (Slovenian Center of Excellence for Space Sciences and Technologies) is a microsatellite (65 kg) developed at UTIAS/SFL of Toronto, Canada in cooperation with SPACE-SI. The NEMO-HD (Next-generation Earth Monitoring and Observation-High Definition) satellite is a high precision interactive remote sensing mission for acquiring multispectral images and real time HD video.

• FSSCat (Federated Satellite Systems on Cat) is the winner of the 2017 Copernicus Master “ESA Sentinel Small Satellite Challenge (S³)”. Proposed by the Universitat Politèctica de Catalunya (UPC) and developed by a consortium composed of UPC (ES), Deimos Engenharia (PT), Golbriak Space (EE), COSINE (NL) and Tyvak International (IT).

• Phi-Sat-1 (Φ-Sat-1) is the first on-board ESA initiative (6U CubeSat) on Artificial Intelligence (AI) promoted by the Φ Department of the Earth Observation Directorate and implemented as an enhancement of the FSSCat mission. Among mission objectives, scientific goals are Polar Ice and Snow monitoring, soil moisture monitoring, terrain classification and terrain change detection (i.e. hazard detection and monitoring, water quality), while technological goals are optical Inter-Satellite Link (OISL) demonstration.

• The RTAFSAT-1 (Royal Thai Air Force Satellite-1) mission, also referred to as NAPA-1, is a 6U CubeSat, the first remote sensing CubeSat mission for Thailand. The satellite will carry out an Earth Observation Demonstration mission with SCS Gecko Camera and Simera TriScape-100 payloads; the designed lifetime is 3 years.

• DIDO-3, a commercial 3U CubeSat mission of SpacePharma. The objective is to gather data by researching the effects of a microgravity environment on biological materials. SpacePharma from Israel will be is on board of SSMS POC with DIDO-3 Nanosatellite to perform biological experiment under Microgravity for several customers involved in pharmaceutical business, supported by Italian Space Agency (ASI) and Israeli Space Agency (ISA). Dido-3 will be monitored from the Ground Station developed by SpacePharma in Switzerland.

• SIMBA (Sun-Earth Imbalance), a 3U CubeSat mission led by the Royal Meteorological Institute Belgium, The objective is to measure the TSI (Total Solar Irradiance) and Earth Radiation Budget climate variables with a miniaturized radiometer instrument. This mission will help in the study of the global warming. This science mission will have a design lifetime of 3 years and the satellite performances will be monitored from ground station located in The Netherlands.

• TARS-1, a 6U CubeSat of Kepler Communications, developed at ÅAC Clyde Space for IoT (Internet of Things) applications. TARS-1 features deployable solar arrays, software defined radios (SDR), a narrowband communications payload and high gain antennas.

• OSM-1 Cicero, the first nanosatellite developed in Monaco by OSM (Orbital Solutions Monaco engineers, a 6U CubeSat with a mass of ~10 kg) based on the Tyvak Nano-Satellite Systems design. OSM plans to build nanosatellites to gather environment and climate data.

• TTU100, a 1U CubeSat developed at the Tallin University of Technology, Estonia. The objective is to test earth observation cameras and high-speed X-band communications. It will perform remote sensing in the visible and IR electromagnetic spectrum.

• UPMSat-2 (Universidad Politecnica de Madrid Satellite-2), a demonstration microsatellite (45 kg) of IRD-UPM.

Spacecraft

The FSSCat mission will demonstrate for the first time worldwide a reliable optical intersatellite link (O-ISL) between two 6U CubeSats flying in LEO (Low Earth Orbit). We have developed a technology demonstration with a full duplex O-ISL terminal of 1.5U in size (15cm x 10cm x 10cm), 85mm optical aperture, suitable for LEO to LEO operations on coarse pointing CubeSats (pointing accuracy < 0.5º).

The terminal adopts a novel adaptive variable divergence laser mechanism with a hybrid payload/platform seek-and-track pointing algorithm, and an amplitude modulated signal to ensure reliable optical communications at 1 Gbit/s at a nominal intersatellite distance of up to 2,000 km in LEO. In addition of establishing an intersatellite link, the terminal provides the ability for simultaneous imaging through the optical aperture and the improvement of the attitude determination capability of the hosting satellite through the mixing of optical information in the overall platform attitude feedback control loop. ¹²⁾

Mission Status

• May 12, 2022: Could we capitalise on the Earth-based digital revolution to make our satellites smarter? ¹³⁾

- Over the last decade, rapid advances in computing techniques, combined with exponential growth in the amount of data being collected, have led to a true revolution of AI feeding on big data on Earth.

- ESA launched the first artificially intelligent Earth observation satellite in September 2020. Named φ-sat (pronounced phi-sat), the satellite carried an AI-accelerator chip that automatically discards cloudy images and send only useful data down to Earth.

- Then, in April 2021 ESA Discovery invited business and academic communities to propose innovative ideas to fly onboard ESA's OPS-SAT mission, which is open to the public to experiment with. AI had already been used on OPS-SAT to rapidly process images of Earth in flight and to fully automate the mission; some of the new ideas also involve a variety of AI tools for novel applications.

- The latest step came in October 2021, when the Discovery element of ESA's Basic Activities – working together with F-lab and ESA's new commercialisation department – launched a call for ideas on how we can make space systems more powerful using novel computing techniques and AI. The most interesting ideas have now been selected for funding.

- "Following the announcement of the call during Φ-week by ESA's Director General, industry and academia could propose projects addressing innovative technologies and applications," explains Gabriele Meoni, an ESA Φ-lab research fellow focusing on AI. "We received a really interesting suite of ideas, addressing a variety of space domains, showing that there is big interest from European stakeholders on cognitive computing in space."

- Applying modern computing techniques – for example, processing data directly on board satellites – could transform space activities and the space economy in the coming decade. The ability to make satellites smarter offers significant benefits for future space missions and business models.

- “We intentionally made the scope of the campaign very broad to explore the full potential of this new computing paradigm,” continues Gabriele.

- Leopold Summerer, head of ESA's Advanced Concepts and Studies Office adds: "The call for ideas was driven by the vision of ESA Agenda 2025, which calls for Europe to step up its role in space, support commercialisation and help create new space markets."

- "We think that in-space cognitive computing offers substantial commercial potential for first movers, and had planned to fund up to ten teams to develop their ideas in more detail," continues Leopold. "However, European industry and academia proposed so many good ideas that we decided to increase the number of funded projects to 12. We expect these to help Europe take a step further towards fully benefitting from the still-emerging field of cognitive computing in space."

- "The selected ideas involve new technologies developed outside the space sector – including blockchain, edge computing, and neuromorphic computing (inspired by the workings of the human brain)," says Gabriele. "The ideas address amazing applications across the space domain including early detection of methane gas and natural disasters, autonomous rovers on the Moon, space surveillance and tracking – all because information could be processed more efficiently."

- The 12 projects are being funded by ESA's Discovery element; each will run for six months with a budget of 100,000 EUR. All are financed using the ESA Initial Support for Innovation (EISI) scheme, which focuses on knowledge and skill development rather than achieving specific results.

- "What is the potential of new computing techniques in space? What are the new applications, or new science to be carried out? What would their business value be? These studies will help ESA further explore how cognitive computing could reshape the future of space missions," concludes Gabriele.

• October 14, 2021: How can a digital replica of Earth help us understand our planet’s past, present and future? As part of the fourth edition of Φ-week taking place this week, a group of European scientists have put forward their ideas on the practical implementation of Digital Twins and the potential application areas for a Digital Twin Earth in the real world. ¹⁴⁾

- In the coming decades, population growth and human activities are expected to amplify the current pressures on critical resources such as fresh water and food, intensify the stress on land and marine ecosystems, as well as increase environmental pollution and its impacts on health and biodiversity.

- Responding to these challenges, ESA came together at the 2020 edition of Φ-week to discuss how Earth observation can contribute to the creation of a digital twin of Earth – a dynamic, digital replica of our planet which accurately mimics Earth’s behavior.

- Constantly fed with Earth observation data, combined with in situ measurements and artificial intelligence, Digital Twin Earth will help visualize and forecast natural and human activity on the planet. The model will be able to monitor the health of the planet, perform simulations of Earth’s interconnected system with human behavior, and help support European environmental policies (see Figure 17).

- In September 2020, ESA launched several Digital Twin Earth Precursor Activities to explore some of the main scientific and technical challenges in building a digital twin of Earth. These activities included: Forest, Hydrology, Antarctica, Food Systems, Ocean and Climate Hot Spots.

- Each activity addressed a different scientific, technical and operational challenge regarding Digital Twin Earth including the role of artificial intelligence and consistent data, stakeholder engagement scientific credibility and role of sectorial models and Information and Communication Technology (ITC) infrastructure.

- At this year’s Φ-week, experts from the community came forward with the results of the activities over the last year.

Digital Twin Antarctica

- Antarctica is a major reservoir of freshwater in the word, with a huge potential to contribute to sea level rise in the future. Current ice sheet models present major differences and deviations among models, as well as strong variability in unstable areas.

- Therefore, a digital twin of Antarctica is necessary. Noel Gourmelen, from the University of Edinburgh commented, “By harnessing satellite observations, numerical simulations, and Artificial Intelligence, we have built a twin of the Antarctic ice sheet system, its hydrology, surrounding ocean, atmosphere, and biosphere. We have used the Antarctic twin to track the whereabouts of melt water on and under the ice sheet, and to explore how fringing ice shelves melt under various hydrology scenarios.”

Digital Twin Food Systems

- The Food Systems digital twin simulates agricultural activities and interactions within ecosystems on a daily basis. Different models can be run separately for each simulation unit, depending on crop, water and irrigation management system.

- Chandra Taposea, from CGI IT UK Lt, said, “Digital Twin Earths and the scope we are trying to achieve is vital in helping us reach the next step in sense-making and decision-making, and be able to help both individual users and large-scale policy makers. Our Food Systems Digital Twin has managed to integrate models from different domains, looking at how extreme precipitation would affect global crop models, but not without its trials and tribulations.”

Digital Twin Hydrology

- Luca Brocca, from the National Research Council, Italy, explains what the Hydrology Digital Twin entails, “In the ESA Digital Twin Earth Hydrology project, we have developed a 4D reconstruction of dynamic hydrology at unprecedented resolution through the integration of Earth observation and an advanced modelling system. The DTE Hydrology Prototype has been used for water resources management and for identifying locations and times of risk for landslides and flooding in the Po River Basin, in northern Italy.”

Digital Twin Climate Impacts

- The Climate Impacts Digital Twin will enable decision makers, without expert technical knowledge, to generate and visualize, in real-time, decision-relevant information related to regionalised impacts of climate change.

- Robert Parker, from the University of Leicester, said, “Our Climate Impact Explorer Digital Twin, initially focused on African drought, utilizes an innovative combination of Earth observation, environmental modelling and Machine Learning to bring enhanced decision support capabilities directly to our stakeholders.

- “By emulating these complex models and deploying them as fast and simple cloud-based tools, our prototype helps democratize access to these expert systems, giving stakeholders the capability to explore potential climate-driven drought responses.”

Digital Twin Forest

- Matti Motus, Principal Scientist at VTT, explains how the Forest Digital Twin works: “This digital twin will be a specialized Digital Twin of Earth, providing a reconstruction of the forest system at levels of detail not possible with generic land surface models. Satellite-based Earth observation, especially the high-quality Copernicus Sentinel data, allows us to get unique and uniform information for all forests of the globe.

- “Translating this into understanding on forest structure and to drive models of forest functioning requires local measurements, which are far more scattered and heterogeneous. In the precursor project, we have learned how to overcome these obstacles and provide growth and carbon balance predictions for different forests in Europe. We know now that we have the basic tools and the computing power to build a fully functioning digital twin of forests. It has been a very exciting, yet demanding journey, especially considering that it was fully implemented during the Covid-related restrictions.”

• October 13, 2021: With a boom in the global market for Earth observation information and data products, participants at this year’s Φ-week conference have been digging deep into the ‘market perspective’. This important topic includes how to gain a better understanding of what governments, industry, the public and other users of Earth observation products and services need and expect to create value for society and the economy. ¹⁵⁾

- According to the latest report from the Geospatial Industry Outlook and Readiness Index, GeoBuiz, the Earth observation industry as a whole is estimated to have been worth almost US $58 billion in 2019, rising to almost US $76 billion in 2020. Moreover, it is thought that this steep rise merely marks the beginning of bigger things to come.

- In addition, with respect to Europe industry, the 2021 survey published by the European Association of Remote Sensing Companies, EARSC, indicates that in 2020 there was a 24% growth in the number of companies associated with the Earth observation industry, as well as a 24% increase in revenue and a 17% rise in employees, compared to 2019.

- Essentially, Earth observation satellite data made available through ESA and through Europe’s Copernicus program are free and open to all. While this traditional approach of providing Earth observation data works extremely well for the scientific community and established users of data-based services, and support business creation in Europe, the rapidly growing commercial market for Earth observation products opens a myriad of opportunities – for the Earth observation industry, and for the protection of our planet.

- During the session dedicated to the market perspective at Φ-week, Steve Bochinger from Euroconsult delivered an extremely positive view of the future Earth observation market with emerging new verticals, albeit not completely free from potential complications.

- ESA’s Geraldine Naja showed how ESA, with its newly established Directorate of Commercialization, Industry and Procurement, will take a new approach to supporting European industry, taking benefits from worldwide commercial and New Space opportunities.

- During his talk, Marc Nolla Harvey, Director of Strategic Initiatives at SAP, said simply, “Earth observation benefits companies by allowing them to make more profit, be more efficient and be more sustainable.”

- And, Raffaele Mauro, General Partner at Primo Space Fund, a venture investor organization focused on the space business, explained the why private investors such as venture capitalists increasingly see the New Space economy as a very interesting investment target, which explains the current boom in private investment.

- ESG (Environmental, Social and Governance) criteria are also increasingly important to companies, their investors and other stakeholders. With growing concern about companies’ ethical and sustainability practices, Earth observation has a critical role to play.

- While ESG certification picks up pace, the growth in Europe’s market for Earth observation is also closely linked the European Commissions’ Green Deal – which aims to transform the EU into a modern, resource-efficient and competitive economy, ensuring no net emissions of greenhouse gases by 2050. Businesses and industry in many sectors such as agriculture, energy, insurance, security, are clearly using Earth observation to help in their efforts to curb greenhouse emissions and adopt more sustainable and profitable practices.

- This growth in the Earth observation market comes at a time when there are an increasing number of commercial companies developing small satellites relatively quickly and selling the resulting data products directly to their customers alongside space agencies such as ESA, which tend to take the ‘slow road’ developing satellite missions, which either prove new observing techniques, respond to gaps in scientific knowledge or deliver systematic data over many years.

- Simonetta Cheli, ESA’s Head of the Earth Observation Strategy, Program & Coordination Office, said, “The need for new ways for ESA to support the commercial Earth observation industry to develop innovative satellite missions, products and services remains more important than ever, particularly with respect to growing market demands and to monitoring and managing the impacts of climate change. We are adopting the concept of New Space to explore different ways of bringing benefits to the commercial world, and ultimately help building a stronger and leading European EO economy.”

- Giuseppe Borghi, Head of ESA’s of Φ-lab Division, added, “Sessions at this year’s Φ-week are certainly sparking some interesting and lively discussions. We are hearing a lot about the needs of different sectors such as the agriculture, energy, re/insurance, security and private investors and how ESA can support the growth of a flourishing ecosystem. Commercialization is one of the priorities in ESA’s Agenda 2025 and we, as ESA, have to work together with the market, the Earth observation industry and private investors to ensure that everyone can benefit from the wealth of information we get from space.

- “We can do this in a number of ways. We are working hard to tap into disruptive technologies and business models, including quantum computing, machine learning and, in general, artificial intelligence, the internet of things and cloud computing in space. We are making transformative innovation funding available through challenges as announced by the ESA Director General during the Φ-week opening session. A key instrument is also our commercialization co-funding program InCubed, backed by support from our Member States and the partnerships we build with leading research centers, industry and private investors.”

- While Φ-week continues and ‘the market perspective’ becomes clearer, there are already other moves towards blending traditional Earth observation with business initiatives. For example, ESA has just signed a contract with ICEYE, a commercial provider of satellite radar imagery, to include their constellation of small satellites in the fleet of missions contributing to Europe’s Copernicus environmental monitoring program – a perfect example of New Space.

- This week has also seen the signing of a contract with the German Research Center for Artificial Intelligence to support a new development initiative focused on artificial intelligence for Earth Observation.

• October 11, 2021: Kicking off with a bold flourish, Φ-week 2021 promises to bring space even closer to the forefront of addressing society’s biggest challenges, namely issues associated with the climate crisis, while boosting the economy through transformative New Space, artificial intelligence, and quantum and cognitive computing. ¹⁶⁾

- ESA’s Φ-week is now in its fourth edition, and each year this forward-looking event is increasingly relevant to society, business and the economy.

- In his opening address, Josef Aschbacher, ESA’s Director General said, “Europe faces unprecedented societal, economic, and security challenges. Space has enormous untapped potential to play in tackling pressing current and future crises, while simultaneously providing new impulse for the European space sector. We need to develop smart, automated services and applications to protect lives of people and to increase the protection of assets and natural resources.”

- Dr Aschbacher went on to explain how his priorities include transforming ESA and boosting green and digital commercialization, as outlined in Agenda 2025.

- He also announced a brand-new challenge, making €1 million available for funding studies centered on artificial intelligence and cognitive computing in space.

- Launched in 2020, Φ-sat-1 – the first artificial intelligence to be carried on a European Earth observation mission – demonstrated the ability to run machine learning algorithms in space on an ultra-low power, off-the-shelf artificial intelligence accelerator chip.

- Φ-sat-1 enables the pre-filtering of Earth observation data so that only images with usable information are downlinked to the ground, thereby significantly reducing aggregated downlink costs.

- The next artificial intelligence mission, Φ-sat-2, is currently under development and will demonstrate new ways of collecting, distributing and analyzing Earth observation data.

- Dr Aschbacher continued, “Taking the potential of this new paradigm forward, ESA is planning a challenge to explore how the use of computing and artificial intelligence in space could help transform the way we develop space missions and applications. This will not be exclusive to the field of Earth observation, but also open to ideas linked to telecoms, exploration, robotics and more. We plan to select 10 activities to the value of €100 000 each.”

- The opening ceremony also saw talks from Toni Tolker-Nielsen, ESA Acting Director of Earth Observation Programs; Ekaterini Kavvada, Director of Innovation and Outreach at the European Commission’s DG DEFIS; Kris Peeters, Vice President of the European Investment Bank; Mark Boggett, co-founder and CEO of Seraphim Capital; and Rafal Modrzewski CEO and co-founder of ICEYE – all discussing their visions of New Space and the economy.

- As the speakers highlighted, this year’s Φ-week is set to focus on how the New Space economy is developing in Europe alongside fierce competition worldwide. New technology is key, and the week will dig deeper into the latest computing breakthroughs and trends such as quantum computing and cognitive computing.

- Toni Tolker-Nielsen said, “This week we are naturally building on the successes of the previous Φ-weeks, but importantly taking stock of the fast-pace of change all around us. The Earth observation market is booming worldwide, driven largely by demands linked to Europe’s Green Deal and by the New Space economy and opportunities with private investors.

- “ESA is in the perfect position to align with the commercial world, but more importantly help lead the way. We are already busy helping industry be more competitive though our co-funding InCubed program – which has boomed since the program’s inception. We will of course be taking this much further and exploring many new opportunities though out the week and beyond.”

• October 20, 2020: As ubiquitous as artificial intelligence has become in modern life — from boosting our understanding of the cosmos to surfacing entertaining videos on your phone — AI hasn’t yet found its way into orbit. ¹⁷⁾

- That is until Sept. 2, when an experimental satellite about the size of a cereal box was ejected from a rocket’s dispenser along with 45 other similarly small satellites. The satellite, named PhiSat-1, is now soaring at over 17,000 mph (27,500 km/h) in sun-synchronous orbit about 329 miles (530 km) overhead.

- PhiSat-1 contains a new hyperspectral-thermal camera and onboard AI processing thanks to an Intel® Movidius™ Myriad™ 2 Vision Processing Unit (VPU) — the same chip inside many smart cameras and even a $99 selfie drone here on Earth. PhiSat-1 is actually one of a pair of satellites on a mission to monitor polar ice and soil moisture, while also testing intersatellite communication systems in order to create a future network of federated satellites.

- The first problem the Myriad 2 is helping to solve? How to handle the large amount of data generated by high-fidelity cameras like the one on PhiSat-1. “The capability that sensors have to produce data increases by a factor of 100 every generation, while our capabilities to download data are increasing, but only by a factor of three, four, five per generation,” says Gianluca Furano, data systems and onboard computing lead at the European Space Agency, which led the collaborative effort behind PhiSat-1.

- At the same time, about two-thirds of our planet’s surface is covered in clouds at any given time. That means a whole lot of useless images of clouds are typically captured, saved, sent over precious down-link bandwidth to Earth, saved again, reviewed by a scientist (or an algorithm) on a computer hours or days later — only to be deleted.

- “And artificial intelligence at the edge came to rescue us, the cavalry in the Western movie,” says Furano. The idea the team rallied around was to use onboard processing to identify and discard cloudy images — thus saving about 30% of bandwidth.

- “Space is the ultimate edge,” says Aubrey Dunne, chief technology officer of Ubotica. The Irish startup built and tested PhiSat-1’s AI technology, working in close partnership with cosine, maker of the camera, in addition to the University of Pisa and Sinergise to develop the complete solution. “The Myriad was absolutely designed from the ground up to have an impressive compute capability but in a very low power envelope, and that really suits space applications.”

- The Myriad 2, however, was not intended for orbit. Spacecraft computers typically use very specialized “radiation-hardened” chips that can be “up to two decades behind state-of-the-art commercial technology,” explains Dunne. And AI has not been on the menu.

- Dunne and the Ubotica team performed “radiation characterization,” putting the Myriad chip through a series of tests to figure out how to handle any resulting errors or wear-and-tear.

- ESA “had never tested a chip of this complexity for radiation,” says Furano. “We were doubtful we could test it properly ... we had to write the handbook on how to perform a comprehensive test and characterization for this chip from scratch.”

- The first test, 36 straight hours of radiation-beam blasting at CERN in late 2018, “was a very high pressure situation,” Dunne says. But that test and two follow-ups “luckily turned out well for us.” The Myriad 2 passed in off-the-shelf form, no modifications needed.

- This low-power, high-performance computer vision chip was ready to venture beyond Earth’s atmosphere. But then came another challenge.

- Typically, AI algorithms are built, or “trained,” using large quantities of data to “learn” — in this case, what’s a cloud and not a cloud. But given the camera was so new, “we didn’t have any data,” says Furano. “We had to train our application on synthetic data extracted from existing missions.”

- All this system and software integration and testing, with involvement of a half-dozen different organizations across Europe, took four months to complete. “We were very proud to be able to be so quick and so efficiently flexible, to put everything on board in such a short time,” says Max Pastena, PhiSat-1 officer at ESA. As far as spacecraft development goes, the timeline “is a miracle,” adds Furano.

- “Intel has given us background support on the Myriad device when we’ve needed it, to enable PhiSat-1’s AI using our CVAI Technology,”

says Dunne. “That’s very much appreciated.”

- Unfortunately, a series of unrelated events — delays with the rocket, the coronavirus pandemic and unfriendly summer winds — meant the teams had to wait more than a year to find out if PhiSat-1 would function in orbit as planned.

- The Sept. 2 launch from French Guiana — a first-of-its-kind satellite ride-share run by Arianespace — went fast and flawlessly. For the initial verification, the satellite saved all images and recorded its AI cloud detection decision for each, so the team on the ground could verify that its implanted brain was behaving as expected.

- After a three-week deep breath, Pastena was able to proclaim: “We have just entered the history of space.”

- ESA announced the joint team was “happy to reveal the first-ever hardware-accelerated AI inference of Earth observation images on an in-orbit satellite.”

- By only sending useful pixels, the satellite will now “improve bandwidth utilization and significantly reduce aggregated downlink costs” — not to mention saving scientists’ time on the ground.

- Looking forward, the usages for low-cost, AI-enhanced teensy satellites are innumerable — particularly when you add the ability to run multiple applications.

- “Rather than having dedicated hardware in a satellite that does one thing, it’s possible to switch networks in and out,” says Jonathan Byrne, head of the Intel Movidius technology office. Dunne calls this “satellite-as-a-service.”

- Consider: When flying over areas prone to wildfire, a satellite can spot fires and notify local responders in minutes rather than hours. Over oceans, which are typically ignored, a satellite can spot rogue ships or environmental accidents. Over forests and farms, a satellite can track soil moisture and the growth of crops. Over ice, it can track thickness and melting ponds to help monitor climate change.

- Many of these possibilities will soon be tested: ESA and Ubotica are working together on PhiSat-2, which will carry another Myriad 2 into orbit. PhiSat-2 will be “capable of running AI apps that can be developed, easily installed, validated and operated on the spacecraft during their flight using a simple user interface.”

- For Intel, the potential impact is unquestionable. As Pastena puts it, we can eventually understand “the pulse of our planet.”

• September 28, 2020: ESA’s 2020 -week event kicked off this morning with a series of stimulating speeches on Digital Twin Earth, updates on Φ-Sat-1, which was successfully launched into orbit earlier this month, and an exciting new initiative involving quantum computing. ¹⁸⁾

Digital Twin Earth

- The third edition of the Φ-week event, which is entirely virtual, focuses on how Earth observation can contribute to the concept of Digital Twin Earth – a dynamic, digital replica of our planet which accurately mimics Earth’s behavior. Constantly fed with Earth observation data, combined with in situ measurements and artificial intelligence, the Digital Twin Earth provides an accurate representation of the past, present and future changes of our world.

- Today’s session opened with inspiring statements from ESA’s Director General, Jan Wörner, ESA’s Director of Earth Observation Programs, Josef Aschbacher, ECMWF’s Director General, Florence Rabier, European Commission’s Deputy Director General for Defence Industry and Space, Pierre Delsaux, as well as Director General of DG CONNECT at the European Commission, Roberto Viola.

- Pierre Delsaux commented, “As our EU Commission President repeated recently during her State of the Union speech, it’s clear we need to address climate change. The Copernicus program offers us some of the best instruments, satellites, to give us a complete picture of our planet's health. But space is not only a monitoring tool, it is also about applied solutions for our economy to make it more green and more digital.”

- Roberto Viola said, “Φ-week is the week for disruptive technology and it is communities like this that our European programs were designed to support.”

- Florence Rabier added, “Machine learning and artificial intelligence could improve the realism and efficiency of the Digital Twin Earth – especially for extreme weather events and numerical forecast models.”

- Jan Wörner concluded, “Φ-week is the perfect example of the New Space approach focusing on disruptive innovation, artificial intelligence, agility and flexibility.”

- During the week, experts will come together to discuss the role of artificial intelligence for the Digital Twin Earth concept, its practical implementation, the infrastructure requirements needed to build the Digital Twin Earth, and present ideas on how industries and the science community can contribute.

Φ-sat-1

- Earlier this month, on 3 September, the first artificial intelligence (AI) technology carried onboard a European Earth observation mission, Φ-Sat-1, was launched from Europe’s spaceport in French Guiana. An enhancement of the Federated Satellite Systems mission (FSSCat), the pioneering artificial intelligence technology is the first experiment to improve the efficiency of sending vast quantities of data back to Earth.

- Today, ESA, along with cosine remote sensing, are happy to reveal the first ever hardware-accelerated AI inference of Earth observation images on an in-orbit satellite – performed by a Deep Convolutional Neural Network, developed by the University of Pisa.

- Φ-Sat-1 has successfully enabled the pre-filtering of Earth observation data so that only relevant part of the image with usable information are downlinked to the ground, thereby improving bandwidth utilization and significantly reducing aggregated downlink costs.

- Initial data downlinked from the satellite has shown that the AI-powered automatic cloud detection algorithm has correctly sorted hyperspectral Earth observation imagery from the satellite’s sensor into cloudy and non-cloudy pixels.

- Massimiliano Pastena, Φ-sat-1 Technical Officer at ESA, commented, “We have just entered the history of space.”

- Today’s successful application of the Ubotica Artificial Intelligence technology, which is powered by the Intel Movidius Myriad 2 Vision Processing Unit, has demonstrated real on-board data processing autonomy.

- Aubrey Dunne, Co-Founder and Vice President of Engineering at Ubotica Technologies, said, “We are very excited to be a key part of what is to our knowledge the first ever demonstration of AI applied to Earth Observation data on a flying satellite. This is a watershed moment both for onboard processing of satellite data, and for the future of AI inference in orbital applications.”

- As the overall 2017 Copernicus Masters winner, FSSCat, was proposed by Spain’s Universitat Politècnica de Catalunya and developed by a consortium of European companies and institutes including Tyvak International.

Quantum computing

- Also mentioned in his opening speech this morning, Josef Aschbacher made a special announcement regarding an exciting new ESA initiative, the “EOP AI-enhanced Quantum Initiative for EO - QC4EO” in collaboration with the European Organization for Nuclear Research (CERN).

- Quantum computing has the potential to improve performance, decrease computational costs and solve previously intractable problems in Earth observation by exploiting quantum phenomena such as superposition, entanglement and tunnelling.

- The initiative involves creating a quantum capability which will have the ability to solve demanding Earth observation problems by using artificial intelligence to support programs such as Digital Twin Earth and Copernicus. The initiative will be developed at the Φ-lab – an ESA laboratory at ESA/ESRIN (ESA’s Center for Earth Observation) in Frascati, Italy, which embraces transformational innovation in Earth observation.

- ESA and CERN enjoy a long-standing collaboration, centered on technological matters and fundamental physics. This collaboration will be extended to link to the CERN Quantum Technology Initiative, which was announced in June 2020 by the CERN Director General, Fabiola Gianotti.

- Through this partnership, ESA and CERN will create new synergies, building on their common experience in big data, data mining and pattern recognition.

- Giuseppe Borghi, Head of the Φ-lab, said, “Quantum computing together with AI are perhaps the most promising breakthrough to come along in computer technology. In the coming years, we will see more Earth or space science disciplines employing current or future quantum computing techniques to solve geoscience problems.”

- Josef Aschbacher added, “ESA will exploit the broad range of specialized expertise available at ESA and we will place ourselves in a unique position and take a leading role in the development of quantum technologies in the Earth observation domain.”

- Alberto Di Meglio, Coordinator of the CERN Quantum Technology Initiative, said, “Quantum technologies are a rapidly growing field of research and their applications have the potential to revolutionize the way we do science. Preparing for that paradigm change, by building knowledge and tools, is essential. This new collaboration on quantum technologies bears great promise.”

References

1) ”First Earth observation satellite with AI ready for launch,” ESA, 12 September 2019, URL: http://www.esa.int/Our_Activities/Observing_the_Earth/First_Earth_observation_satellite_with_AI_ready_for_launch

2) ”Smallsats win big prize at Copernicus Masters,” ESA, 8 November 2017, URL: http://www.esa.int/Our_Activities/Observing_the_Earth/Copernicus/Smallsats_win_big_prize_at_Copernicus_Masters

3) Marco Esposito, Bernardo Carnicero Domınguez, Massimiliano Pastena, Nathan Vercruyssen, Simon Silvio Conticello, Chris van Dijk, Pierluigi Foglia Manzillo, Rick Koeleman,”Highly integration of hyperspectral, thermal and artificial intelligence for the ESA PhiSat-1 mission,” Proceedings of the 70^th IAC (International Astronautical Congress), Washington DC, USA, 21-25 October 2019, URL: https://iafastro.directory/iac/proceedings/IAC-19/IAC-19/B4/4/manuscripts/IAC-19,B4,4,4,x54611.pdf

4) Massimiliano Pastena, Bernardo Carnicero Domínguez, Pierre Philippe Mathieu, Aman d a Re ga n,Marco Esposito, Simon Conticello, Chris Van Dijk, Nathan Vercruyssen, Pierluigi Foglia Manzillo, Rick Koelemann, John Hefele, ” Earth Observation Directorate NewSpace Initiatives,” Proceedings of the 70^th IAC (International Astronautical Congress), Washington DC, USA, 21-25 October 2019, paper: SSC19-V-05, URL: https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=4391&context=smallsat

5) A Camps, A. Golkar, A. Gitierrez, J. Ruit-de-Azua, J. Muñoz‐Martin, L. Fernandez-Capon, C. Diez, A. Aguilella, S. Briatore, N. Garzaniti, F. Nichele, R. Mozzillo, M. Vanotti, M. Esposito, B. Carnicero, G. Filippazzo, and A. Regan, “FSSCat: A Cubesat-based Tandem Mission for Earth Observation of the Polar Regions” Living Planet Symposium ‘19, Milan, May 2019

6) Marco Esposito, Alessandro Zuccaro Marchi, “In-Orbit Demonstration of the first hyperspectral imager for nanosatellites” ICSO ( International Conference on Space Optics) 2018, Chania, Greece

7) Pierluigi Foglia Manzillo, Ljubisa Babic, Marco Esposito, Cristian Corneliu Chitu, Marco Canetri, Paolo Corradi, Jesus Gil Fernandez, Michael Kueppers. ”TIRI: A Multi-Purpose Thermal Infrared Payload for Planetary Exploration” Proceedings of the 69th IAC (International Astronautical Congress) Bremen, Germany, 1-5 October 2018, paper: IAC-18,A3,4B,4, URL: https://iafastro.directory/iac/proceedings/IAC-18/IAC-18/A3/4B/manuscripts/IAC-18,A3,4B,4,x47339.pdf

8) ”Vega return to flight proves new rideshare service,” ESA Enabling & Support, 3 September 2020, URL: https://www.esa.int/Enabling_Support/Space_Transportation/Vega/Vega_return_to_flight_proves_new_rideshare_service

9) ”With Vega, Arianespace successfully performs the first European mission to launch multiple small satellites,” Arianespace Press Release, 3 September 2020, URL: https://www.arianespace.com/press-release/with-vega-arianespace-successfully-performs-the-first-european-mission-to-launch-multiple-small-satellites/

10) ”Vega flight VV16,” Wikipedia, URL: https://en.wikipedia.org/wiki/Vega_flight_VV16

11) Mike Safyan, ”Planet’s First Launch of 2020: 26 SuperDoves on a Vega,” Planet, 13 February 2020, URL: https://www.planet.com/pulse/planets-first-launch-of-2020-26-superdoves-on-a-vega/

12) ”Overview of the FSSCAT Optical Intersatellite Link (O-ISL) - Technology Demonstration,” EPFL, 2019, URL: https://memento.epfl.ch/event/overview-of-the-fsscat-optical-intersatellite-link/

13) ”Smarter satellites: ESA Discovery accelerates AI in space,” ESA Enabling & Support, 12 May 2022, URL: https://www.esa.int/Enabling_Support/Preparing_for_the_Future/Discovery_and_Preparation/Smarter_satellites_ESA_Discovery_accelerates_AI_in_space

14) ”Working towards a Digital Twin of Earth,” ESA Applications, 14 October 2021, URL: https://www.esa.int/Applications/Observing_the_Earth/Working_towards_a_Digital_Twin_of_Earth

15) ”Putting the Earth observation into ‘the market perspective’,” ESA Applications, 13 October 2021, URL: https://www.esa.int/Applications/Observing_the_Earth/Putting_the_Earth_observation_into_the_market_perspective

16) ”Spotlight on climate and the New Space economy,” ESA Applications, 11 October 2021, URL: https://www.esa.int/Applications/Observing_the_Earth/Spotlight_on_climate_and_the_New_Space_economy

17) ”Intel Powers First Satellite with AI on Board,” Intel, 20 October 2020, URL: https://newsroom.intel.de/news/intel-powers-first-satellite-with-ai-on-board/#gs.jvbakw

18) ”Digital Twin Earth, quantum computing and AI take centre stage at ESA’s Φ-week,” ESA Applications, 28 September 2020, URL: https://www.esa.int/Applications/Observing_the_Earth/Digital_Twin_Earth_quantum_computing_and_AI_take_centre_stage_at_ESA_s_Ph-week
 

The information compiled and edited in this article was provided by Herbert J. Kramer from his documentation of: ”Observation of the Earth and Its Environment: Survey of Missions and Sensors” (Springer Verlag) as well as many other sources after the publication of the 4th edition in 2002. - Comments and corrections to this article are always welcome for further updates (eoportal@symbios.space).