EO-ALERT — Next Generation Satellite Processing Chain for Rapid Civil Alerts
The EO-ALERT project is an H2020 European Union research activity coordinated by Deimos Space, started in January 2018 and lasting three years, aiming at achieving very high throughput and low latency in the delivery of Earth observation images. This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 776311. The partners of the project are Deimos Space, DLR, Technische Universität Graz, Politecnico di Torino, OHB Italia and Deimos Imaging, with the participation of the Spanish State Meteorological Agency (AEMET) as a third party. The consortium represents a balance over the full R&D cycle, from university to industry, and over the full EO value chain. 1)
Earth observation (EO) data provided by deployed remote sensing satellites now provides a basic service to society, with great benefits to the civilian. The data generated by EO satellites is now ubiquitously used throughout society for a range of diverse applications, such as environment and resource monitoring, emergency management and civilian security; in Europe, for example, the Copernicus program provides EO data products for many of these fundamental needs, and Europe plays key roles in the global EO networks. EO data provides the basis for governmental monitoring of the Earth’s resources to support planning and management, in particular for global warming and smart cities, allows for the informed response of authorities in the case of emergencies and provides the basis for a number of commercial EO products for civil market End Users, such as precision agriculture, oil & gas, and green energy, to name just a few. As such, EO satellites, and their data products, are fundamental to modern society and are one of the cornerstones of what ESA terms “Space 4.0” , which is expected to transform the lives of citizens, decision and policymakers, and businesses in the near future.
Current trends in the EO market, driven by the sharp growth in EO products applications, with a potential European EO downstream market at EUR 2.8 billion, are towards ever greater demands on the amount, type and quality of the EO products available to the End User, placing upstream demands on the satellite and its constellations, increasing the amount of data generated and transferred to ground. This is aligned with increase in image resolution.
Data latency is also a key requirement. End Users require data is available in a very short time period, with low latency – Near Real Time (NRT) – or very low latency vs enhanced-NRT, and also reliably. This has lead to the New Space “EO 2.0” actors to focus on new missions, and large constellations. Notably, what it is currently referred to as NRT consists of the provision of image products in the range of 1 to 3 hours; e.g. Sentinel-1 makes ocean products available within 1 hour of observation over NRT areas with a subscription . New Space actors call for latencies below 30 minutes, and falling.
This trend is also confirmed by the initiative “Space Data Highway for Europe” promoted by ESA and based on EDRS, whose main goal is to relay EO raw data between LEO satellites and Earth, via satellites in GEO. It enables transmission of large amounts of raw data with reduced delay, using innovative laser communication. The latency of this approach is announced to be 10-15 minutes. Although effective, this solution focuses on a complex and expensive communication infrastructure (2 or 3 GEO satellites), which precludes from a wide utilisation. Importantly, it does not pay attention to technology developments like those proposed by EO-ALERT, which will enhance the satellite onboard data handling and processing capabilities in a broad sense, to achieve comparable or better latency figures. EO-ALERT proposes a different approach.
EO-ALERT proposes the definition and development of the next-generation EO data and processing chain, based on a novel flight segment architecture that moves optimized key EO data processing elements from the ground segment to on-board the satellite, with the objective of providing the EO products to the end user with very low latency (enhanced-NRT) for increased throughput. 2)
The main objective of EO-ALERT is that of developing, in a fully integrated approach, the technological building blocks required to achieve the primary goal of a next-generation EO data and processing chain, to provide enhanced EO products and services in terms of high availability rate and very low latency (e.g. rapid meteorological and civil security image products and warnings).
Objective 1: Identify and define, based on market needs, an innovative EO data and processing chain for next generation satellites. The objective is the identification of the specific needs driving the innovative next-generation data and processing chain, via the role of the multiple End Users in the project, the identification of EO data chain solution approaches and their trades, through to the definition of the data chain, its user requirements (scientific, institutional and commercial) and the applications and scenarios where it is most needed and will have the most impact.
Objective 2: Design, develop, prototype to breadboard and verify the next-generation EO data and processing chain key technologies. Consistent with COMPET-3 being a call focused on technology research, a core objective of EO-ALERT is devoted to develop the basic technologies that will play a key role to enhance the performance of the EO data and processing chain. The objective is to arrive at a set of key HW and SW technology products, as the building blocks of the data chain, that are innovative, so as to meet the challenge of the new architecture, and fully verified against their respective requirements, for later integration within the project to test the full data chain and for exploitation individually or collectively in other projects/programs. The verification using a representative space HW test bench will allow achieving TRL 5/6.
Objective 3: Verify the key EO data & processing chain technologies as an integrated chain in the HW bench. The objective is to integrate the EO data and processing chain building blocks, in the key HW and SW technology products, to form the integrated data and processing chain, and perform its testing and evaluation, to achieve a TRL5 for the full chain. For this the HW/SW products will be integrated into a high-speed avionics test bench, with representative space HW. The testing will use archived real EO sensor data from representative scenarios, to allow evaluation of the performance and TRL.
Objective 4: Experimentally validate and evaluate the key technologies as an integrated data and processing chain using EO sensor data acquired for the project. A dedicated experimentation campaign will be carried out to determine the potential of the proposed next-generation EO data and processing chain, and its integrated technology HW and SW products, against requirements identified by potential end users (scientific, institutional and commercial). The evaluation of the results will be performed by several end users.
Objective 5: Position the technologies in the EO market for exploitation in coming European and international EO programs. A key objective is to inject the EO-ALERT technological products in upcoming commercial and institutional programs. According to the market report issued by Copernicus , EO data and value added services has shown a continuous growth over the last years to reach EUR 2.75 billion in 2015 and it is expected to continue this trend to reach EUR 5.3 billion by 2020. This scenario will open up a number of opportunities to actors not only in the domain of downstream applications but in the whole value chain, provided they are able to promptly answer to evolving user needs.
August 2021: The proposed EO-ALERT architecture is enabled by on-board processing, recent improvements in processing hardware using Commercial Off-The-Shelf (COTS) components, and persistent space-to-ground communications links. EO-ALERT combines innovations in the on-board elements of the data chain and the communications, namely: on-board reconfigurable data handling, on-board image generation and processing for the generation of alerts (EO products) using Machine Learning (ML) and Artificial Intelligence (AI), on-board AI-based data compression and encryption, high-speed on-board avionics, and reconfigurable high data rate communication links to ground, including a separate chain for alerts with minimum latency and global coverage. 3)
Over the past 50 years, the classical EO data chain that has been mastered, involves the acquisition of sensor data on-board the satellite, its compression and storage on-board, and its transfer to ground by a variety of communication means, for later processing on ground and the generation of the downstream EO products. The data provided by this data chain is nowadays ubiquitously used throughout society for a range of diverse applications, such as environment and resource monitoring, emergency management and civilian security.
Data latency has become a key requirement in the EO market, since end-users require that data is available in a very short time interval. In the case of polar platform satellites, what it is currently referred to as near realtime (NRT) consists of the provision of image products in the range of 1 to 3 hours; e.g. Sentinel-1 makes ocean products available within 1 hour of observation over NRT areas. Local services are also offered in quasi-real-time, referring to latencies in the order of 15 minutes to 30 minutes, but this importantly is not offered globally .
Current market trends are moving beyond NRT applications, to applications with quasi-real-time latencies in the order of 30 minutes to 15 minutes, and provided globally. The latency performance concept behind EO-ALERT is to achieve latencies well below 15 minutes for the EO products delivery. To be precise, the definition of latency can be taken to be: the time from the collection of the last photons by the payload, through to the time that the data is converted to a specified EO product and that this product is delivered to the user (i.e. user portal).
Based on this performance concept, and latency definition, EO-ALERT has a goal latency of less than one minute, globally, and requires a maximum latency of less than 5 minutes, globally, for both SAR and optical EO products.
To achieve the challenging latency objectives, EO-ALERT has performed innovations in both the functional and physical architecture of the EO processing and data chain, from the output of the payload, followed by the on-board processing, through the communications and finally the reception on the ground of the EO products.
Functional Architecture: Figure 2 shows the high-level functional architecture implemented in EO-ALERT. To achieve the target latency in EO products delivery, the proposed functional architecture includes several innovative elements: on-board processing of the payload raw data to L1, processing on-board of the L1 product to generate the EO product (e.g. ship detection alert), reconfigurable on-board data handling to prioritize the EO products over raw data, and multiple communications channels, to provide for a global alerts (EO product) delivery via a satellite-relay.
To ensure the system is suitable for several mission scenarios and multiple payload types, the functional architecture is designed to be modular, scalable and reconfigurable. The entire data-chain is divided into several functional blocks, each one implemented on dedicated software and/or hardware computing resources. Each function can be configured or changed with no or little impact on the others, and the available processing power can be assigned to each function based on the mission requirements. With this approach, the system can process different data types (e.g. optical and SAR data) from several sensors, over a wide range of dataset sizes.
The elements of concern to the developments in the EO-ALERT project are those of the image processing chain (IPC), from the payload on-board the satellite, through the on-board data processing unit (the payload data processing unit, PDPU), the flight-to-ground communications subsystem, and finally the ground units for decryption/decompression, as needed.
Here we focus on the physical architecture and HW selection for the PDPU and the communications subsystem.
Payload Data Processing Unit Physical Architecture:
Hardware selection: The physical design of the PDPU avionics is implemented as a hybrid solution, i.e. a solution that uses both COTS (Commercial Off-The-Shelf ) and space qualified components. COTS are used in conjunction with mitigation techniques to increase robustness of the design against radiation effects, whereas space qualified components are used for the critical functions. This choice allows keeping weight, volume and cost of the PDPU (Payload Data Processing Unit) low with respect to an all space-grade design and it takes advantage of the stateof-the-art technology and processing power of the latest COTS components. This last point has proven to be fundamental to reach the project’s latency goals.
Reliability in the LEO (Low Earth Orbit) environment is addressed through the employment of radiation mitigation techniques, redundancy, extensive telemetry collection and FDIR (Fault Detection, Isolation and Recovery) actions so that the solution is more robust than an all-COTS design.
Physical Architecture for the PDPU: The top-level physical architecture of the EO-ALERT PDPU is shown in Figure 3. The PDPU weighs less than 8kg in a 10U volume, although smaller Size, Weight and Power (SWaP) constraints can be met if needed. The architecture is based on the Compact Peripheral Component Interconnect (cPCI) Serial Space standard which guarantees a modular and scalable approach to on-board processing. This standard provides a backplane with two system slots and up to seven peripheral boards. The whole system is supervised by a shelf controller that can check the status of the boards and control their power supply.
The system slots are at the center of a star connection with point-to-point high-speed links to the peripheral boards, which in turn are interconnected with a mesh of high-speed links.
In the EO-ALERT configuration system slots are used for scheduling, compression, encryption and data handling tasks, while five peripheral slots are dedicated to processing functions. Each system slot is connected directly, using the star links, to two “Master” processing boards. Master boards can offload computation to slave boards using mesh links.
Each peripheral board can be reconfigured from System Boards to handle optical (Visible Near Infrared VNIR, TIR) or SAR data so that the system can dynamically adapt to workloads and recover from failures; the two system slots implement an intrinsic redundancy without the need of duplicating the entire PDPU.
The shelf controller design features all space-grade components and contains Latch-up Current Limiting (LCL) circuitry for each board and a supervising microcontroller. All boards are based on the powerful Xilinx Zynq US+ ZU19EG Multi-processor System-on -Chip (MPSoC), featuring a quad core ARM processor and a large FPGA (Field-Programmable Gate Array) built onto the same die.
Communications Physical Architecture: Figure 4 depicts the block diagram of the communications system for EO-ALERT. It consists of a redundant Ka-band transmitter, an S-band downlink chain for local alert delivery and the INMARSAT IDRS ( Inter-satellite Data Relay System) modem for global alert delivery.
Processed data from the avionics system enter a multiplexer which directs the data to the relevant subsystem:
• Raw data, processed images and alerts are forwarded to the Ka-band system.
• Alerts are sent simultaneously to the S-band transmitter and the IDRS modem.
The Ka-band transmitter consists of a FEC (Forward-Error Correction) encoder, a modem and a solid-state power amplifier with an output power of 10 W. QPSK, 8PSK, 16APSK, 32APSK and 64APSK are the envisaged modulation schemes, the coding scheme is SCCC (Serial Concatenated Convolutional Coding). Data rates up to 2.6 Gbit/s are supported by a commercial product offered by TESAT. In our case the effective user data rate was set to 1.8 Gbit/s. The Ka-band transmitter is fully redundant, feeding horn antennas, one for right-hand circular polarization, the other one for left-hand circular polarization.
The S-band system for local alert transfer consists of a FEC encoder (LDPC), a QPSK modulator, an SSPA and a patch antenna, supporting a data rate of 1 Mbit/s to small hand-held terminals for rescue teams.
The IDRS modem for global alert transmission is connected to a switching antenna composed of 7 patches. The patch with best coverage to an INMARSAT satellite is automatically activated. The maximum data rate is 250 kbit/s.
Two reference scenarios are used in EO-ALERT to test and demonstrate, using real satellite and EO payload data, the correctness of the architecture and the performance of the system. More information can be found in reference [4)].
Ship Detection Scenario
A very low latency ship detection and monitoring service was selected as one of the reference scenarios. The intention was to be able to develop on-board the satellite an alert, similar to the EMSA (European Maritime Safety Agency) Vessel Detection System (VDS), and send this directly to an end user globally, including the following parameters, with a very short latency (goal of 1 minute, requirement below 5 minutes):
- Position & movement (velocity, heading).
- Ship details (size, width, etc)
- Ship image (clipping thumbnail).
Both a SAR (TerraSAR-X) and Optical (DEIMOS-2) satellite, with VHR payload, are used to assess this service.
Extreme Weather Scenario
For the extreme weather scenario, two types of detections have been considered: convective storms (nowcasting) and wind speed.
Convective Storm Nowcasting: A very low latency meteorological nowcasting service for severe convective storms was selected as one of the extreme weather scenarios. The intention is to develop on-board the satellite an alert, similar to the NWCSAF [5)] RDT -CW (Rapidly Developing Thunderstorms – Convection Warning), and send this directly to an end user globally. For this the SEVIRI optical VIS/TIR instrument, on-board the MSG satellite, is used to assess this service.
Sea Sate Wind Speed: A very low latency maritime wind speed and wave height service was selected as one of the extreme weather scenarios. This uses SAR (TerraSAR-X) satellite data to derive the desired information directly from the sea surface.
In order to perform the verification and validation activities of the EO-ALERT architecture, an avionics test-best is employed (Figure 5). The Avionics Test-Bench (ATB) consists of representative test environment for the full EO-ALERT image processing chain. It includes a scaled-down version of the PDPU subsystem design (Figure 6), composed by commercial hardware (not cPCI Serial Space compliant) and offering four boards instead of seven. The ATB includes a transceiver/receiver communications subsystem emulator and communications subsystem emulator and communications hardware (Figure 7), which is needed to test the transmission to the Ground Segment (GS), both using the satellite relay (L-band GEO-relay, Figure 9) and the different direct-to-ground channels (Ka-band and S-band, Figure 7 and Figure 8).
End-to End Data Chain Results
A key outcome of the EO-ALERT project to date is that the performance of the data chain has been confirmed, both analytically and through hardware testing, covering the full data chain (payload to ground). This section presents the results of the project in terms of the current latency of the EO products, in the different reference scenarios.
Ship Detection Scenario Results
The ATB (Avionics Test-Bench), including the communications units and emulator, have been used in ground testing to quantify the latency of the ship detection service.
For the ship scenario, the optical processing uses the optical DEIMOS-2 VHR satellite raw data. The testing is performed in a configurable multi-board scheme. Each board processes about 100 km2 at ~0.9 m resolution. To process this area, the entire on-board processing chain (Figure 10), from raw data to EO product delivery to the communications subsystem,takes less than 45 s running on a single Xilinx Zynq US+ board. Including the communication of alerts (ship detection and thumbnail image) through a global communications link, the total time is for alert generation and delivery is typically less than 1.5 minutes, and below 1 minute in the case that the number of alerts to be transmitted is small (i.e. less than 20 ship detection alerts). If the processing is parallelized over multiple boards, the ship service can be performed in less than 1 minute in all scenarios. More information on the optical processing chain for ship detection, and HW testing, can be found in [6)], [7)].
For the ship scenario, the SAR processing uses the SAR TerraSAR-X satellite data. The testing is performed in a single-board scheme. Each board processes about 400 km2 at ~4 m resolution. The entire on-board processing chain (see Figure 11) takes less than 40 s running on a single Xilinx Zynq US+ board. Including the communication of alerts (ship detection and thumbnail image) through a global communications link, the total time is for alert generation and delivery is typically less than 1.5 minutes, and below 1 minute in the case that the number of alerts to be transmitted is small (i.e. less than 20 ship alerts). More information on the SAR processing chain can be found in [8)].
Extreme Weather Scenario Results
The ATB (Avionics Test-Bench) has also been used in ground testing to quantify the latency of the extreme weather service.
For the extreme weather scenario for wind speed and wave height, using the satellite TerraSAR-X data, the latency for the product provision is similar to that for the ship scenario with SAR. The total time for alert generation and delivery is less than 1.5 minutes.
For the extreme weather nowcasting for convective storm detection and monitoring, using SEVIRI optical VIS/TIR data, the total time for alert generation (Figure 12) and delivery is less than 1 minute, noting that in this case, due to the GEO satellite use, both a direct-to-ground and global communications link suffice. More information on the extreme weather processing chain and scenario in can be found in [9)].
In summary, the EO-ALERT EO data processing chain and architecture, based on a novel flight segment architecture enabled by on-board processing, recent improvements in processing hardware using COTS (Commercial Off-The-Shelf) components, and persistent space-to-ground communications links, has been shown to be feasible and performing through avionics test-bench testing. The architecture provides the service of delivery of EO products to the end user with very low latency (in almost-real-time). Hardware testing shows latencies below 1.5 minutes for EO products delivery, globally, in all scenarios, reaching latencies below 1 minute in some scenarios, such as the ship detection service when the on-board processing isparallelized or the number of alerts is limited. Further ATB testing during Q3 2021 aims to demonstrate the complete viability of the EO-ALERT concept and architecture, and fully assess its performance. This will lead to the achievement of TRL 4-5 maturity for the architecture and technologies, positioning the EOALERT architecture and technologies for use in upcoming programmes, where EO product latency, and general satellite autonomy, are key mission enablers.
• January 21, 2021: The concept behind EO-ALERT H2020 project led by DEIMOS Space has been validated after successful tests performed at Graz University of Technology, in Austria. 10)
- The tests were conducted on the global real-time relay service using the Inmarsat and Addvalue Innovation Inter-satellite Data Relay System (IDRS) service, now operational in low Earth orbit (News). The objective was to prove the suitability of using the IDRS system for one of the EO-ALERT application scenarios: ship detection and classification, based on an EMSA Vessel Detection Service (VDS)-like service. The end-to-end global delivery time was confirmed to be below 1 minute for a single alert, meaning that ship detection and classification alerts from satellite can be delivered to ground in just seconds after image acquisition.
- Results show that the IDRS system is suitable for EO-ALERT global alert message delivery with a transmission rate of up to 250 kbit/s achieved in testing for the transmission of Earth observation products from Optical and SAR observations. The global delivery time below 1 minute for a single alert was obtained after adding the time required by the entire on-board processing chain (about 35 seconds for SAR image and ship product generation and about 20 to 40 seconds for optical image and ship product generation) to the IDRS transmission latency.
- The EO-ALERT project focuses on the definition and development of the next-generation EO data and processing chain, based on a novel flight segment architecture that moves optimized key EO data processing elements from the ground segment to on-board the satellite, with the objective of providing the EO products to the end user with very low latency (enhanced near-real time) for increased throughput. In the frame of the EO-ALERT project, the IDRS service is a very compelling option for delivering the on-board processed products to the End User globally and within seconds.
- In order to demonstrate and validate the IDRS system as a global data relay service for alerts in the frame of the EO-ALERT project, a series of tests has been performed at Graz University of Technology using a standard Inmarsat Broadband Global Area Network (BGAN) terminal, which emulates the same service as the Addvalue space transceiver. Transfer of representative alerts (EO products) has been tested and the latency and throughput measured.
- Deimos Space CEO Ismael Lopez remarked: “DEIMOS Space and the EO-ALERT consortium will showcase the full EO-ALERT concept, including Addvalue IDRS transceiver hardware and service, as part of the avionics bench testing in 2021. This will provide further confirmation of the EO-ALERT concept and solution as a solid global delivery service for real-time earth observation products, for which DEIMOS is integrating in its commercial small satellite solutions such as SAT4EO+. Including the full IDRS capabilities is a key element of the next-generation EO data and processing chain.”
- ”The tests concluded by Graz University of Technology again highlight our IDRS service as a critical enabler to accomplish sophisticated, advanced EO missions where the requirements of low latency, 24/7 on-demand, and a reliable IP data service cannot be compromised”, said Francis Low, Head of Advanced Development at Addvalue Innovation. “IDRS is indeed the sui generis data connection solution for the new space industry. We are very pleased with the outcome of these tests and confident that Deimos partners in the EO-ALERT project, DLR and OHB Italia will embrace this ground breaking concept as an integral part of their new satellite missions.”
3) M. Kerr, S. Tonetti, S. Cornara, J. I. Bravo, R. Hinz, A. Latorre, F. Membibre, A. Ramos, A. Moron, C. Solimini, S. Wiehle, H. Breit, D. Günzel, S. Mandapati, B. Tings, U. Balss, O. Koudelka, F. Teschl, E. Magli, T. Bianchi, A. Migliorati, P. Motto Ros, M. Caon, M. Martina, R. Freddi, F. Milani, G. Curci, S. Fraile, C. Marcos, ”A Novel Satellite Architecture for the Next Generation of Earth Observation Satellites Supporting Rapid Alerts,” Proceedings of the 35th Annual AIAA/USU Virtual Conference on Small Satellites, August 7-12, 2021, Logan, UT, USA, paper: SSC21-VIII-07, URL: https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=5065&context=smallsat
4) Stefania Tonetti, Stefania Cornara, Gonzalo Vicario de Miguel, Livio Carzana, Murray Kerr, Roberto Fabrizi, Silvia Fraile, Cecilia Marcos Martín, Domenico Velotto, “EO-ALERT: Next Generation Satellite Processing Chain for Security-Driven Early Warning Capacity in Maritime Surveillance and Extreme Weather Events,” ESA Living Planet Symposium, Milan, Italy, 13-17 May 2019, URL:
6) F. Membibre, et. al., “PIL testing of the Optical On-board Image Processing Solution for EO-ALERT”, ESA European Workshop on OBDP (n-Board Data Processing), 14-17, 2021, Online event, https://atpi.eventsair.com/QuickEventWebsitePortal/obdp-2021/website/
7) Antonio Latorre, Francisco Membibre, Juan Ignacio Bravo, Robert Hinz, Murray Kerr, Alexis Ramos, Aniello Fiengo, ”On-board Low Latency Processing of Earth Observation Products in a Multi-board Scheme Using Multi-core and FPGA-based Architecture,” 71st IAC 2020, TheCyberSpace Edition, URL:
8) Stefan Wiehle, Dominik Günzel, Björn Tings, ”SAR satellite on-board ship, wind, and sea state detection,” IGARSS, Brussels, Belgium, 12-14 July 2021, URL:https://igarss2021.com
9) R. Hinz, J. I. Bravo, M. Kerr, C. Marcos, A. Latorre, F. Membibre, ”EO-ALERT: Machine Learning-Based On-Board Satellite Processing for Very-Low Latency Convective Storm Nowcasting,” ECMWF-ESA Workshop on Machine Learning for Earth System Observation and Prediction, ECMWF Virtual Event, 5-8 October 2020, http://eo-alert-h2020.eu/wp-content/uploads/2020/12/Machine-Learning-WS_Hinz.pdf
10) EO-ALERT: Raoid Alerts from satellite can be deliverd in seconds, Joint EO-ALERT-AddValue Press Release, 21 January 2021, URL: http://eo-alert-h2020.eu/
The information compiled and edited in this article was provided by Herbert J. Kramer from his documentation of: ”Observation of the Earth and Its Environment: Survey of Missions and Sensors” (Springer Verlag) as well as many other sources after the publication of the 4th edition in 2002. - Comments and corrections to this article are always welcome for further updates (firstname.lastname@example.org).