Machine-learning system can detect dark vessels faster

A newly developed Artificial Intelligence (AI) technology, built with ESA, S[&]T, Vake and EmLogic in Norway, will be used for marine intelligence, making it possible to locate ships in hours instead of days. The system, which will be up and running almost immediately after launch, will be used mostly  to identify illegal ships but could also be used in future for applications such as  preventing collisions and redirecting routes in real time.

The project has adapted an existing machine learning ship detection process which takes photos from existing satellites sent to a ground station and allows users to filter the ones containing a boat. This method, of downlinking images, processing them in traditional image processing pipelines and making them available to end users is not done in realtime. It can take hours, even days, to download them. Once the end users have the image, they can run on ground ship detection algorithms on the data but it is likely the ships will have already moved on meaning the data has limited usefulness for realtime applications.

The new ship-detection system, called REMIS, works on the principle that a machine learning AI software onboard could pre-filter the images to prioritise sending those that include a ship, reducing the number of images being sent. Fewer images means less data heavy downlinks,  less time spent processing the images on ground and faster overall detection times for the ships. The AI model was developed by Vake, Norway. The activity was one of three that came out of a push from the Norwegian Space Agency (NOSA). All three were a direct result of a NOSA competition in 2020 for payloads on an ESA In Orbit Demonstration and collaboration with Norway’s ESA Business Incubation Centre.

“Ship detection is a well-established AI task, many neural networks are already trained to perform it so there is lots of publicly available data for training the network and companies use AI on ground to post process data as a service,” explains Maris Tali, the technical officer in charge of the activity. “In addition to the neural network REMIS also developed the on-board data pre-processing so we are able to test the whole system in a prototype hardware implementation.”

“If we can already identify ships on-board and potentially even combine it with Artificial Intelligence Systems we could already mark any dark vessels and prioritise the scenes containing ships or their locations,” she continues.

One of the scenarios where the system could be used is to detect so-called dark vessels, helping governments to protect their coastal waters from for example illegal fishing. The on-board implementation would greatly improve the time between the ship being detected and the authorities being alerted. 

Neural networks in ground applications can be big and complex. One challenge for the activity was finding a simple enough neural network capable of performing the interference efficiently while also being suitable for use in space. “Even the very simple neural network eventually used performed quite well compared to networks on ground,” says Tali.

Where the images showed a fairly straightforward similar-coloured background, objects like a ship, especially large ones like a tanker, stand out. In these cases the neural network performed very well. The processing was slightly less reliable where coastal areas were introduced, as things like offshore windmills and oil platforms all show up as small objects.

“It’s possible to map the coast and block it out, but since the AI errs on the side of detecting too many things, so it would be down to post-processing where a classification neural network could be used to filter the images once more,” Tali details. “Currently, the on ground processing for ship detection often includes some attempt to classify the ship type but on board a small cubesat demonstration mission we would not have the required processing capabilities or  power budget to make this feasible.”

The neural network was developed with an In Orbit Demonstration mission in mind. Most likely the demonstration missions would be a small cubesat with limited resources. Increasing the efficiency of the processing and limiting downlink bandwidth is vital for these small missions. REMIS was built for these missions specifically. By using an small neural network the activity would reduce the resources needed, since you could launch with a neural network on board trained to the images you expect from the hyperspectral camera hardware used. Although the network would immediately be operational, it would be foreseen to retrain the network with real images to improve the performance. Instead of reuploading the entire network, it would be possible to only send a small update to the network.

The idea for the activity stemmed from the 2022 GSTP compendia, which addressed key strategic technologies in Digitalisation, Quantum technology, Cybersecurity and Artificial Intelligence. The compendia covered 21 activities in AI, ranging from guidance navigation and control (GNC) and on-the-edge artificial intelligence to  autonomous mission operations. Artificial Intelligence offers a new approach to engineering that can replace many traditional software tools, substituting the traditional software coding with algorithms that mimic human brain processes and can also enable new applications at the moment not covered by any software tool, e.g. clustering, automation of process requiring decision making, etc. AI is advantageous in complementing model-based techniques with data-driven methods, providing integrated onboard machine learning and enhanced engineering tools for improved efficiency. 

Edited on 28/10 to add input on Vake and Norwegian Space Agency.