Swiss-Japanese Innovation: Pioneering Space Debris Mitigation with AI and Cutting-Edge Camera Technology
Key takeaways
Due to severe Covid restrictions, it was unfortunately impossible to travel personally to Japan during the project. Nevertheless, two technical workshops for the exchange of information in preparation for larger projects were organised, leading to very intense technical in-depth discussions. While the first workshop was focused on the “Selection of COTS FPGA for Space Application”, the second workshop was oriented on AI- and ML-algorithmic requirements for target recognition and debris tracking. Despite the 10’000 km separating the project partners, the two workshops turned out to be very intense and productive, leading to the first full design of the HW/SW partitioning of the AI- and ML algorithms, building a basis for the next development steps. This was a very fruitful and memorable experience.
Due to severe Covid restrictions, it was unfortunately impossible to travel personally to Japan during the project. Nevertheless, two technical workshops for the exchange of information in preparation for larger projects were organised, leading to very intense technical in-depth discussions. While the first workshop was focused on the “Selection of COTS FPGA for Space Application”, the second workshop was oriented on AI- and ML-algorithmic requirements for target recognition and debris tracking. Despite the 10’000 km separating the project partners, the two workshops turned out to be very intense and productive, leading to the first full design of the HW/SW partitioning of the AI- and ML algorithms, building a basis for the next development steps. This was a very fruitful and memorable experience.
The Competence Centre Intelligent Sensors and Networks (CC ISN) at HSLU has an outstanding R&D record and knowledge in intelligent video analysis and vision sensor technologies for various applications in both computer vision and industry. For addressing the space debris issue, visual guidance using image processing is being considered an effective method to guide debris-removal satellites toward unresponsive targets. For example, artificial intelligence (AI), machine learning (ML) and image processing techniques can be used to estimate the distance between a debris-removal satellite and the items of debris and to estimate the debris movement. In cooperation with the Zürich-based company Sirin Orbital Systems AG – which specialised in innovative space technologies, HSLU’s efforts are concentrated on the development of a smart processor board that provides the possibility for real-time detection and classification of space debris based on video image streams from a 2D matrix camera. The Japanese company Nikon Corporation, has excellent know-how and technologies for developing terrestrial cameras and has an intense interest in utilising such technologies in space. In particular, the quality of Nikon optics has a great reputation. Thereby, in order to achieve a final reliable product for the end-users and its commercialisation, a Swiss-Japanese consortium was established in the frame of the “Innovation Partnership Grant 2020”, thanks to the Leading House Asia’s precious support and encouragement.
This research cooperation is shaped around the skills of each Swiss and Japanese consortium member and their common goal to develop a market-ready product addressing the space debris issue. The consortium’s long-term goal is the development of a low-cost, high-performance intelligent space camera for guidance and navigation systems of Active Debris Removal (ADR) satellites by effective utilisation of commercial off-the-shelf (COTS) components that will enable reliable mitigation of space debris. This high-level object has secured great international cooperation, uniting Lucerne University of Applied Sciences and Arts (HSLU) and Nikon Corporation in the first place, extended by Tokyo University of Science’s Research Center for Space System Innovation and Sirin Orbital System AG. The consortium has initiated a project [in the fundraising phase], and the role of each partner is shown in the picture above.
Due to severe Covid restrictions, it was unfortunately impossible to travel personally to Japan during the project. Nevertheless, two technical workshops for the exchange of information in preparation for larger projects were organised, leading to very intense technical in-depth discussions. While the first workshop was focused on the “Selection of COTS FPGA for Space Application”, the second workshop was oriented on AI- and ML-algorithmic requirements for target recognition and debris tracking. Despite the 10’000 km separating the project partners, the two workshops turned out to be very intense and productive, leading to the first full design of the HW/SW partitioning of the AI- and ML algorithms, building a basis for the next development steps. This was a very fruitful and memorable experience.
Leading House Asia’s funding instrument provided a unique platform for the establishment of initial contacts between the Swiss and Japanese partners. The “Innovation Partnership Grant” enables us to prepare a solid basis for a long-term project through the exchange of Swiss and Japanese technology know-how at excellent levels and the preparation of joint proposals for submission to innovation-related funding agents, including the European Space Agency (ESA) grants. In the modern space era, problems associated with space debris have become more serious as space development has become more active. Furthermore, because collisions among existing debris generate new items of debris, the volume of space debris is projected to increase exponentially, even if no new satellites are launched. If these fragments collide with other active satellites or space infrastructures, serious problems will arise. Thus, the management of spacecraft activity in orbit is difficult without debris surveillance. Consequently, satellites for debris removal must be developed immediately. Owing to such a critical situation, not only the governmental public approaches but also commercial private approaches have emerged. An imperative function of a debris-removal satellite is to autonomously recognise and approach the target debris by sensing its distance and movement.
In the frame of “Innovation Partnership Grants 2020”, the Swiss and Japanese teams succeeded in establishing a fundamental research collaboration addressing one of the most challenging issues upfront, the future of sustainable space exploration, namely space debris peril. This grant allowed the establishment of initial contacts between the Swiss and Japanese partners, which yielded two technical workshops for the exchange of information in preparation for larger projects. The first workshop was focused on the “Selection of COTS FPGA for Space Application”; while the full development of AI algorithms for visual guidance camera system and their implementation on the selected COTS FPGA is taking place in Switzerland, we are learning from our Japanese partner’s experiences in diverse Active Debris Removal (ADR) missions, including the most recent one, ELSA-d of Astroscale, where Nikon provided a tracking camera. The second workshop was oriented on AI- and ML-algorithmic requirements for target recognition and debris tracking, whereas the first workshop was more focused on the choice of COTS.
Despite the problems caused by the COVID crisis, especially in the context of a project focused on exchange, the overall experience was very positive. This was mainly due to the fact that the technical competencies in the team were very complementary but nevertheless synergetically very well united as a whole. All project partners gave a clear signal for further intensive cooperation, which was already initiated in the form of additional project applications.
