For best experience please turn on javascript and use a modern browser!
You are using a browser that is no longer supported by Microsoft. Please upgrade your browser. The site may not present itself correctly if you continue browsing.
Informatics Institute Informatics Institute

New consortium develops advanced AI for surveillance robots

5 September 2024

TU Delft, University of Amsterdam, TNO and the Royal Netherlands Marechaussee are joining forces to develop artificial intelligence (AI) for surveillance robots. The OpenBots consortium, officially launched on 4 September, focuses on creating intelligent systems that can support security staff in their daily tasks.
Cees Snoek at the launch of the OpenBots consortium. Photo TNO

Cees Snoek group leader of the Video & Image Sense Lab (VIS) of the Informatics Institute is involved in this project. The UvA receives funding for 2 PhD positions.

The project uses the latest developments in generative AI, similar to the technology behind ChatGPT. Such a ‘language model’ enables computers to produce human-like texts and answer complex questions. The challenge lies in developing and applying this technology in the unpredictable public space, in which surveillance robots will operate.

Surveillance along fencelines

‘Robots can already move independently, but we still need to teach them to assess situations and react appropriately,’ explains Dr Joris Sijs, project leader from TU Delft. ‘We focus on scenarios where robots walk surveillance rounds along fences, where they have to recognise objects and people and assess what action is needed.’

The use of robots in the surveillance sector offers several advantages. They can operate 24/7, are able to perceive small details that humans may overlook, and can assess dangerous situations without human risk. Moreover, they can provide a solution to the expected staff shortage in this field.

We see to move and we move to see. We are going to make this ecological principle computable within OpenBots to allow robots to function independently in an open and unpredictable environment.

Human-robot collaboration

The consortium stresses that the aim is not to replace human security guards, but to support them. ‘We see a future where humans and robots work closely together,’ said a spokesperson for the Royal Netherlands Marechaussee. ‘Robots can take over routine tasks, allowing human security guards to concentrate on more complex situations that require human insight.’

The five-year project will appoint five PhD students who will focus the first three years on developing the AI systems. In the last two years, these systems will be tested in practice at the Royal Netherlands Marechaussee.

Consortium

Each partner in the consortium brings unique expertise. For instance, TU Delft provides knowledge in robotics and AI algorithms, UvA develops new AI algorithms for robot perception and generative machine learning. TNO contributes with their experience in applied research and innovation, and the Royal Netherlands Marechaussee provides the practical environment and expertise in security and surveillance. With this ambitious project, the OpenBots consortium hopes to make a significant contribution to the future of security and surveillance in the Netherlands.