The University of Aalborg, Denmark, has been honored for advancements in Harbor Safety Technology through a collaborative project with Milestone Systems. Specifically, video technology and synthetic datasets deployed in a Danish harbor were recently awarded the Danish Spar Nord Foundation Research Award for research into detecting fatal accidents in harbors using artificial intelligence (AI)-trained models for thermal cameras.

Milestone Systems said the research project furthers the AI capabilities of thermal cameras and Milestone’s video technology software to monitor and assess unusual behavior and enhance safety in harbors around the world.

Every year reportedly 236,000 people drown around the world. In Denmark, the harbors have witnessed numerous drowning incidents over the years, with 1,647 lives lost between 2001 and 2015, according to the announcement. A quarter of these tragedies occurred in the harbors themselves. Identifying a clear need to prevent such accidents, a research team at The University of Aalborg leveraged AI combined with video technology to enhance safety in one of Denmark’s busiest ports, Aalborg Harbor. In doing so, they created the largest outdoor thermal dataset for video analytics, covering a nine-month period.

The action of people falling into a harbor is a specific activity that cannot be easily, or ethically, replicated in the real world. Volunteers cannot be asked to fall into the water for safety reasons. Using a combination of a test-dummy filled with warm water detectable by thermal cameras and the largest published collection of annotated thermal images, AI models were trained to promptly detect and alert rescue teams if a person fell into the harbor.

The research team created an advanced synthetic dataset that could effectively train the AI models in the sudden, involuntary movements involved in falling. The training data was expanded to include wheelchair users, skaters, and bicycle riders, to provide as many scenarios of different people falling as possible for the AI model.

“By knowing the normal behavior on the waterfront, the model can detect abnormal events such as a person falling off the dock and immediately trigger an alarm,” explained lead researcher and Ph.D. student, Neelu Madan, The University of Aalborg. “This way, rescue personnel can be called out quickly — even in cases where the accident occurs without witnesses.”

As well as saving lives, Neelu Madan’s award-winning research demonstrates the power of synthetic data used to train AI models in scenarios that are difficult or unsafe to replicate in the real world. According to Madan, the same model has applications in other situations. For example, it could be used by manufacturing companies to improve production processes.

“During manufacturing, for instance, it is crucial that automated processes proceed as they should,” she said. “If not, the final product may end up with undetected errors that can be expensive in many ways, not least financially.”

Combining AI and video technology is making harbors safer and redefining safety standards across the transportation sector — ultimately saving countless lives in years to come as it is adopted on a global scale. The research, which is based on a partnership between The University of Aalborg and Milestone Systems, has demonstrated the possibilities of combining video technology software and AI to benefit the broader society.

“We are incredibly proud of Neelu and excited about the potential application of her work,” said Rahul Yadav, Chief Technology Officer, Milestone Systems. “The development of the thermal dataset by The University of Aalborg and Milestone Systems at Aalborg Harbor represents a groundbreaking step forward in video technology, providing the community with the largest annotated thermal dataset.”