Video surveillance is a facet of the security industry that has deployed algorithms for some time. With the rapid advancement of artificial intelligence (AI), the accuracy of that detection has grown exponentially — as has what can be detected. Ahead, industry experts delve into what AI is solving for video surveillance customers, how that AI is trained, and what differences exist between AI being deployed at the edge, in the cloud, or somewhere in between.

Motion Detection in the Past

The motion detection process traditionally revolves around pixels. “In security, people and vehicles are the two most desired objects in a scene to identify,” says Quang Trinh, business development manager, platform technologies, Axis Communications, Chelmsford, Mass. “Traditional video motion detection is based on algorithms that analyze pixel clusters. With this type of detection, the end customer is able to adjust the targeted pixel density based on what an object would fill in the scene. This approach results in some false positives attributed to pixel changes in the scene from lighting, shadows, and swaying objects in the background.

“With modern AI architectures built on deep learning, detection based on pixel clusters can now be turned into object detection and classification,” Trinh adds. Previously, the process tended toward false positives, as many sources of motion are non-human. “As AI-based object detection and classification become more efficient on edge devices like IP cameras, false positives compared to basic video motion detection are greatly reduced. The environmental factors that resulted in false positives in pixel-based video motion detection are minimized with these types of AI models, since they are built solely to detect and classify people and vehicles.”

Aaron Saks, director of sales enablement, Hanwha Vision America, Teaneck, N.J., adds, “Traditional systems used pixel-based motion, which led to a higher number of false positives and created excessive bandwidth and storage space constraints. Surveillance camera operators can easily get bombarded with too much information — and too many false event alarms — caused by lighting changes, shadows, or leaves blowing in the wind.”

These models are using similar neural networks to how the human brain would react and learn from experience. … They keep getting trained over and over and they learn from experience.

Today’s AI can be trained beyond the simplicity of registering changes in pixels.

“AI-based models improve over time as feature extractions from people and vehicles add context to the scene,” Trinh says. “An AI-based model that detects people would also look for hands, legs, upper/lower clothing color, head, face, and other features that distinguish people from other moving objects. For vehicles, sub-features such as vehicle types like bicycles, motorcycles, cars, SUVs, trucks, vans, and buses can isolate specific vehicle types that are of importance in a scene.”

AI is looking to identify problems in real time and even prevent problems from arising, but it is also deployed to increase the efficiency of sifting through video post-incident. Instead of human beings using hours and hours to sift through footage, AI can be deployed to search a video backlog in a fraction of that time.

“AI has also been deployed in what we call Smart Video Search,” says Dean Drako, CEO, Eagle Eye Networks, Austin, Texas. “As you know it can take a very long time for a business owner or security director to sift through video to find the person, object or vehicle they want to find. AI-powered smart video search works just like searching the web. You type in a search term like ‘red Honda car’ or ‘man with backpack’ and it instantly brings up the video you’re looking for.”

How the Training Works

In a way, AI learns much like we do. It is given a large data-set — or examples — of behaviors that it deems to be within a “normal range.” The AI is then introduced to countless examples of behavior that falls outside of that range, so that it can have as complete an understanding of what behavior constitutes a problem or signals that one may arise.

“Deep learning uses neural networks behind the scenes,” says Priya Serai, chief information officer, Zeus Fire and Security, Paoli, Pa. “Neural networks really are like layers and layers of algorithms that process data in a way that is very similar to the human brain. That’s why with generative AI models like chatGPT you get responses that are similar to human responses. These models are using similar neural networks to how the human brain would react and learn from experience. It’s not like you train them once and it’s done. They keep getting trained over and over and they learn from experience.”

Dave deLisser, vice president, product management, IDIS Americas, Coppell, Texas, gives the following examples from the retail market: “A deep learning model trained on footage of a retail store might, for example, recognize the natural flow of customers through and around the store,” he says. “It alerts operators accordingly when it detects abnormal behavior, such as a person loitering or a large crowd forming. This makes it far more proactive by enabling early threat detection or emergency detection, enabling security teams to act faster before losses or accidents occur.”

The benefits of automating this process are numerous. “Until the advent of deep learning, surveillance operators monitored and followed objects in real time, such as people or cars, often across multiple video streams,” deLisser says. “This led to operator fatigue and missed events, especially when viewing multiple camera feeds from busy scenes. Today, deep learning algorithms can accurately detect and categorize objects using spatial-temporal patterns in video data. This capability increases the effectiveness of surveillance systems. It also aids various tasks such as identifying suspects in criminal investigations, traffic patterns for better urban and city planning, and heatmaps and directional footfall for optimizing casino, hospitality or store layouts.”

Tim Palmquist, vice president, Americas, Milestone, Oswego, Ore., says that Milestone is rolling out a new data insights project, “What we call project Hafnia, which is our data insights product,” he says. “Data insights is the new frontier for the future of our industry because quality training data to make the AI products actually perform is what’s needed. Today we have a lot of promises but not a lot of results. As we improve the quality of the training data, then you will get much better results on the other side.”

This project was born out of the search for quality training data, Palmquist says. “The problem is their tools have been trained on generic data instead of the higher quality data of real life,” he says. “So what Milestone has proposed is what if we can get a lake of data, real life data, we can do it in responsible way so we are protecting privacy, but we’re providing excellent quality data to train AI models. Wouldn’t that help move the industry faster along?”

Continued Service From the Integrator

What about the lifecycle of the AI: How are updates handled? What role does the integrator play in this? The answer is — as it so often is — it depends.

“In most edge-based surveillance systems, no training is needed on the user side,” Saks says. “It’s all done during software development and shipped with the product. There could be updates over time to enhance the model and improve its performance, but that’s just part of the normal installation and maintenance process to keep everything up to date. Some systems allow users to do custom training on a new type of object.”

Security dealers and integrators can enhance model accuracy by regularly providing updated training data, implementing adaptive learning systems, and incorporating feedback loops where users label false positives or negatives to refine performance.

Saks continues, “There also are server-based products that can learn over time or, again, custom train on an unknown object. Other systems can add training on the fly, but it’s simply not going to be as in-depth as factory-based training because you have a much smaller data set. The best approach is following best practice guidelines of camera installation parameters.”

Trinh offers the following, “Will security dealers be able to offer this custom AI training as a service in the future? Absolutely,” he says. “There are already open-source platforms that allow security dealers to monetize and build a business from developing custom AI models. To capitalize on this opportunity, the security industry will need to develop a specialized workforce capable of tackling AI-specific tasks, but the opportunity is there.

“The opportunity is also there for vendors and security dealers to establish industry-wide guidelines around the ethical use and development of AI,” Trinh continues. “Until legislation is in place at the global, federal, and state levels, AI’s ethical and responsible use relies on self-governance.”

Hamish Dobson, corporate vice president, Avigilon & Pelco Product, Motorola Solutions, Chicago, says, “Meanwhile, some manufacturers offer means for fine tuning in the field. This allows for better accuracy over time. Both dealers and end users can typically select a number of false and true events and teach the device to do better next time.”

As for the role that the integrator plays, Satish Raj, chief technology officer at Pro-Vigil, San Antonio, Texas, says, “Security dealers and integrators can enhance model accuracy by regularly providing updated training data, implementing adaptive learning systems, and incorporating feedback loops where users label false positives or negatives to refine performance.”

The AI Advantage

To state it more explicitly, the video surveillance market is being improved in terms of accuracy and efficiency thanks to AI and deep learning. “Across many vertical sectors, customers quickly recognized the potential of AI video to improve their security posture further and tackle other operational challenges,” Dobson says. “As we continue to invest in innovative AI-enhanced solutions, we view the applications as almost limitless, particularly as the accuracy and power of AI continue to advance rapidly. From crime prevention, health and safety, regulatory compliance, improved customer service, and in-store and customer behavioral intelligence, users constantly find new uses for established analytics tools and ask for new capabilities.”

Speaking to health and safety and regulatory compliances in addition to crime prevention, Raj adds, “By automating the tracking of vehicles and people, AI reduces the need for constant manual monitoring, freeing up staff for higher-priority tasks. Many customers also seek AI surveillance for compliance and safety enforcement, such as ensuring that employees adhere to safety protocols like wearing required protective equipment in hazardous work environments.”

Looking to the future, Scott R. Elkins, CEO, Zeus Fire and Security, says, “As the manufacturers’ technology advances, there will be more AI onboard — whether it’s a camera or a network video recorder (NVR). The question that dealers will have to ask and answer — whether they’re small, medium or large, whether they’re integrators or traditional security alarm companies — is are they going to be in the business of creating unique cases that customers want relative to their specific market? I guarantee you that grocery versus retail versus health care versus schools will all have unique use cases and specific needs that won’t happen out of the box.”