How powerful are today’s Internet protocol (IP) cameras? They are literally “a computer with a lens — with the ability to do much more than just send video images,” describes Steve Carney, director, product management, Video, Tyco Security Products, Westford, Mass. Because IP cameras have improved digital signal processing (DSP) and better compression, this is enabling some analytics to be placed right on the camera, or embedded. Also, while analytics have historically required a significant amount of processing power and were a server-based application, the algorithms keep getting more efficient, requiring less processing power, says Kim Loy, vice president of global marketing and chief product officer,” DVTEL Inc., Ridgefield Park, N.J.  “As a result, analytics can be DSP or chip-based and placed on the camera or other edge device, allowing the video content analysis to be applied at the edge,” she explains.

Analytics have actually been running at the edge — for example, inside the IP camera or video encoder — for more than a decade with embedded motion detection, but advances in camera/encoder chip technology and software development have led to a range of more powerful edge applications, says Robert Muehlbauer, business development programs manager and manager of the Application Development Partner Program for North America, Axis Communications, Chelmsford, Mass. 

While the more powerful and processor-intensive analytics, such as facial recognition or custom analytics, often need to run on a server, that is changing too. “Because of current in-camera processing power, software developers may be forced to limit some functionality in their analytics so they can run safely and more accurately at the edge. However, with advances in chip technology following Moore’s Law, within the next few years almost all analytics could be run at the edge,” Muehlbauer predicts.

While certain applications still require a server-side analytics solution, today’s cameras are more capable than previous generations of handling analytics right on the camera, and manufacturers are taking advantage of it.

SightLogix Inc., Princeton, N.J., has embedded the equivalent of five servers inside its smart camera since it first started selling products in 2007, according to the company.

“This magnitude of video processing, applied to the raw video, is the only way to detect intruders outdoors without nuisance alerts or ‘mis-detects’ under all conditions and at all times,” explains John Romanowich, the company’s chief executive officer. Video analytics can become repeatable, accurate, and cost effective only when powered by a high degree of video processing embedded in the camera and directly connected to the imager, he adds.

In theory, how embedded analytics and server-based analytics function (the end result) can be identical, but they both come with unique advantages and possible disadvantages. “The decision to choose one or the other is largely dependent on the existing infrastructure of cameras and the requirements of each application,” Loy says.

“The one advantage of server-based analytics is they can be applied to existing analog cameras as well,” says Bob Germain, product leader for City of Industry, Calif.-based Hikvision USA.  “In that case, either the embedded DVR would need to support the analytic functions, or a separate server would be required.” This can add to the overall system cost and complexity, which is one of the drawbacks of server-side analytics. “With on-board analytics, there is greater efficiency in that the image required for video analysis can be ‘grabbed’ at the source,” Germian explains.

Because embedded analytics can take advantage of the raw video data as it comes off the imager, when used in conjunction with sufficient video processing, video analytics embedded inside a camera can detect with higher accuracy than analytics placed outside the camera, Romanowich believes. “When analytics are placed outside the camera, the video must be compressed for transmission over the network, resulting in an enormous loss of detail and scene information. Such video compression results in decreased accuracy and shorter camera range, and is one of the reasons that video analytics have performed so poorly in the past,” he describes.

Running embedded analytics also can save resources and costs. “Less bandwidth is consumed because only relevant video or events are transferred, less hardware is needed on the server side, and it allows for more flexible installation options where a server may not be practical to install, like at a remote power station,” Muehlbauer explains.

Special Considerations

Performance is enhanced with a few installation considerations for embedded analytics. Camera placement, lighting and other environmental factors must be taken into consideration. “There cannot be a ‘one-size-fits-all’ approach to deployment, Germain advises. “There is likely to be some tweaking of settings, but with proper training and experience, excellent results can be achieved,” he emphasizes.

Analytics-enabled cameras and general security cameras don’t get installed the same way, says Greg Tomasko, applications engineer, Honeywell Security Products Americas, Melville, N.Y. “Analytics cameras are mounted higher up and usually have a directional focus. You may also not have the greatest overview of activity. If you are looking for something specific, make sure you mount the camera to capture that,” he describes.

Like any technology, embedded analytics can come with its own set of challenges. For Mark Ring, director of Integration Services, Xentry Systems Integration, Miamisburg, Ohio, the challenges are not selling or promoting analytics but the simple everyday logistics of nature and the different areas to be monitored. “Customers know what they need, but often the challenges are trees, fixed objects or blind spots that inhibit their ability to cover an area or requirement. Much of the time, it’s not a question of adding an extra camera but whether it will look too intrusive. Each condition has its challenge, but that’s what we thrive on,” Ring says.

Over the past few years, analytics has played a key role for customers that are susceptible to terrorist threats, system loss and perimeter breach, as well as for all those who must ensure the safety of people and property, Ring details.

Garrett LeTourneau, president, Imperial Surveillance Inc., Arlington Heights, Ill., uses embedded analytics for perimeter protection, landscaping companies, scrap yards, construction sites, government facilities, shopping malls, manufacturing facilities with raw material, and more.

“If explained properly, embedded analytics sells itself,” LeTourneau says. “Preventative surveillance watches cameras for you so you can monitor the area more effectively. This increases response time, which also reduces security guard staff requirements and enables end users to enhance their profitability. It also notifies the security staff of potential issues automatically without having to watch 64 or more camera feeds,” he describes.

The top analytic requests he has are parking lots, perimeters, restricted areas, and loitering.

The best applications for embedded analytics today include areas where the customer doesn’t have a lot of human presence, according to Tomasko. “Analytics becomes more alarms than you need when there are a lot of people. It is most useful in areas where human activity isn’t as common. Those areas where you may not always see a person, but when you do, you want to know. Tampering is critical in remote areas as well. Our cameras provide the video you need or the notification that something is happening to the video you need with the anti-tampering analytics,” he says.

Those basic analytics features are a great start, and open the door to more. By placing the analytics capabilities as part of the edge device, the customer has the option to more easily get familiar with analytics and how to apply them to the business environment.

“As users begin to better understand the capabilities, they will apply them to new and innovative situations that are not limited to just a person climbing the fence to unlawfully enter a property,” says Reinier Tuinzing, strategic alliances manager for Milestone Systems Inc., Beaverton, Ore.

He lists, as examples, analytics that can be used to count the number of shoppers in line at checkout to determine when to bring in new cashiers or simply counting the number of shoppers entering through a specific door at different times of the day.

As today’s IP cameras can handle more analytics embedded right in the cameras and encoders, users will discover the even more capabilities, benefits, and new usage applications, getting even more out of their surveillance system. n

Some embedded analytics can serve as the first step in a series of actions to provide better security. For example, edge analytics can be used to perform the initial analysis of a scene (trip wire, motion detection, etc.), and then an operator can step in and perform a more detailed inspection to determine if further action or response is needed. To enable this, you need to make sure your embedded analytics link up to the video management system (VMS) that your end user uses.

“Success is more than just including new technology on the camera. The functionality to use those capabilities has to be enabled in the video management software as well,” says Milestone’s Reinier Tuinzing.

“For example, when vendors started installing edge storage on their cameras, some of them did not provide an API to access the storage from the VMS system for the central archiving and retrieval needs. Consequently the only means of getting the video data off the camera was to manually pull the SD cards from the individual cameras. Not very practical, obviously. Basically, functionality embedded in the camera — whether storage or video analytics — must also be made available in the VMS in order to work with the total information (view, save, search, export),” he says.

FLIR Commercial Systems Inc., Goleta, Calif., offers its FC series fixed thermal imagers with on-board analytics capable of detection and simple classification of vehicles and people. This solution has been utilized on projects ranging from substations, mining operations and petro-chemical facilities, to higher-end residences and commercial storage facilities. 

Matt Bretoi, vice president, North America Field Sales, Security and Surveillance, FLIR, says the benefits of thermal imaging combined with on-board analytics include:

• Providing a good useable image both day and night and in adverse conditions. This increases the reliability of the analytics, as well as visual assessment capabilities of the operator receiving the alarm.
• No lighting is required. Customers may be able to turn site lighting off or minimize it. In addition to significant cost savings, there are often light pollution restrictions in place.
• Low power and bandwidth consumption.
• Low maintenance and cost of ownership.
• Much lower false alarm rate.
• Classification-based alarms (human, vehicle).

Embedded analytics can be used to solve outdoor security challenges for correctional facilities across North America.

“A primary objective for correctional facilities is to keep people from leaving the property while simultaneously preventing contraband from entering,” describes John Romanowich, SightLogix.

One recent success story involves a detention center bordering a public area. Perpetrators were able to approach the perimeter undetected and launch contraband over the fence to insiders. Thermal SightSensors were installed to provide an automated, early warning buffer zone of security beyond the fence. Shortly after the system was running, it accurately detected perpetrators in the act of delivering contraband, alerting prison security personnel to the precise location of the crime as it was unfolding, leading to a successful interception.

Other correctional facility applications where the SightSensor is installed include rooftop security, roadside applications that detect vehicles loitering along the perimeter beyond a specific period of time, and securing sterile zones between perimeter fences as an added layer of protection. On-board image processing is used to help eliminate false detection alarms caused by wind, rain, small animals and other anomalies.

Three types of analytic solutions are host-based/server-based, embedded, and hybrid.

• Host-based or server-based analytics such as license plate capture and facial recognition require a large amount of processing power which, while easily handled by a server or NVR, could be too much for a standard camera.
• Hybrid solutions work by running lower processing-intensive analytics on the camera while hosting more processing-intensive applications at the server or NVR, allowing for better scalability.
• Embedded analytics are the most limited (today) but still add an amazing amount of value to any surveillance system. Whether the savings are in the amount of video being recorded, or in the alerts that are produced to help solve or mitigate crime, analytics are a useful tool for a surveillance user.

As with any security system installation, dealers need to be aware of what the users’ needs and requirements are. Are there security guards monitoring video 24/7? Are there areas of interest that require event alerts? Is the analytic data being utilized for marketing or other uses such as people-counting or queue times? This will help determine which type of analytics can provide the best solution. — Contributed by Wendi Burke, director of marketing, IQinVision, San Juan Capistrano, Calif.

PULLQUOTES

Tyco’s Illustra line of HD IP cameras is equipped with face detection, which finds faces in the scene and improves the quality of the image around the face while reducing the bit rate in the rest of the scene. This further reduces bandwidth and storage costs.

 Honeywell’s Active Alert can provide real-time alarms based on user-defined rules to detect abnormal or suspicious behavior without the need for human supervision. This capability enhances both manned and unmanned operations by working 24/7, reducing the amount of video data operators must review, and enabling a high level of monitoring for any size of video system.