To the naked eye, it is pitch-black along the shore near a major airport located on a body of water. Suddenly the scrape of a kayak on the shore is hurriedly muffled while a figure dressed in black emerges from the craft with a shoulder-mounted weapon that with one blast could take down a jetliner. Is this a security challenge or a piece of cake?

Before the person can even take a position, lights flood the area, sirens sound and the Coast Guard arrives to take the saboteur away. With hundreds of cameras and little available light, how has such an encounter been foiled?

Active infrared cameras flooded the area with invisible infrared light and provided a picture of the figure as if it were in broad daylight. An intelligent video surveillance system detected the motion of the kayak as being in a direction that violated preset rules. Its size also alerted the system – larger vessels that frequent the area were filtered out.

Innovative video technology replaces multiple monitors with a single one that shows a three-dimensional virtual map of a site that the operator uses to access actual views from multiple cameras. Photo courtesy of Pyramid Vision
The lights and sirens were activated automatically and the location of the saboteur pinpointed to the Coast Guard as part of the rules for such a violation in the video surveillance system.

Even with hundreds of cameras at such an airport installation, intelligent video surveillance software was able to alert operators to a threat they may have missed with conventional equipment.

Although this is a hypothetical situation, such technology is available now to detect suspicious motion in a moving scene with water, to discriminate between a human figure and that of a deer, to alert security personnel and take predefined courses of action.

A small boat is identified by this video surveillance system as it approaches an area marked off near an airport’s shoreline. Photo courtesy of ObjectVideo


Many of the most innovative video surveillance installations are for government agencies and cannot be discussed for security reasons. However, the technology can be used in a variety of non-classified locations, such as in museums, construction zones, automotive car lots, fuel depot areas, chemical plants, railroad yards, prisons or schools.

For example, a visitor to an exhibition in a Dallas art museum of art objects from the Forbidden City in Beijing, China, is drawn to a huge, priceless jade sculpture. The jade glistens so irresistibly in the special exhibition’s lighting that the visitor moves closer and is drawn to touch it.

Without realizing it, the visitor has moved closer to the sculpture than the video masking on the camera that is watching the sculpture allows. An administrator in the museum’s command center is alerted to the situation by intelligent surveillance software.

The administrator radios a gallery attendant with an earpiece in the room with the sculpture and the attendant moves over and asks the visitor to step back from it.

Potential damage to the priceless object, possibly a massive insurance payout and even a geopolitical embarrassment has been averted. (See related article in “Technology at Work.”)

Another use of intelligent video systems is at the Bank of America Plaza in Dallas. The 72-story skyscraper was one of several in the United States that had areas relating to the way the building functions videotaped by an alleged terrorist, such as parking garages, street-level entrances and other sensitive locations.

After being alerted by the FBI, the building’s management hired SYSTEMSgroup, Dallas, to update the skyscraper’s video surveillance system.

It now flags such suspicious activity through the use of video masks and alerts operators to any potential security breaches, reports Michael Lagow, president and CEO of SYSTEMSgroup. The new system also archives 30 days of video to review in case of a security breach or question.

In this scene, the tree branches that are moving in the yellow boxes are considered extraneous but the activities of the person in the red box by this reservoir are subject to further examination. Photo courtesy of GE INFRASTRUCTURE, SECURITY


Video surveillance systems also are being used at Miami International Airport, where Marconi, Pittsburgh, installed a communications backbone to link all security functions including sophisticated video surveillance. Much of the installation’s details cannot be revealed, insists Jack Woskowicz, Marconi’s program director for Miami International Airport.

“What makes it all possible is that the Marconi backbone we put in here is very conducive to video, audio and data at the same time,” Woskowicz explains.

Live and recorded video along with access control is converged into one security operator work station, he points out.

Many details of another video surveillance installation at Port Everglades, Fla., also cannot be revealed, explains Tony Belotto, CPP, director of special projects for Security One Systems, Ft. Lauderdale, Fla.

More than 100 video cameras throughout the facility have certain rules or trip wires programmed into them. When one of the rules is broken or a wire is “tripped,” an alarm is set off and a live image of what is happening appears on a video monitor.

This enables officials to assess the nature of the security breach and decide on a course of action, such as dispatching a security officer.

The installation uses a virtual matrix, which is an electronic matrix for all the video cameras converted to Internet protocol (IP) and controlled by servers instead of using a hardwired matrix. This means that any camera’s image can be viewed on any monitor, Belotto notes.

The goal of such advanced video surveillance eventually is to reduce manpower and improve protection.

“If you look at the typical thing of guards watching a monitor, they’re only good for a half-hour before their eyes glaze over and they don’t see anything,” Belotto maintains. “This helps in larger installations with hundreds of cameras trying to protect something inside. When computers are watching constantly, they don’t get tired.”

Baggage left unattended in a moving scene can be flagged by intelligent video surveillance systems. Photo courtesy of ObjectVideo


The type of monitoring that can be achieved with so-called “intelligent” video surveillance software and equipment includes perimeter protection, alerts for movement in the wrong direction, suspect movements or loitering, suspicious vehicles and a host of other potential security challenges.

Because video is the only security sensor that is effectively self-validating – what you see is very well what you may be getting – its popularity in installations is on the rise.

But as the number of video security cameras at an installation proliferate, their effectiveness is diminished without full-time monitoring. Deploying additional cameras without a corresponding increase in monitoring can be a wasted expenditure.

Can anyone keep track of all 27 NFL football games when they are displayed in a sports bar?

Correspondingly, monitoring by trained operators can be expensive and sometimes ineffective if their alertness drops and they are lulled by monotony into missing an important event.

A solution thought to be more effective is employing intelligent video systems to analyze views against preset rules and alert operators when those rules are compromised. Systems such as these have been evolving over the last few years and have graduated from beta testing to full-fledged implementation.

Intelligent software also is being included in equipment like digital video recorders (DVRs). Some on the market can alert operators to human motion in the wrong direction (entering an airport exit, for example) or when a certain number is exceeded (50 employees enter a door but the 51st sets off an alert).

Some intelligent DVRs also can store and search for video that can indicate when an event occurred, such as when a suspicious vehicle first arrived or left a parking lot. These products are being designed for easy operation and installation. Putting intelligence in these devices and cameras, and eliminating the need for computer control, is one possible direction in which these products may be developed.

Image-recognition behavioral-analysis software does not treat all motion as bad – it discriminates among specific motions by persons, vehicles or other objects in certain directions. The parameters or rules of that motion that triggers an alert are programmed into the system and vary for each portion of an installation.

This software can liberate a security response team from monitor-watching and mobilize them to respond proactively to security breaches that a computer detects. Security guards checking in vehicles at a front gate can be alerted by such systems.

The operator can orient different camera views into an overall perspective with this system and respond to perimeter breaches from multiple directions. Photo courtesy of Pyramid Vision


Sophisticated, proprietary algorithms are used to describe movements that are not allowed by the systems. These movements usually can be programmed easily with user-friendly interfaces by trained operators.

In the first example of the shoreline by the airport, movements of people can be followed and only certain movements would be brought to the attention of system operators.

So a couple strolling along the beach would not set off alarms, but if they moved too close to the fence, their movements would be highlighted by having a colored box (usually red or yellow) placed around their images and the image brought up on the screen.

Extraneous movement setting off false alarms has been a problem with video surveillance in the past, when all motion was considered bad, so intelligent surveillance software is designed to reduce this.

Methods include masking out moving tree branches, flying birds or shoreline wave movement. Depending on the application, outlining the object under surveillance and separating it from its background is a method used by some systems. Other systems can “learn” a background continuously and subtract it out.

Some systems can pick black baggage out of the black flooring on which it is placed, which might be lost by some CCTV systems.

Intelligent surveillance systems can flag vehicles that move in the wrong direction or stay too long in one place. Some of these systems can distinguish between different types of vehicles and alert operators to human movement from vehicle to vehicle, which might suggest theft is being considered, or other erratic behavior.

One system replaces multiple monitors with a single one that shows a three-dimensional virtual map of a site. The operator uses a joystick to fly over the map and access actual views from multiple cameras. This enables the operator to orient the different camera views into an overall perspective and respond to perimeter breaches from multiple directions.

Other capabilities include the ability of some cameras to inspect the entire field of vision instead of just the center and to process images while they are moving. This capability is important in installations that use PTZs or where natural forces such as high winds buffet cameras.

The technology is available to read the numbers on containers at ports from cameras on the cranes unloading them or to identify 12-inch numbers on ships at night in fog by beaming active infrared light for 5 kilometers.

A customer leaving this retail establishment can be highlighted for additional analysis. Photo courtesy of GE INFRASTRUCTURE, SECURITY


Licensing arrangements are making the image-recognition behavioral-analysis software of one company available on the hardware of video system suppliers. This can place the licensee’s image-recognition software on the same server as the video retrieval software and reduce the number of servers required by one.

Using off-the-shelf computer devices rather than requiring proprietary hardware can reduce a system's cost.

Video retrieval software can tag events and search and sort by the type of event, such as a car going too fast in one direction or a person standing in a lobby for more than two minutes.

Geography is not a limiting factor with some surveillance products. Cameras throughout the United States can be monitored from a central facility.

Small systems down to four cameras also can use image-recognition software.

Redundancy is built into some systems for the power supply, operating system and memory storage buffering. Systems that do not require this can be priced more economically.

Building management systems also are including image-recognition behavioral-analysis software so all a structure’s functions – HVAC, security, CCTV and more – can be accessed from one interface.

Many image-recognition software systems can operate with a variety of input, such as analog, infrared, thermal, monochrome, color or even previously recorded video by converting it to digital for pixel analysis.

“It’s up-and-coming technology, and once they see it, they’ll be impressed with it – it’s the next evolution of a video system,” Belotto of Security One Systems concludes.

Shane McClelland, director of marketing and market manager for Marconi, agrees.

“The trend is starting just now and will migrate into more of the business environment,” he predicts. “That transition is definitely happening, and I think it’s going to happen fairly quickly.”

Security dealers, system integrators and their customers need to study the variety of image-recognition behavioral-analysis systems available for the specific features and ease of operation they offer.

Selection of systems may vary based on each installation’s requirements. Intelligent video surveillance is not a one-size-fits-all application. The quality of operator training offered by a system also needs to be considered in the purchase decision.