Evaluating Camera Performance in Challenging Lighting Situations
Evaluating camera performance in challenging lighting requires much more than a specification sheet can provide.
As anyone who’s worked with IP cameras, particularly higher-megapixel models, knows, lighting challenges can pose major problems in surveillance installations. While camera performance is improving, this longstanding challenge remains today.
“Low-light is always a challenge in the security industry, especially for high-resolution megapixel cameras. It’s simple physics — you’re trying to capture light on very small pixels,” says Brad Donaldson, director of product management for Glendale, Calif.-based Arecont Vision.
In addition to low-quality images, the noise levels in low-light video tend to be increased, which has implications across the board for video surveillance.
“As light decreases, the challenge with the network and storage are pushed to the limit. An increase in noise increases storage and bandwidth requirements,” says Scott Schafer, executive vice president of sales for Arecont Vision. “Unfortunately, bad things tend to happen in the dark.”
Without technology to mitigate these worst-case scenarios, getting usable, valuable video can be difficult if not impossible.
“The challenge becomes how to extract video in those conditions,” says Kevin Saldhana, senior product manager for Pelco by Schneider Electric, Clovis, Calif. “Think about the average mug shots you see on TV from surveillance footage. They’re terribly grainy and you can barely see anything.”
However, simply applying technology to overcome challenging lighting doesn’t guarantee success. In some cases, that very technology can exacerbate many of the challenges associated with these applications. For example, automatic gain control (AGC) amplifies the signal coming in to a camera, allowing it to see clearer in lower-light situations. But this boost also comes at the cost of an increase in noise in the image.
“Where video compression is concerned, the more noise or snow equates to higher bandwidth usage. A camera with ineffective digital noise reduction or DNR can use more than four times the bandwidth and hard drive space of a camera with efficient DNR,” says Patrick Love, technology training specialist for Louisville, Ky.-based Honeywell Security Group. “Even in broad daylight without the additional noise of AGC, bit rates can often vary by up to 300 percent between manufacturers. Bandwidth should always be considered when evaluating the total cost of an IP video solution due to its stress on a network and storage requirements,” Love adds.
Another trend in the lighting space is to maintain a low-noise color image in low light rather than switching to black-and-white or changing to an IR illuminator setting. One way this is made possible is with better sensors, specifically larger sensors with more pixels. For example, says Joel White, senior product manager for IP cameras for the American Dynamics brand at Westford, Mass.-based Tyco Security Products, while ¹⁄3-inch sensors are still typical for most cameras, ½-inch sensors are making their way into the industry. While these larger sensors provide better low-light sensitivity, they aren’t ideal for many installations at the moment.
“The challenges with ½-inch sensors are that lens options are limited, larger in size and more expensive, resulting in mini-domes with a larger overall footprint. Most cameras with a ½ inch sensor are box-style cameras as they can readily accommodate the larger lens size needed,” he says.
The Numbers Game
As if working with challenging lighting conditions at a particular scene weren’t enough on its own, dealers and integrators face the added complication of trying to determine which cameras from the vast number available are worth evaluating closely to find the right match for an application. While comparing spec sheets can be a timesaver, it could actually end up being a time waster.
“It’s very difficult to rely on specs, which is a sad statement for the industry. Many manufacturers don’t put the proper specs on their cameras. They may even claim similar performance as higher-quality chipsets than they actually use,” says Matt Sailor, chief executive officer (CEO) of Pompano Beach, Fla.-based IC Realtime. “If you’re relying only on spec sheets, you might as well just throw a dart in the dark.”
The problem stems from the lack of a standard test for determining the specs listed on that sheet. Different manufacturers use different camera settings for testing.
“You can’t just look at the lux rating because it’s really often misinterpreted,” says Matt Bretoi, director of sales, eastern region and Canada for FLIR, based in Wilsonville, Ore. “Lux rating is based on shutter speed, so if the manufacturer is slowing that way down to reach a certain lux level, you’re going to get an unusable image. Are those cameras better in low light? No. I always tell people to let your eyes be the best judge.”
In some cases, the numbers reported on a spec sheet don’t even come from the camera manufacturers themselves, White says.
“Often specs are based on the camera sensor manufacturer’s claim for the sensor sensitivity range rather than whether usable surveillance video can be streamed and relied on in extreme low-light conditions,” he says. “The proof is in the video, not in the spec sheet. Our industry is so focused on paper versus testing results. A spec sheet that says it delivers an image at 0.001 lux can be subjective, as that light level is complete darkness.”
And while the majority of reported specifications are accurate based on the conditions under which they were achieved, they don’t tell the whole story, especially with regard to minimum illumination.
“A manufacturer can put any number on a spec sheet and still be technically correct. But if you slow the shutter speed by half, you increase the sensitivity almost twofold, but you also increase distortion,” says Allen Chan, senior product manager, Security Systems Division, Sony Electronics, based in Park Ridge, N.J. “That’s why it’s important to know how the manufacturer specified that minimum illumination.”
The disparity in reported specs only serves to harm the industry as a whole, Sailor says. “It really does everyone a disservice, and it happens with everything — DVRs, NVRs, sensors, etc.,” he says.
One attempt to address this lack of uniformity in determining camera specifications is UL 2082 — Performance Testing of Camera Image Quality. IC Realtime has been a supporter of this effort, and many other manufacturers are also supporting the standard.
“It’s really a barometer regulated by a non-related committee. You submit your cameras and pay for the review to get an unbiased evaluation,” Sailor says. “There are 100 tests, and they score cameras from one to 100. If it’s good, they’ll say it’s good. If it’s not, they’ll say it’s not. If the specs you’re reporting are wrong, they’ll correct them. I love the concept of an independent evaluation from an organization that doesn’t accept money or advertising.”
The main thing Sailor hopes UL 2082 will address is the lack of knowledge on the part of end users about why some cameras are more expensive than others.
“The end user doesn’t know the difference between a $39 camera they can buy online and a higher-end, higher-performance camera, but you get what you pay for,” he says. “On paper, two cameras may look identical, so why is one $100 and the other $200? You can look at the testing and see that the $100 camera gives less performance.”
So if spec sheets can be unreliable as a measuring stick, how can dealers and integrators make sure they’re getting the right camera for their situation? Seeing is believing when it comes to a camera’s performance, so decisions need to be based on hands-on evaluation or shootouts. That’s not always possible, though.
Sailor suggests putting the onus for proving camera capabilities on the manufacturer.
“I would recommend that prior to settling on a camera, integrators ask the manufacturer to give them a live online demo of the product,” Sailor says. “Put more diligence on the manufacturer to prove performance visually. It shouldn’t be difficult because we’re selling video.”
The real key is doing a proof of concept, Schafer says. “A surveillance camera is an important product installed in an important place, so you don’t want to be surprised with performance that doesn’t meet expectations,” Schafer says.
Chuck Westfall, technical advisor, professional engineering and solutions division for Canon USA, based in Melville, N.Y., suggests a number of performance indicators installers and integrators should put at the top of their lists when looking at cameras for challenging lighting situations.
“Key features to look for include exceptional image quality, outstanding low-light sensitivity, superior product reliability, and robust interoperability with other security camera system components such as video management systems,” Westfall says.
On top of everything, setting realistic expectations is perhaps the most crucial element to ensuring good camera performance in all lighting conditions.
“When you’re dealing with outdoor low-light conditions, you have to do a reality check. What are the realistic chances of possibly getting a facial identification? Are you forsaking the other benefits of surveillance for that 5 percent or so chance that you can actually get identification? If that’s what you’re designing to, you’re not getting the full benefit of surveillance,” Bretoi says. “Too often, the expectation is set by the customer that they want to get facial identification, but in most cases that’s not realistic. The image is too small or something else. You can increase the megapixels and get more pixels for the frame rate, but where’s the low-light performance?”
Saldhana encourages installers and integrators to put cameras to the test in the worst-case scenario.
“Find the toughest situation in the place you’re trying to protect, then bring in cameras and see how they perform,” he says.
In addition to evaluating different manufacturers’ cameras against one another, it’s also a good idea to evaluate cameras’ performance against themselves throughout a variety of situations. This real-world testing and careful selection will lead to better results in low-light conditions. n
Rolling with the Changes
While performance capabilities in low-light situations seems to lead the discussion of challenging lighting, there are other factors to consider, most notably varying lighting conditions within the same scene.
“It’s rarely the case that you have uniformly low light in a scene. Often there are both dark and brightly-lit areas. The human eye extracts these differences quickly, but cameras aren’t able to do this on their own,” says Kevin Saldhana at Pelco by Schneider Electric. “When there’s too much light coming into a pixel, it will saturate over into other pixels, which is called blooming, so headlights will look like minor explosions.”
A good example of this type of situation would the entrance area of a store or bank where there are large windows that will be much brighter than darker corners inside the store. Wide dynamic range (WDR) renders differently-lit areas differently, with recent advances allowing for greater image usability in these applications.
“Just as too little light is a problem, so is too much. In general surveillance situations, cameras with a higher dynamic range provide details in both bright and dark areas of an image,” says James Marcella of Axis Communications. “With increased processing power, the camera is able to apply local tone mapping as opposed to global tone mapping, resulting in an image that is more suitable for forensic review of the situation.”
How cameras adapt to changing lighting conditions — such as in an outdoor installation when the sun rises — is another variable to consider, and the speed with which these changes occur can be critical. Consider, for example, an intruder switching on the lights in a dark office at night.
“A camera’s ability to adapt to a change in lighting means the difference between a whiteout versus being able to see,” explains Scott Schafer of Arecont Vision. “So if it takes a camera two minutes to adapt to a change in lighting, someone could come in and monkey with the cables, but the camera hasn’t adapted, so you don’t see what happened.”
Even if a camera is tamper-proof, that might not make a difference in these situations.
“Depending on how long it takes the camera to adjust, someone could be in and out in the time it takes the camera to adjust,” notes Matt Bretoi of FLIR.
“With regard to how quickly a camera can adjust to changes in light, there’s no specification around that,” says Allen Chan at Sony Electronics. “With WDR, that’s where the numbers start to be all over the place. It depends on how the camera was tested, so again, it’s good to ask the manufacturer how they got these numbers,” Chan says.
The good news is that WDR technology is improving dramatically.
“Camera manufacturers have been integrating sensors with increased dynamic range, providing better performance on both sides of the lighting spectrum,” Marcella says. “Another advantage of improved processing performance is the camera’s ability to automatically shift from WDR mode into low-light mode based on the scene.”
Multiple-exposure sensors can simulate the human eye’s ability to adjust to varying lighting conditions in a scene, Saldhana says. “With a multi-exposure sensor, in any time slice, it can expose different parts of an image differently, then stitch those exposures together to provide good detail in all parts of a scene.”
Let There Be (the Right Kind of) Light
Infrared (IR) illumination has been gaining traction as a way to increase cameras’ low-light performance, with many manufacturers integrating IR illuminators into their cameras to provide this capability. However, there is also an understanding that illuminators may not be enough.
“We’re still considerably behind Europe and the United Kingdom in terms of illumination per outdoor camera installed, but we’re seeing people starting. The big camera manufacturers understand that even illumination built in to the camera has limitations in terms of distance and width,” says Mike Jamieson, CEO of Ottawa-based Raytec. “We only make lights, so we’re partnering with manufacturers of cameras and video management systems.”
The most significant challenge of IR illumination is making sure it’s being used appropriately.
“IR is good if it’s used correctly. Everyone thinks it’s all the same, but that’s a fallacy. IR comes in different flavors, sizes and powers,” says IC Realtime’s Matt Sailor. “The pick-up sensor also matters; you can have high-range IR but don’t have the sensor to pick that up. The problem is not IR technology; it’s how it’s mated with the sensor. People don’t sell based on long- , medium- and short-range IR. It’s more about high-intensity IR, which isn’t necessarily what’s needed for every application.”
Jamieson agrees that using IR incorrectly poses problems, adding that integrators should rely on manufacturers to help them determine the right lighting for their installation.
“You have to base the illumination on what the camera is seeing. You don’t want it to be wider or narrower than that, unless there’s something unusual with the installation,” he says.
Often the perception of IR is that it is capable of illuminating any and every scene, but Sailor says that’s just not the case.
“IR has always suffered over distance where there’s no reverb point to bounce back. So people will set up a camera aimed at a field and be disappointed that the IR doesn’t illuminate the scene. IR won’t bounce off air.”
One development Sailor calls the most disrupting he has seen in the illumination space is the move from ambient IR toward intelligent IR technologies.
“The problem with IR has been that a lot of the time it’s either over-saturated or under-saturated, so you either get a flashlight-looking glow or everything is a pale, ghostly white,” he says. “Intelligent IR will meter the distance to an object and adjust accordingly. For closer objects, it can dim the IR or turn it off. For something farther away, it can intensify the IR. This changes the way we see in low light.”
When selecting an illuminator, Jamieson suggests overestimating rather than underestimating the requirement.
“A 10 megapixel camera requires much more illumination than a one megapixel camera. We usually recommend going with the next size up,” he says. “You can always turn the illumination down, but once you have it at full power, you can’t go any higher.”
Of all the factors to consider, power could end up being the biggest hurdle. Jamieson says power-related questions make up about 50 to 70 percent of Raytec’s support calls.
“Usually it’s that there’s not enough power or they’re trying to extend the distance beyond what the gauge of the wire allows,” he says. “Most of our illuminators are low-power, but the larger ones require 48-150 watts. We offer a both product selector and a cable length selector that calculates what gauge of wire is needed.”