Unlimited Technology Inc. (UT), a security integrator based in Chester Springs, Pa., began receiving warning notices from a customer’s site that security devices on one floor were going offline. Not only were cameras dropping out, but so were other IP-based devices. Next, the failures started cascading to devices on the floors below. The customer dispatched its building security team to investigate, who found that a broken water pipe was flooding down through the network closets on each floor and causing failures of various monitored systems due to water infiltration.

“Had our network monitoring service not been in place, the flooding would not have been discovered as quickly and damage would have been significantly worse,” says Jim DeStefano, senior vice president at Unlimited Technology, SDM’s 2021 Systems Integrator of the Year.

There are very clear benefits for both security integrators and their customers of having a service that proactively monitors the health of a network or — if not the entire network — at least the physical security devices riding on the network. With monitoring, customers can ensure their networks are operating at an optimal level to support all of the connected devices, including video surveillance cameras and intelligent access control products. Security integrators may find network health monitoring provides them with a direct recurring revenue stream. But if not, network monitoring may provide revenue indirectly through a maintenance agreement or even just in more efficient service.

IMAGE COURTESY OF UNLIMITED TECHNOLOGY INC.

‘Coupled with our Fusion Center and RST (Remote Service Technicians) response, customers enjoy quicker return to service and lower service call-outs, as many items can be diagnosed and treated remotely without the need to roll a truck to the site for onsite repairs.’

— Jim DeStefano, Unlimited Technology Inc.



A Growing Need

“I think we’re at the front edge of it right now and it’s something that’s going to continue to grow,” says Rick Fairhurst, manager of strategic partnerships at Skyline Technology Solutions, Glen Burnie, Md. “As the bandwidth increases between buildings in most of the organizations we work with, monitoring and alerting becomes more and more efficient. People will see the benefit of being able to respond to an incident much quicker.”

Among common problems detected by network monitoring, Fairhurst cites network outages, which happen frequently, as well as fiber cuts from construction mishaps. One of the more interesting network occurrences, he recalls, was with a government customer. Skyline monitors a county’s shared network used by three different entities, including the government services, the department of emergency services, and the school system. He says soon after the network was built, the school system transitioned to online testing. “All the kids in the school system logged onto online testing at the same time, grabbing all the bandwidth from the county government and the department of emergency services — which was definitely not a good thing and caused some real network communication issues,” Fairhurst says.

Within 24 hours, Skyline got the state to work with the county to increase bandwidth during the students’ testing times. This resulted in the county not having to pay phenomenal fees to expand bandwidth full-time and instead just add it temporarily through the testing period so it wouldn’t affect the rest of the network. “It sounds like it’s not a big deal, but it was huge when it affects the emergency response, so monitoring that bandwidth is really important,” Fairhurst describes.


Tangible Benefits From Network Health Monitoring

There are clear benefits to security integrators and their customers for monitoring the networks on which security systems are hosted, as well as the health of the devices themselves.

1. Reduced time for issue resolution. One huge advantage of network health monitoring is the tremendous amount of time that can be saved by being alerted the instant a device goes down.

“Typically, with many of our customers, we don’t hear about it until they put in a service ticket or call and say, ‘Hey, we have a camera down,’” says Skyline Technology Solutions’ Rick Fairhurst. “And typically, the way they found out is from the end user that noticed a camera was down. Sometimes they don’t notice it for two or three days, and sometimes even longer.

“With large entities it really can free up their resources if they’re notified in advance that something’s down. They can already get on the issue and start resolving it rather than having a request come in through the service desk and then [wait for us to] dispatch somebody. It just avoids headaches and frees up a lot of resources if they can stay on top of the outages rather than wait until they become critical and start causing problems,” Fairhurst says.

For Unlimited Technology, network monitoring, coupled with the company’s Fusion Center and Remote Service Technicians’ response, gives customers faster return to service and lowers service calls because many items can be diagnosed and treated remotely, without the need to roll a truck to the site for onsite repairs, says Jim DeStefano of Unlimited Technology. The company’s virtual global 24/7/365 Fusion Center provides a full range of services including preventive maintenance plans, master service and software support agreements, training programs, and remote diagnostics.

“In the case a service visit is required, much of the diagnosis has already been accomplished remotely and the onsite repair completed quickly,” DeStefano says.


2. Better risk management. Taking a proactive approach to monitoring your customers’ security systems can help them manage their risk better.

“If you don’t monitor your security system, you find out your cameras are not working when you’re looking for an incident and you don’t have the video,” says Peter Pavlov at Skyline Technology Solutions. “So obviously that brings you liability, for example, in a school where you have an active-assailant situation and we can’t see inside the school because the cameras are down. The risk is the response or investigation.”


3. Customer compliance with standards. When it comes to metadata going over a network, there are compliance standards that must be adhered to. Monitoring the network can help your customers be compliant.

“It benefits customers by closing the back door to criminal activity and fostering system health and IT compliance,” says Kevin Popejoy of Netronix Integration. “Every single company has their own hardened approach to that. The most important thing to understand is that this helps our customers be compliant with their own standards. Compliance, auditing, and security all go hand-in-hand.”


4. Generates recurring revenue. There is an opportunity for security integrators to have network health monitoring services become a direct revenue stream.

“There is a financial benefit to having a long-term, robust, recurring revenue model,” Popejoy believes. “This is a value to the customer that most people don’t ever want to get rid of.

“We have a responsibility more than ever before in the types of products that we align ourselves with and the types of services that we provide. Because it’s no longer just watching the bad guys on a camera or watching who is going in and out what door.

“We are now all at risk when we place things on a network that we don’t want our customers to have to go through, where they could end up in the news because customer data is compromised, their financials are compromised, and most importantly, their company’s trade secrets and identities are compromised,” Popejoy says.


Skyline Technology Solutions operates both a network engineering division and a physical security division. In most cases, network monitoring falls under the purview of the network engineering division, because that is the group that builds the customers’ networks. The company offers several different categories of network monitoring services, which have a threefold purpose: first, to provide visibility and awareness of the overall network health; second, to resolve incidents quickly to increase uptime on the network availability; and third, to provide preventative maintenance and root cause analysis to proactively mitigate potential network issues.

“It’s primarily anything that affects the quality of the network and how it operates,” Fairhurst says. “We manage each case and get to the root cause through various best practices and troubleshooting. And we’ll provide the resolution status throughout and estimated time for restoration of services.”

Peter Pavlov, principal architect for physical security at Skyline, explains that having network knowledge in-house is a key differentiator for their company. “One of our main differences to other security integrators is that we never deploy a dedicated security network. The best practice from network design is to have one network, which is logically segregated. … In most cases, our security devices sit on the general network.”


IMAGE COURTESY OF SKYLINE TECHNOLOGY SOLUTIONS

‘If you don’t monitor your security system, you pretty much find out your cameras are not working when you’re looking for an incident and you don’t have the video. So obviously that brings you liability.’

— Peter Pavlov, Skyline Technology Solutions



Monitoring Challenges

However, one drawback to that comes with monitoring, because a binary protocol only has the ability to indicate if a device is up or down. So in most cases, Skyline also relies on software provided by security industry vendors. Pavlov says most video management systems have robust self-monitoring built in, which provides more information than whether the camera is just up or down.

“Just because a camera is up doesn’t mean it’s sending you good video, or the video is being recorded,” Pavlov says. “In most cases, we rely on security software to provide us this kind of information. And that’s the reason traditional network monitoring is excluded for the security devices. If a site is offline or there are bandwidth issues at a specific location, they’ll be caught by the network monitoring. If there are issues with specific security devices, typically they’ll be caught by the security software.”


It may be some time before a network health monitoring product can broadly and effectively monitor physical security devices. In the meantime, the physical security industry continues to develop software and programs that at least can monitor the health of their video surveillance and access control products. SEVENTYFOUR/ISTOCK / GETTY IMAGES PLUS VIA GETTY IMAGES


Sometimes a larger client will monitor the network themselves and Skyline will monitor the security devices. To have a full awareness of what’s going on, you need to do both because one doesn’t tell the story of the other, he explains.

Most of Skyline’s physical security customers self-monitor, and therefore it is not a separate revenue stream to monitor the health of security devices. “We configure the software to provide reports or real-time alerts, and they either periodically check the system or receive the notifications real-time and they let us know if there’s an issue or they need help troubleshooting. We don’t really monitor the security devices real-time the way we monitor the network,” Pavlov describes. However, if alerts on the security equipment is packaged in a maintenance contract as a turnkey solution with the network monitoring, then it could be set up that way.

Unlimited Technology uses a base product for its health monitoring services, which provides a device-agnostic platform for full network protection and delivers a single dashboard for all devices. The monitoring can be tailored based on the customer’s needs.

“Most security monitoring systems may tell you something is wrong, but many are vendor-specific and unable to see the interrelationships between all end points that may be contributing to the issue, and they don’t tell you about the problem until it’s already happened,” DeStefano describes. “Our service allows you to set thresholds to keep you informed of the health and identify problems before they become critical.

UT’s service can monitor the health of any and all IP devices on a network. It can find authorized and unauthorized devices and software across a customer’s infrastructure and categorize them more reliably in an asset inventory system of servers, workstations, IP and IoT devices. It also reports network devices that are using default manufacturer passwords; collects installed software and firmware versions; creates security risk reports (with scores) and explanation of risks found; and much more.

The monitoring solution can be used by any customer looking to get all of their IP components’ health data on a single platform, and it is able to mix and match manufacturers and technologies and report on them.


Conditions That May Be Monitored

What are all of the states that a network health monitoring product can detect?

Netronix Integration’s Popejoy describes them in three categories. First and most basically, a monitoring service will check that devices are online and functioning, Popejoy says. Second, it will watch for irregularities in performance that could indicate a network or product malfunction; for example, a surge in power consumption that could spell a maintenance problem with a device. Third, it will monitor the network from a cybersecurity position — for example, to detect if a certain IP address was potentially compromised or if there was an unusual entry through a specific port.

With cybersecurity monitoring, “The easiest first place is in firmware,” Popejoy explains. “If there’s a firmware out of date, that’s going to leave a vulnerability. Obviously, it blocks password defaults from being used. And there are certain message certificates that happen on the Internet of Things, so monitoring those certificates to make sure that they’re correctly authenticated and accepted.”

Most security systems have some capacity for possessing device health or logs. “The issue with those is that everything is captured, but it’s a text file — like you’re trying to find the needle in the haystack when something actually happened and why it happened,” says Skyline Technology Solutions’ Peter Pavlov.

He likens Montreal-based manufacturer Genetec’s health monitoring tools to those on the IT side of network monitoring, because they look at both the hardware and the software of security devices. “If any component of the software is broken — if my recording engine is not working properly, it doesn’t really help me if the cameras are online. It’s the whole ecosystem, not just the edge devices,” Pavlov says.

Unlimited Technology’s DeStefano details the myriad conditions that may be found through the health monitoring product his company offers from Exero. “In addition to setting thresholds to keep you informed of system health and identifying problems before they become critical, our service also allows you to isolate anomalies; collect sensor health and contextual data to establish baselines; learn the behavior of the critical infrastructure and dynamically re-establish baselines and thresholds; listen for Windows events, syslogs, emails, and SNMP traps; react to incident alert thresholds which can be static or dynamic; enforce two-factor authentication for remote access; conduct backups to your storage, Amazon Web Services (AWS), or the Exero cloud; report file share and group permissions for compliance audits; automate OS patch management and report missing patches; update anti-virus and anti-malware signatures and report out-of-compliance devices.”

Finally, Pavlov thinks that it doesn’t matter how many events or conditions your customer’s network monitoring captures if it’s not easy to view the information and identify the issues. “Genetec does a lot with the visualization of the data, not just capturing the data. Without that, you have to send logs to the tech support team; they dig through the logs and try to figure out what happened. In Genetec, the end user can easily view that data and if they have issues, we as an integrator can interpret the data very easily,” he says.


“It can see IP video systems from different manufacturers and in one dashboard, see the health of the multiple NVRs, cameras and more, along with the status of panels, switches, UPS devices, and any other IP technology, making it an ideal all-around solution for customers,” DeStefano says.

At Unlimited Technology, networking monitoring pricing is through a recurring monthly, quarterly, or annual billing, depending upon how the customer arranges the service. Generally, the pricing is an initial one-time setup and then on a recurring, per monitored IP device, basis.

“It also provides a powerful dashboard engine to let you continually monitor mission-critical environments, set benchmarks, and view device history to isolate anomalies, identify trends, and more,” DeStefano says.

IMAGE COURTESY OF SKYLINE TECHNOLOGY SOLUTIONS

‘With large entities it can free up their resources if they’re notified in advance that something’s down. They can already start resolving it rather than having a request come in through the service desk and then come to us and then we’ll dispatch somebody.’

— Rick Fairhurst, Skyline Technology Solutions



New Business Models

When it comes to monitoring network health, Kevin Popejoy, vice president for sales for North America at Netronix Integration, San Jose, Calif., thinks most companies in the security space are only watching part of the network and not the whole network. “IT-specific network monitoring is monitoring every single thing in the entire network — every data point — no matter if it touches the security integration devices or not. In the security industry right now, there are overlays or software” for monitoring specific vendor’s devices, he says, describing how Netronix approaches the market.

Kevin Thompson, regional sales manager Western U.S. at Netronix, which is a Security-Net company, says more and more security systems manufacturers offer these types of solutions. Users can access dashboards that show them if their cameras or access control devices are online or offline and their general health. Netronix is leveraging such technology available today, but not necessarily as a monetized service offering, Thompson says.

IMAGE COURTESY OF NETRONIX INTEGRATION

‘Now that everybody leverages this whole notion of a public network, there’s no one entity that owns all of it. So it’s challenging to really know exactly what is going on across the entire network.’

— Kevin Thompson, Netronix Integration


But Popejoy sees network health monitoring growing into a bigger piece of the revenue pie, in part based on better collaboration between the security industry and the IT side of customers. “So that way we could work closer together to solve these types of problems and/or offer these types of services. Without better collaboration, it could result in a scenario where some of the customer’s systems carry some type of IT health monitoring and security companies are adding systems that have their own IT health monitoring.

But here’s the challenge: nothing’s talking to each other,” Popejoy says.

It may be some time before a network health monitoring product can broadly and effectively monitor physical security devices. In the meantime, the physical security industry continues to develop software and programs that at least can monitor the health of their video surveillance and access control products. And security integrators are approaching the issue intelligently and wisely — taking steps to learn about the need and possible solutions. The benefits of good health are always abundant.