Can video surveillance cameras be used to perform music? The answer is now yes!

Global network technology company, Axis Communications launched an experiment called “the object detection orchestra,” a musical performance played entirely on AI-powered video surveillance cameras and speakers, pushing the limits of what surveillance technology can be used for.

Pairing high-performance video surveillance cameras with AI-based analytics, Axis Communications created a new kind of instrument that reads everyday objects like tennis balls, coffee cups, and fire extinguishers as musical cues. As a predetermined item moves across the camera’s field of view, different zones trigger different notes, turning a video surveillance camera into a fully functional musical instrument.

“With this experiment, we wanted to challenge what is possible with our video surveillance cameras and AI-based analytics when it comes to detecting custom objects and triggering actions in real time,” said Ghaith Sankari, experienced software engineer, Axis Communications.

The piece performed is a reimagining of Richard Strauss’s Also Sprach Zarathustra, made iconic by its use in 2001: A Space Odyssey. But instead of strings and horns, this version uses tennis balls, a coffee cup, and a fire extinguisher.

“We could have used any objects really, even sounds can trigger actions. What you train the cameras and analytics to detect and what you make them do is totally up to you,” Sankari said.

The musical arrangement is created by Swedish music producer Jonas Quant, known for his work with Kylie Minogue, No Doubt, and Hurts.

“What fascinated me was treating object detection itself as a new kind of instrument,” Quant said. “Since we’re not using a keyboard, this experiment posed some interesting creative challenges, like teaching the musicians how to play an invisible instrument in thin air.”

The instrument works by detecting specific objects in zones assigned with a note. As the camera detects the right object, it sends an MQTT signal to a MIDI that plays a sound. Each note was represented as a square within the camera frame. Four musicians playing four cameras connected to a computer and a live mixer board were needed to perform the reimagined Also Sprach Zarathustra.

“By combining AI-based object detection, real-time MQTT signaling and integration with external systems, in this case MIDI and synthesizers, we have transformed surveillance hardware into a responsive, programmable system,” Sankari said. “It’s a compelling proof-of-concept for how intelligent network technology can unlock new business applications far beyond traditional security.”