The Case for Flexible Data Routing in Modern Data Management

Every second, billions of connected devices quietly monitor the pulse of the physical world: measuring pressure in refineries, tracking vibrations on turbine blades, adjusting the temperature of precision manufacturing lines, counting cars at intersections, and watching valves that regulate clean water. This is the telemetry that keeps our world running. It is also increasingly what’s putting the world at risk.

Why is OT telemetry becoming a cybersecurity priority?

In 2021, attackers tried to poison a water plant in Oldsmar, Florida, by changing chemical levels. In 2022, ransomware actors breached Tata Power in India, exfiltrating operational data and disrupting key functions. These weren’t IT breaches – they targeted operational technology (OT): the systems where the digital meets the physical. When compromised, they can halt production, damage equipment, or endanger lives.

Despite this growing risk, the telemetry from these systems – the rich, continuous streams of data describing what’s happening in the real world – aren’t entering enterprise-grade security and analytics tools such as SIEMs.

What makes OT telemetry data so hard to integrate into security tools?

For decades, OT telemetry was designed for control, not correlation. Its data is continuous, dense, and expensive to store – the exact opposite of the discrete, event-based logs that SIEMs and observability tools were built for. This mismatch created an architectural blind spot: the systems that track our physical world can’t speak the same language as the systems that secure our digital one. Today, as plants and utilities connect to the cloud, that divide has become a liability.  

OT Telemetry is Different by Design

Security teams managed discrete events – a log, an edit to a file, an alert. OT telemetry reflects continuous signals – temperature, torque, flow, vibrations, cycles. Traditional security logs are timestamped records of what happened. OT data describes what’s happening, sampled dozens or even thousands of times per minute. This creates three critical mismatches in OT and IT telemetry data:

• Format: Continuous numeric data doesn’t fit text-based log schemas
• Purpose: OT telemetry optimizes continuing performance while security telemetry is used to flag anomalies and detect threats
• Economics: SIEMs and analytics tools charge on the basis of ingestion. Continuous data floods these models, turning visibility into runaway cost.

This is why most enterprises either down-sample OT data or skip it entirely; and why most SIEMs don’t have the capacity to ingest OT data out of the box.

Why does this increase risk?

Without unified telemetry, security teams only see fragments of their operational truth. Silent sources or anomalous readings might seem harmless to OT engineers but might signal malicious interference; but that clue needs to be seen and investigated with SOCs to uncover the truth. Each uncollected and unanalyzed bit of data widens the gap between what has happened, what is happening, and what could happen in the future. In our increasingly connected and networked enterprises, that’s where risk lies.

From isolation to integration: bridging the gap

For decades, OT systems operated in isolated environments – air-gapped networks, proprietary closed-loop control systems, and field devices that only speak to their own kind. However, as enterprises sought real-time visibility and data-driven optimization, operational systems started getting linked to enterprise networks and cloud platforms. Plants started streaming production metrics to dashboards; energy firms connected sensors to predictive maintenance systems, and industrial vendors began managing equipment remotely.  

The result: enormous gains in efficiency – and a sudden explosion of exposure.

Attackers can now reach into building control systems inside manufacturing facilities, power plants, and supply chain networks to reach what was once unreachable. Suddenly, a misconfigured VPN or a vulnerability in middleware systems that connect OT to IT systems (current consensus suggests this is what exposed the JLR systems in the recent hack) could become an attacker’s entry point into core operations.

Why is telemetry still a cost center and not a value stream?

For many CISOs, CIOs, and CTOs, OT telemetry remains a budget line item – something to collect sparingly because of the cost of ingesting and storing it, especially in their favorite security tools and systems built over years of operations. But this misses the larger shift underway.

This data is no longer about just monitoring machines – it’s about protecting business continuity and understanding operational risk. The same telemetry that can predict a failing compressor can also help security teams catch and track a cyber intrusion.  

Organizations that treat this data and its security management purely as a compliance expense will always be reactive; those that see this as a strategic dataset – feeding security, reliability, and AI-driven optimization – will turn it into a competitive advantage.

AI as a catalyst: turning telemetry into value

AI has always been most effective when it’s fed by diverse, high-quality data. This is the mindset with which the modern security team treated data, but ingestion-based pricing made them allergic to collecting OT telemetry at scale. But this same mindset is now reaching operational systems, and leading organizations around the world are treating IoT and OT telemetry as strategic data sources for AI-driven security, optimization, and resilience.

AI thrives on context, and no data source offers more context than telemetry that connects the digital and physical worlds. Patterns in OT data can reveal early indications of faltering equipment, sub-optimal logistical choices, and resource allocation signals that can help the enterprise save. It can also provide early indication of attack and defray significant business continuity and operational safety risk.

But for most enterprises, this value is still locked behind scale, complexity, and gaps in their existing systems and tools. Collecting, normalizing, and routing billions of telemetry signals from globally distributed sites is challenging to build manually. Existing tools to solve these problems (SIEM collectors, log forwarders) aren’t built for these data types and still require extensive effort to repurpose.  

This is where Agentic AI can become transformative. Rather than analyzing data downstream after extensive tooling to manage data, AI can be harnessed to manage and govern telemetry from the point of ingestion.

• Automatically detect new data formats or schema drifts, and generate parsers in minutes on the fly
• Recognize patterns of redundancy and noise and recommend filtering or forking of data by security relevance to store everything while analyzing only that data which matters
• Enforce data governance policies in real time – routing sensitive telemetry to compliant destinations
• Learn from historical behavior to predict which signals are security-relevant versus purely operational

The result is a system that scales not by collecting less, but by collecting everything and routing intelligently. AI is not just the reason to collect more telemetry – it is also the means to make that data valuable and sustainable at scale.

Case Study: Turning 80 sites of OT chaos into connected intelligence

A global energy producer operating more than 80 distributed industrial sites faced the same challenge shared by many manufacturers: limited bandwidth, siloed OT networks, and inconsistent data formats. Each site generates between a few gigabytes to hundreds of gigabytes of log data daily – a mix of access control logs, process telemetry, and infrastructure events. Only a fraction of this data reached their security operations center. The rest stayed on-premise, trapped in local systems that couldn’t easily integrate with their SIEM or data lake. This created blind spots and with recent compliance developments in their region, they needed to integrate this into their security architecture.

The organization decided to re-architect their telemetry layer around a modular, pipeline-first approach. After an evaluation process, they chose Databahn as their vendor to accomplish this. They deployed Databahn’s collectors at the edge, capable of compressing and filtering data locally before securely transmitting it to centralized storage and security tools.

With bandwidth and network availability varying dramatically across sites, edge intelligence became critical. The collectors automatically prioritized security-relevant data for streaming, compressing non-relevant telemetry for slower transmission to conserve network capacity when needed. When a new physical security system needed to be onboarded – one with no existing connectors – an AI-assisted parser system was built in a few days, not months. This agility helped the team reduce their backlog of pending log sources and immediately increase their visibility across their OT environments.

In parallel, they used policy-driven routing to send filtered telemetry not only to their security tools, but also to the organization’s data lake – enabling business and engineering teams to analyze the same data for operational insights.

The outcome?

• Improved visibility across all their sites in a few weeks
• Data volume to their SIEM dropped to 60% despite increased coverage, due to intelligent reduction and compression
• New source of centralized and continuous intelligence established for multiple functional teams to analyze and understand

This is the power of treating telemetry as a strategic asset: and of using the pipeline as the control plane to ensure that the increased coverage and visibility don’t come at the cost of security posture or by destroying the IT/Security budget.

Continuous data, continuous resilience, continuous value

The convergence of IT and OT has and will continue to represent an increase in the attack surface and the vulnerability of digital systems deeply connected to physical reality. For factories and manufacturers like Jaguar Land Rover, this is about protecting their systems from ransomware actors. For power manufacturers and utilities distributors, it could mean the difference between life and death for their business, employees, and citizens with major national security implications.  

To meet this increased risk threshold, telemetry must become the connective tissue of resilience. It must be more closely watched, more deeply understood, and more intelligently managed. Its value must be gauged as early as possible, and its volume must be routed intelligently to sanctify detection and analytics equipment while retaining the underlying data for bulk analysis.

The next decade of enterprise security and AI will depend upon how effectively organizations bridge this divide from the present into the ideal future. The systems that today are being kept out of SIEMs to stop them from flooding will need to fuel your AI. The telemetry from isolated networks will have to be connected to power real-time visibility across your enterprise.

The world will run on this data – and so should the security of your organization.

Security teams have long relied on an endless array of SIEM and business intelligence (BI) dashboards to monitor threats. Yet for many CISOs and SOC leads, the promise of “more dashboards = more visibility” has broken down. Analysts hop between dozens of charts and log views trying to connect the dots, but critical signals still slip past. Enterprises ingest petabytes of logs, alerts, and telemetry, yet typically analyze less than 5% of it, meaning the vast majority of data (and potential clues) goes untouched.

The outcome? Valuable answers get buried in billions of events, and teams waste hours hunting for insights that should be seconds away. In fact, one study found that as much as 25% of a security analyst’s time is spent chasing false positives (essentially investigating noisy, bogus alerts). Security teams don’t need more dashboards – they need security insights.  

The core issue is context.

Traditional dashboards are static and siloed; each tells only part of the story. One dashboard might display network alerts, another shows user activity, and another displays cloud logs. It’s on the human analyst to mentally fuse these streams, which just doesn’t scale. Data is scattered across tools and formats, creating fragmented information that inflates costs and slows down decision-making. (In fact, the average enterprise juggles 83 different security tools from 29 vendors, leading to enormous complexity.) Meanwhile, threats are getting faster and more automated – for example, attackers have reduced the average time to complete a ransomware attack in recent years far outpacing a human-only defense. Every minute spent swiveling between dashboards is a minute an adversary gains in your environment.  

Dashboards still provide valuable visibility, but they were never designed to diagnose problems. It isn’t about replacing dashboards, it’s about filling the critical gap by surfacing context, spotting anomalies, and fetching the right data when deeper investigation is needed.

To keep pace, security operations must evolve from dashboard dependency to automated insight. That’s precisely the shift driving Databahn’s Reef.

The Solution: Real-Time, Contextual Security Insights with Reef  

Reef is Databahn’s AI-powered insight layer that transforms high-volume telemetry into actionable intelligence the moment it needs. Instead of forcing analysts to query multiple consoles, Reef delivers conversational, generative, and context-aware insights through a simple natural language interface.

In practice, a security analyst or CISO can simply ask a question or describe a problem in plain language and receive a direct, enriched answer drawn from all their logs and alerts. No more combing through SQL or waiting for a SIEM query to finish – what used to take 15–60 minutes now takes seconds.

Reef does not replace static dashboards. Instead, it complements them by acting as a proactive insight layer across enterprise security data. Dashboards show what’s happening; Reef explains why it’s happening, highlights what looks unusual, and automatically pulls the right context from multiple data sources.

Unlike passive data lakes or “swamps” where logs sit idle, Reef is where the signal lives. It continuously filters billions of events to surface clear insights in real time. Crucially, Reef’s answers are context-aware and enriched. Ask about a suspicious login, and you won’t just get a timestamp — you’ll get the user’s details, the host’s risk profile, recent related alerts, and even recommended next steps. This is possible because Reef feeds unified, cross-domain data into a Generative AI engine that has been trained to recognize patterns and correlations that an analyst might miss. The days of pivoting through 6–7 different tools to investigate an incident are over; Reef auto-connects the dots that humans used to stitch together manually.

Under the Hood: Model Context Protocol and Cruz AI

Two innovations power Reef’s intelligence: Model Context Protocol (MCP) and Cruz AI.

• MCP keeps the AI grounded. It dynamically injects enterprise-specific context into the reasoning process, ensuring responses are factual, relevant, and real-time – not generic guesses. MCP acts as middleware between your data fabric and the GenAI model.
• Cruz AI is Reef’s autonomous agent – a tireless virtual security data engineer. When prompted, Cruz fetches logs, parses configurations, and automatically triages anomalies. What once required hours of analyst effort now happens in seconds.

Together, MCP and Cruz empower Reef to move beyond alerts. Reef not only tells you what happened but also why and what to do next. Analysts effectively gain a 24/7 AI copilot that instantly connects dots across terabytes of data.    

Why It Matters  

Positioning Reef as a replacement for dashboards is misleading — dashboards still have a role. The real shift is that analysts no longer need to rely on dashboards to detect when something is wrong. Reef shortens that entire cycle by proactively surfacing anomalies, context, and historical patterns, then fetching deeper details automatically.

• Blazing-Fast Time to Insight: Speed is everything during a security incident. By eliminating slow queries and manual cross-referencing, Reef delivers answers up to 120× faster than traditional methods. Searches that once took an analyst 15–60 minutes now resolve in seconds.  
  ‍
• Reduced Analyst Workload: Reef lightens the load on your human talent by automating the grunt work. It can cut 99% of the querying and analysis time required for investigations. Instead of combing through raw logs or maintaining brittle SIEM dashboards, analysts get high-fidelity answers handed to them instantly. This frees them to focus on higher-value activities and helps prevent burnout.  
  ‍
• Accelerated Threat Detection: By correlating signals across formerly isolated sources, Reef spots complex attack patterns that siloed dashboards would likely miss. Behavioral anomalies that span network, endpoint, and cloud layers can be baselined and identified in tandem. The outcome is significantly faster threat detection – Databahn estimates up to 3× faster – through cross-domain pattern analysis.
  ‍
• Unified “Single Source of Truth”: Reef provides a single understanding layer for security data, ending the fragmentation and context gaps. All your logs and alerts – from on-premise systems to multiple clouds – are normalized into one contextual view. This unified context closes investigation gaps; there’s far less chance a critical clue will sit forgotten in some corner of a dashboard that nobody checked. Analysts no longer need to merge data from disparate tools or consoles mentally; Reef’s insight feed already presents the whole picture.  
  ‍
• Clear Root Cause & Lower MTTR: Because Reef delivers answers with rich context, understanding the root cause of an incident becomes much easier. Whether it’s pinpointing the exact compromised account or identifying which misconfiguration allowed an attacker in, the insight layer lays out the chain of events clearly. Teams can accelerate root-cause analysis with instant access to all log history and the relevant context surrounding an event. This leads to a significantly reduced Mean Time to Response (MTTR). When you can identify, confirm, and act on the cause of an incident in minutes instead of days, you not only resolve issues faster but also limit the damage.    

The Bigger Picture  

An insight-driven SOC is more than just faster – it’s smarter.  

• For CISOs: Better risk outcomes and higher ROI on data investments.  
• For SOC managers: Relief from constant firefighting and alert fatigue.
• For front-line engineers: Freedom from repetitive querying, with more time for creative problem-solving.  

In an industry battling tool sprawl, analyst attrition, and escalating threats, Reef offers a way forward: automation that delivers clarity instead of clutter.  

The era of being “data rich but insight poor” is ending. Dashboards will always play a role in visibility, but they cannot keep pace with AI-driven attackers. Reef ensures analysts no longer depend on dashboards to detect anomalies — it delivers context, correlation, and investigation-ready insights automatically.

Databahn’s Reef represents this next chapter – an insight layer that turns mountains of telemetry into clear, contextual intelligence in real time. By fusing big data with GenAI-driven context, Reef enables security teams to move from reactive monitoring to proactive decision-making.  

From dashboards to decisions: it’s more than a slogan; it’s the new reality for high-performing security organizations. Those who embrace it will cut response times, close investigation gaps, and strengthen their posture. Those who don’t will remain stuck in dashboard fatigue.  

See Reef in Action:  

Ready to transform your security team operations? Schedule a demo to watch conversational analytics and automated insights tackle real-world data.

We highlighted how detection and compliance break down when data isn’t reliable, timely, or complete. This second piece builds on that idea by looking at the work behind the pipelines themselves — the data engineering automation that keeps security data flowing.

Enterprise security teams are spending over 50% of their time on data engineering tasks such as fixing parsers, maintaining connectors, and troubleshooting schema drift. These repetitive tasks might seem routine, but they quietly decide how scalable and resilient your security operations can be.

The problem here is twofold. First, scaling data engineering operations demands more effort, resources, and cost. Second, as log volumes grow, and new sources appear, every manual fix adds friction. Pipelines become fragile, alerting slows, and analysts lose valuable time dealing with data issues instead of threats. What starts as maintenance quickly turns into a barrier to operational speed and consistency.

Data Engineering Automation changes that. By applying intelligence and autonomy to the data layer, it removes much of the manual overhead that limits scale and slows response. The outcome is cleaner, faster, and more consistent data that strengthens every layer of security.

As we continue our Cybersecurity Awareness Month 2025 series, it’s time to widen the lens from awareness of threats to awareness of how well your data is engineered to defend against them.

The Hidden Cost of Manual Data Engineering

Manual data engineering has become one of the most persistent drains on modern security operations. What was once a background task has turned into a constant source of friction that limits how effectively teams can detect, respond, and ensure compliance.

When pipelines depend on human intervention, small changes ripple across the stack. A single schema update or parser adjustment can break transformations downstream, leading to missing fields, inconsistent enrichment, or duplicate alerts. These issues often appear as performance or visibility gaps, but the real cause lies upstream in the pipelines themselves.

The impact is both operational and financial:

• Fragile data flows: Every manual fix introduces the risk of breaking something else downstream.
• Wasted engineering bandwidth: Time spent troubleshooting ingest or parser issues takes away from improving detections or threat coverage.
• Hidden inefficiencies: Redundant or unfiltered data continues flowing into SIEM and observability platforms, driving up storage and compute costs without adding value.
• Slower response times: Each break in the pipeline delays investigation and reduces visibility when it matters most.

The result is a system that seems to scale but does so inefficiently, demanding more effort and cost with each new data source. Solving this requires rethinking how data engineering itself is done — replacing constant human oversight with systems that can manage, adapt, and optimize data flows on their own. This is where Automated Data Engineering begins to matter.

What Automated Data Engineering Really Means

Automated Data Engineering is not about replacing scripts with workflows. It is about building systems that understand and act on data the way an engineer would, continuously and intelligently, without waiting for a ticket to be filed.

At its core, it means pipelines that can prepare, transform, and deliver security data automatically. They can detect when schemas drift, adjust parsing rules, and ensure consistent normalization across destinations. They can also route events based on context, applying enrichment or governance policies in real time. The goal is to move from reactive maintenance to proactive data readiness.

This shift also marks the beginning of Agentic AI in data operations. Unlike traditional automation, which executes predefined steps, agentic systems learn from patterns, anticipate issues, and make informed decisions. They monitor data flows, repair broken logic, and validate outputs, tasks that once required constant human oversight.

For security teams, this is not just an efficiency upgrade. It represents a step change in reliability. When pipelines can manage themselves, analysts can finally trust that the data driving their alerts, detections, and reports is complete, consistent, and current.

How Agentic AI Turns Automation into Autonomy

Most security data pipelines still operate on a simple rule: do exactly what they are told. When a schema changes or a field disappears, the pipeline fails quietly until an engineer notices. The fix might involve rewriting a parser, restarting an agent, or reprocessing hours of delayed data. Each step takes time, and during that window, alerts based on that feed are blind.

Now imagine a pipeline that recognizes the same problem before it breaks. The system detects that a new log field has appeared, maps it against known schema patterns, and validates whether it is relevant for existing detections. If it is, the system updates the transformation logic automatically and tags the change for review. No manual intervention, no lost data, no downstream blind spots.

That is the difference between automation and autonomy. Traditional scripts wait for failure; Agentic AI predicts and prevents it. These systems learn from historical drift, apply corrective actions, and confirm that the output remains consistent. They can even isolate an unhealthy source or route data through an alternate path to maintain coverage while the issue is reviewed.

For security teams, the result is not just faster operations but greater trust. The data pipeline becomes a reliable partner that adapts to change in real time rather than breaking under it.

Why Security Operations Can’t Scale Without It

Security teams have automated their alerts, their playbooks, and even their incident response, but their pipelines feeding them still rely on human upkeep. This results in poor performance, accuracy, and control as data volumes grow. Without Automated Data Engineering, every new log source or data format adds more drag to the system. Analysts chase false positives caused by parsing errors, compliance teams wrestle with unmasked fields, and engineers spend hours firefighting schema drift.

Here’s why scaling security operations without an intelligent data foundation eventually fails:

• Data Growth Outpaces Human Capacity
  Ingest pipelines expand faster than teams can maintain them. Adding engineers might delay the pain, but it doesn’t fix the scalability problem.
• Manual Processes Introduce Latency
  Each parser update or connector fix delays downstream detections. Alerts that should trigger in seconds can lag minutes or hours.
• Inconsistent Data Breaks Automation
  Even small mismatches in log formats or enrichment logic can cause automated detections or SOAR workflows to misfire. Scale amplifies every inconsistency.
• Compliance Becomes Reactive
  Without policy enforcement at the pipeline level, sensitive data can slip into the wrong system. Teams end up auditing after the fact instead of controlling at source.
• Costs Rise Faster Than Value
  As more data flows into high-cost platforms like SIEM, duplication and redundancy inflate spend. Scaling detection coverage ends up scaling ingestion bills even faster.

Automated Data Engineering fixes these problems at their origin. It keeps pipelines aligned, governed, and adaptive so security operations can scale intelligently — not just expensively.

The Next Frontier: Agentic AI in Action

The next phase of automation in security data management is not about adding more scripts or dashboards. It is about bringing intelligence into the pipelines themselves. Agentic systems represent this shift. They do not just execute predefined tasks; they understand, learn, and make decisions in context.

In practice, an agentic AI monitors pipeline health continuously. It identifies schema drift before ingestion fails, applies the right transformation policy, and confirms that enrichment fields remain accurate. If a data source becomes unstable, it can isolate the source, reroute telemetry through alternate paths, and notify teams with full visibility into what changed and why.

These are not abstract capabilities. They are the building blocks of a new model for data operations where pipelines manage their own consistency, resilience, and governance. The result is a data layer that scales without supervision, adapts to change, and remains transparent to the humans who oversee it.

At Databahn, this vision takes shape through Cruz, our agentic AI data engineer. Cruz is not a co-pilot or assistant. It is a system that learns, understands, and makes decisions aligned with enterprise policies and intent. It represents the next frontier of Automated Data Engineering — one where security teams gain both speed and confidence in how their data operates.

From Awareness to Action: Building Resilient Security Data Foundations

The future of cybersecurity will not be defined by how many alerts you can generate but by how intelligently your data moves. As threats evolve, the ability to detect and respond depends on the health of the data layer that powers every decision. A secure enterprise is only as strong as its pipelines, and how reliably they deliver clean, contextual, and compliant data to every tool in the stack.

Automated Data Engineering makes this possible. It creates a foundation where data is always trusted, pipelines are self-sustaining, and compliance happens in real time. Automation at the data layer is no longer a convenience; it is the control plane for every other layer of security. Security teams gain the visibility and speed needed to adapt without increasing cost or complexity. This is what turns automation into resilience — a data layer that can think, adapt, and scale with the organization.

As Cybersecurity Awareness Month 2025 continues, the focus should expand beyond threat awareness to data awareness. Every detection, policy, and playbook relies on the quality of the data beneath it. In the next part of this series, we will explore how intelligent data engineering and governance converge to build lasting resilience for security operations.