Cybersecurity Awareness Month 2025: Why Broken Data Pipelines Are the Biggest Risk You’re Ignoring

Financial data flows are some of the most complex in any industry. Trades, transactions, positions, valuations, and reference data all pass through ETL jobs, market feeds, and risk engines before surfacing in reports. Multiply that across desks, asset classes, and jurisdictions, and tracing a single figure back to its origin becomes nearly impossible. This is why data lineage has become essential in financial services, giving institutions the ability to show how data moved and transformed across systems. So, when regulators, auditors, or even your own board ask: “Where did this number come from?” too many teams still don’t have a clear answer.

The stakes couldn’t be higher. Across frameworks like BCBS-239, the Financial Data Transparency Act, and emerging supervisory guidelines in Europe, APAC, and the Middle East, regulators are raising the bar. Banks that have adopted modern data lineage tools report 57% faster audit prep and ~40% gains in engineering productivity, yet progress remains slow — surveys show that fewer than 10% of global banks are fully compliant with BCBS-239 principles. The result is delayed audits, costly manual investigations, and growing skepticism from regulators and stakeholders alike.

The takeaway is simple: data lineage is no longer optional. It has become the foundation for compliance, risk model validation, and trust. For financial services, what data lineage means is simple: without it, compliance is reactive and fragile; with it, auditability and transparency become operational strengths.

In the rest of this blog, we’ll explore why lineage is so hard to achieve in financial services, what “good” looks like, and how modern approaches are closing the gap.

Why data lineage is so hard to achieve in Financial Services

If lineage were just “draw arrows between systems,” we’d be done. In the real world it fails because of technical edge cases and organizational friction, the stuff that makes tracing a number feel like detective work.

Siloed ownership and messy handoffs
Trade, market, reference and risk systems are often owned by separate teams with different priorities. A single calculation can touch five teams and ten systems; tracing it requires stepping across those boundaries and reconciling different glossaries and operational practices. This isn’t just technical overhead but an ownership problem that breaks automated lineage capture.  

Opaque, undocumented transforms in the middle
Lineage commonly breaks inside ETL jobs, bespoke SQL, or one-off spreadsheets. Those transformation steps encode business logic that rarely gets cataloged, and regulators want to know what logic ran, who changed it, and when. That gap is one of the recurring blockers to proving traceability.  

Temporal and model lineage
Financial reporting and model validation require not just “where did this value come from?” but “what did it look like at time T?” Capturing temporal snapshots and ensuring you can reconstruct the exact input set for a historical run (with schema versions, parameter sets, and market snapshots) adds another layer of complexity most lineage tools don’t handle out of the box.  

Scaling lineage without runaway costs
Lineage at scale is expensive. Streaming trades, tick data and high-cardinality reference tables generate huge volumes of metadata if you try to capture full, row-level lineage. Teams need to balance fidelity, cost, and query ability, and that trade-off is a frequent operational headache.  

Organizational friction and change management
Technical fixes only work when governance, process and incentives change too. Lineage rollout touches risk, finance, engineering and compliance, aligning those stakeholders, enforcing cataloging discipline, and maintaining lineage over time is a people problem as much as a technology one.

The real challenge isn’t drawing arrows between systems but designing lineage that regulators can trust, engineers can maintain, and auditors can use in real time. That’s the standard the industry is now being measured against.

What good Data Lineage looks like in finance

Great lineage in financial services doesn’t look like a prettier diagram; it feels like control. The moment an auditor asks, “Where did this number come from?” the answer should take minutes, not weeks. That’s the benchmark.

It’s continuous, not reactive.
Lineage isn’t something you piece together after an audit request. It’s captured in real time as data flows — across trades, models, and reports — so the evidence is always ready.

It’s explainable to both engineers and auditors.
Engineers should see schema versions, transformations, and dependencies. Auditors should see clear traceability and business definitions. Good lineage bridges both worlds without translation exercises.

It scales with the business.
From millions of daily trades to real-time model recalculations, lineage must capture detail without exploding into unusable metadata. That means selective fidelity, efficient storage, and fast query ability built in.

It integrates governance, not adds it later.
Lineage should carry sensitivity tags, policy markers, and glossary links as data moves. Compliance is strongest when it’s embedded upstream, not enforced after the fact.

The point is simple: an effective data lineage makes defensibility the default. It doesn’t slow down data flows or burden teams with extra work. Instead, it builds confidence that every calculation, every report, and every disclosure can be traced and trusted.

Databahn in practice:  Data Lineage as part of the flow

Databahn captures lineage as data moves, not after it lands. Rather than relying on manual cataloging, the platform instruments ingestion, parsing, transformation and routing layers so every change — schema update, join, enrichment or filter — is recorded as part of normal pipeline execution. That means auditors, risk teams and engineers can reconstruct a metric, replay a run, or trace a root cause without digging through ad-hoc scripts or spreadsheets.

In production, that capture is combined with selective fidelity controls, snapshotting for time-travel, and business-friendly lineage views so traceability is both precise for engineers and usable for non-technical stakeholders.

Here are a few of the key features in Databahn’s arsenal and how they enable practical lineage:

• Seamless lineage with Highway
  Every routing and transformation is tracked natively, giving a complete view from source to report without blind spots.
• Real-time visibility and health monitoring
  Continuous observability across pipelines detects lineage breaks, schema drift, or anomalies as they happen — not months later.
• Governance with history recall and replay
  Metadata tagging and audit trails preserve data history so any past report or model run can be reconstructed exactly as it appeared.
• In-flight sensitive data handling
  PII and regulated fields can be masked, quarantined, or tagged in motion, with those transformations recorded as part of the audit trail.
• Schema drift detection and normalization
  Automatic detection and normalization keep lineage consistent when upstream systems change, preventing gaps that undermine compliance.

The result is lineage that financial institutions can rely on, not just to pass regulatory checks, but to build lasting trust in their reporting and risk models. With Databahn, data lineage becomes a built-in capability, giving institutions confidence that every number can be traced, defended, and trusted.

The future of Data Lineage in finance

Lineage is moving from a compliance checkbox to a living capability. Regulators worldwide are raising expectations, from the Financial Data Transparency Act (FDTA) in the U.S., to ECB/EBA supervisory guidance in Europe, to data risk frameworks in APAC and the Middle East. Across markets, the signal is the same: traceability can’t be partial or reactive, it has to be continuous.

AI is at the center of this shift. Where teams once relied on static diagrams or manual cataloging, AI now powers:

• Automated lineage capture – extracting flows directly from SQL, ETL code, and pipeline metadata.
• Drift and anomaly detection – spotting schema changes or unusual transformations before they become audit findings.
• Metadata enrichment – linking technical fields to business definitions, tagging sensitive data, and surfacing lineage in auditor-friendly terms.
• Proactive remediation – recommending fixes, rerouting flows, or even self-healing pipelines when lineage breaks.

This is also where modern platforms like Databahn are heading. Rather than stop at automation, Databahn applies agentic AI that learns from pipelines, builds context, and acts, whether that’s updating lineage after a schema drift, tagging newly discovered sensitive fields, or ensuring audit trails stay complete.

Looking forward, financial institutions will also see exploration of immutable lineage records (using distributed ledger technologies) and standardized taxonomies to reduce cross-border compliance friction. But the trajectory is already clear: lineage is becoming real-time, AI-assisted, and regulator-ready by default, and platforms with agentic AI at their core are leading that evolution.

Conclusion: Lineage as the Foundation of Trust

Financial institutions can’t afford to treat lineage as a back-office detail. It’s become the foundation of compliance, the enabler of model validation, and the basis of trust in every reported number.

As regulators raise the bar and AI reshapes data management, the institutions that thrive will be the ones that make traceability a built-in capability, not an afterthought. That’s why modern platforms like DataBahn are designed with lineage at the core. By capturing data in motion, applying governance upstream, and leveraging agentic AI to keep pipelines audit-ready, they make defensibility the default.

If your institution is asking tougher questions about “where did this number come from?”, now is the time to strengthen your lineage strategy. Explore how Databahn can help make compliance, trust, and auditability a natural outcome of your data pipelines. Get in touch for a demo!

Ask any security practitioner what keeps them up at night, and it rarely comes down to a specific tool. It's usually the data itself – is it complete, trustworthy, and reaching the right place at the right time?

Pipelines are the arteries of modern security operations. They carry logs, metrics, traces, and events from every layer of the enterprise. Yet in too many organizations, those arteries are clogged, fragmented, or worse, controlled by someone else.

That was the central theme of our webinar, From Chaos to Clarity, where Allie Mellen, Principal Analyst at Forrester, and Mark Ruiz, Sr. Director of Cyber Risk and Defense at BD, joined our CPO Aditya Sundararam and our CISO Preston Wood.

Together, their perspectives cut through the noise: analysts see a market increasingly pulling practitioners into vendor-controlled ecosystems, while practitioners on the ground are fighting to regain independence and resilience.

The Analyst's Lens: Why Neutral, Open Pipelines Matter

Allie Mellen spends her days tracking how enterprises buy, deploy, and run security technologies. Her warning to practitioners is direct: control of the pipeline is slipping away.

The last five years have seen unprecedented consolidation of security tooling. SIEM vendors offer their own ingestion pipelines. Cloud hyperscalers push their monitoring and telemetry services as defaults. Endpoint and network vendors bolt on log shippers designed to funnel telemetry back into their ecosystems.

It all looks convenient at first. Why not let your SIEM vendor handle ingestion, parsing, and routing? Why not let your EDR vendor auto-forward logs into its own analytics console?

Allie's answer: because convenience is control and you're not the one holding it.

" Practitioners are looking for a tool much like with their SIEM tool where they want something that is independent or that’s kind of how they prioritize this "

— Allie Mellen, Principal Analyst, Forrester

‍

This erosion of control has real consequences:

• Vendor lock-in: Once you're locked into a vendor's pipeline, swapping tools downstream becomes nearly impossible. Want to try a new analytics platform? Your data is tied up in proprietary formats and routing rules.
• Blind spots: Vendor-native pipelines often favor data that benefits the vendor's use cases, not the practitioners’. This creates gaps that adversaries can exploit.
• AI limitations: Every vendor now advertises "AI-driven security." But as Allie points out, AI is only as good as the data it ingests. If your pipeline is biased toward one vendor's ecosystem, you'll get AI outcomes that reflect their blind spots, not your real risk.

For Allie, the lesson is simple: net-neutral pipelines are the only way forward. Practitioners must own routing, filtering, enrichment, and forwarding decisions. They must have the ability to send data anywhere, not just where one vendor prefers.

That independence is what preserves agility, the ability to test new tools, feed new AI models, and respond to business shifts without ripping out infrastructure.

The Practitioner's Challenge: BD's Story of Data Chaos

Theory is one thing, but what happens when practitioners actually lose control of their pipelines? For Becton Dickinson (BD), a global leader in medical technology, the consequences were very real.

BD's environment spanned hospitals, labs, cloud workloads, and thousands of endpoints. Each vendor wanted to handle telemetry in its own way. SIEM agents captured one slice, endpoint tools shipped another, and cloud-native services collected the rest.

The result was unsustainable:

• Duplication: Multiple vendors forwarding the same data streams, inflating both storage and licensing costs.
• Blind spots: Medical device telemetry and custom application logs didn't fit neatly into vendor-native pipelines, leaving dangerous gaps.
• Operational friction: Pipeline management was spread across several vendor consoles, each with its own quirks and limitations.

For BD's security team, this wasn't just frustrating, it was a barrier to resilience. Analysts wasted hours chasing duplicates while important alerts slipped through unseen. Costs skyrocketed, and experimentation with new analytics tools or AI models became impossible.

Mark Ruiz, Sr. Director of Cyber Risk and Defense at BD, knew something had to change.

With Databahn, BD rebuilt its pipeline on neutral ground:

• Universal ingestion: Any source from medical device logs to SaaS APIs could be onboarded.
• Scalable filtering and enrichment: Data was cleaned and streamlined before hitting downstream systems, reducing noise and cost.
• Flexible routing: The same telemetry could be sent simultaneously to Splunk, a data lake, and an AI model without duplication.
• Practitioner ownership: BD controlled the pipeline itself, free from vendor-imposed limits.

The benefits were immediate. SIEM ingestion costs dropped sharply, blind spots were closed, and the team finally had room to innovate without re-architecting infrastructure every time.

" We were able within about eight, maybe ten weeks consolidate all of those instances into one Sentinel instance in this case, and it allowed us to just unify kind of our visibility across our organization."

‍ — Mark Ruiz, Sr. Director, Cyber Risk and Defense, BD

‍

Where Analysts and Practitioners Agree

What's striking about Allie's analyst perspective and Mark's practitioner experience is how closely they align.

Both argue that convenience isn't resilience. Vendor-native pipelines may be easy up front, but they lock teams into rigid, high-cost, and blind-spot-heavy futures.

Both stress that pipeline independence is fundamental. Whether you're defending against advanced threats, piloting AI-driven detection, or consolidating tools, success depends on owning your telemetry flow.

And both highlight that resilience doesn't live in downstream tools. A world-class SIEM or an advanced AI model can only be as good as the data pipeline feeding it.

This alignment between market analysis and hands-on reality underscores a critical shift: pipelines aren't plumbing anymore. They're infrastructure.

The Databahn Perspective

For Databahn, this principle of independence isn't an afterthought—it's the foundation of the approach.

Preston Wood, CSO at Databahn, frames it this way:

"We don't see pipelines as just tools. We see them as infrastructure. The same way your network fabric is neutral, your data pipeline should be neutral. That's what gives practitioners control of their narrative."

— Preston Wood, CSO, Databahn

‍

This neutrality is what allows pipelines to stay future-proof. As AI becomes embedded in security operations, pipelines must be capable of enriching, labeling, and distributing telemetry in ways that maximize model performance. That means staying independent of vendor constraints.

Aditya Sundararam, CPO at Databahn, emphasizes this future orientation: building pipelines today that are AI-ready by design, so practitioners can plug in new models, test new approaches, and adapt without disruption.

Own the Pipeline, Own the Outcome

For security practitioners, the lesson couldn't be clearer: the pipeline is no longer just background infrastructure. It's the control point for your entire security program.

Analysts like Allie warn that vendor lock-in erodes practitioner control. Practitioners like Mark show how independence restores visibility, reduces costs, and builds resilience. And Databahn's vision underscores that independence isn't just tactical, it's strategic.

So the question for every practitioner is this: who controls your pipeline today?

If the answer is your vendor, you've already lost ground. If the answer is you, then you have the agility to adapt, the visibility to defend, and the resilience to thrive.

In security, tools will come and go. But the pipeline is forever. Own it, or be owned by it.

The MITRE ATT&CK Evaluations have entered unexpected choppy waters. Several of the cybersecurity industry’s largest platform vendors have opted out this year, each using the same language about “resource prioritization” and “customer focus”. When multiple leaders step back at once, it raises some hard questions. Is this really about resourcing, or about avoiding scrutiny? Or is it the slow unraveling of a bellwether and much-loved institution?

Speculation is rife; some suggest these giants are wary of being outshone by newer challengers; other believe it reflects uncertainty inside MITRE itself. Whatever the case, the exits have forced a reckoning: does ATT&CK still matter? At Databahn, we believe it does – but only if it evolves into something greater than it is today.

What is MITRE ATT&CK and why it matters

MITRE ATT&CK was born from a simple idea: if we could catalog the real tactics and techniques adversaries use in the wild, defenders everywhere could share a common language and learn from each other. Over time, ATT&CK became more than a knowledge base – it became the Rosetta Stone of modern cybersecurity.

The Evaluations program extended that vision. Instead of relying on vendor claims or glossy datasheets, enterprises could see how different tools performed against emulated threat actors, step by step. MITRE never crowned winners or losers; it simply published raw results, offering a level playing field for interpretation.

That transparency mattered. In an industry awash with noise and marketing spin, ATT&CK Evaluations became one of the few neutral signals that CISOs, SOC leaders, and practitioners could trust. For many, it was less about perfect scores and more about seeing how a tool behaved under pressure – and whether it aligned with their own threat model.

The Misuse and the Criticisms

For years, ATT&CK Evaluations were one of the few bright spots in an industry crowded with vendor claims. CISOs could point to them as neutral, transparent – and at least in theory – immune from spin. In a market that rarely offers apples-to-apples comparisons, ATT&CK stood out as a genuine attempt at objectivity. In defiance of the tragedy of the commons, it remained neutral, with all revenues routed towards doing more research to improve public safety.

The absences of some of the industry’s largest vendors have sparked a firestorm of commentary. While their detractors are skeptical about their near-identical statements and suggest that this was strategic, it raises questions at a time when criticisms of MITRE ATT&CK Evaluations were also growing more strident, pointing to how results were interpreted – or rather, misinterpreted. While MITRE doesn’t crown champions, hand out trophies, or assign grades, vendors have been quick to award themselves with imagined laurels. Raw detection logs are taken and twisted into “best-in-class" coverage, missing the nuance that matters most: whether detections were actionable, whether alerts drowned analysts in noise, and whether the configuration mirrored a real production environment.

The gap became even more stark when evaluation results didn’t line up with enterprise reality. CISOs would see a tool perform flawlessly on paper, only to watch it miss basic detections or drown SOCs with false positives. The disconnect wasn’t the fault of the ATT&CK framework itself, which didn’t intend to simulate the full messiness of a live environment. But this gave critics the ammunition to question whether the program had lost its value.

And of course, there is the Damocles’ sword of AI. In a time of dynamic threats being spun up and vulnerabilities exploited in days, do one-time evaluations of solutions really have the same effectiveness? In short, what was designed to be a transparent reference point too often CISOs and SOC teams were left to sift through competing storylines–especially in an ecosystem where AI-powered speed rendered static frameworks less effective.

Making the gold standard shine again

For all its flaws and frustrations, ATT&CK remains the closest thing cybersecurity has to a gold standard. No other program managed to establish such a widely accepted, openly accessible benchmark for adversary behavior. For CISOs and SOC leaders, it has become the shared map that allows them to compare tools, align on tactics, and measure their own defenses against a common framework.

Critics are right to point out the imperfections in MITRE Evaluations. But in a non-deterministic security landscape – where two identical attacks can play out in wildly different ways – imperfection is inevitable. What makes ATT&CK different is that it provides something few others do: neutrality. Unlike vendor-run bakeoffs, pay-to-play analyst reports, or carefully curated customer case studies, ATT&CK offers a transparent record of what happened, when, and how. No trophies, no hidden methodology, no commercial bias. Just data.

That’s why, even as some major players step away, ATT&CK still matters. It is not a scoreboard and never should have been treated as one. It is a mirror that shows us where we stand, warts and all. And when that mirror is held up regularly, it keeps vendors honest, challengers motivated, and buyers better informed. And most importantly, it keeps us all safer and better prepared for the threats we face today.

Yet, holding up a mirror once a year is no longer enough. The pace of attacks has accelerated, AI is transforming both offense and defense, and enterprises can’t afford to wait for annual snapshots. If ATT&CK is to remain the industry’s north star, it must evolve into something more dynamic – capable of keeping pace with today’s threats and tomorrow’s innovations.

From annual tests to constant vigilance

If ATT&CK is to remain the north star of cybersecurity, it cannot stay frozen in its current form. Annual, one-off evaluations feel outdated in today’s fast-paced threat landscape. The need is to test enterprise deployments, not security tools in sterilized conditions.  

In one large-scale study, researchers mapped enterprise deployments against the same MITRE ATT&CK techniques used in evaluations. The results were stark: despite high vendor scores in controlled settings, only 2% of adversary behaviors were consistently detected in product. That kind of drop-off exposes a fundamental gap – not in MITRE’s framework itself, but in how it is being used.

The future of ATT&CK must be continuous. Enterprises should be leveraging the framework to test their systems, because that is what is being attacked and under threat. These tests should be a consistent process of stress-testing, learning, and improving. Organizations should be able to validate their security posture against MITRE techniques regularly – with results that reflect live data, not just laboratory conditions.

This vision is no longer theoretical. Advances in data pipeline management and automation now make it possible to run constant, low friction checks on how telemetry maps to ATT&CK. At Databahn, we’ve designed our platform to enable exactly this: continuous visibility into coverage, blind spots, and gaps in real-world environments. By aligning security data flows directly with ATT&CK, we help enterprises move from static validation to dynamic, always-on confidence.

Vendors shouldn’t abandon MITRE ATT&CK Evaluations; they should make it a module in their products, to enable enterprises to consistently evaluate their security posture. This will ensure that enterprises can keep better pace with an era of relentless attack and rapid innovation. The value of ATT&CK was never in a single set of results – but in the discipline of testing, interpreting, and improving, again and again.