Forget the movie scenes. Most days in cybersecurity aren’t about zero-days, red teaming, or duct-taped Python scripts written in the heat of an incident.
The real work often revolves around data.
Security professionals spend a large bulk of their time collecting, interpreting, and responding to streams of telemetry across systems, endpoints, and networks.
Without quality data, robust systems, and intelligent people to interpret and take action — there is no security team.
- You can’t write effective detection rules.
- You can’t hunt for threats retroactively or proactively.
- You can’t investigate, contain, or recover from incidents.
If there’s no visibility into your environment, you’re flying blind. Or just as dangerous is having the data and not knowing how to read it.
That’s why data analytic and statistical knowledge aren’t just nice-to-haves. They’re critical.
In this field, if you don’t understand your environment, you can’t protect it.
If you enjoy this article and want to be the first to see more like it, consider subscribing to my newsletter, the Cybersec Cafe, for free. I post content there first, and here second (I’m currently 15 articles behind on Medium!). Plus, you’ll get it straight to your inbox.
I also know the cybersecurity job market is tough right now — there’s a lot of talent out there, and not enough jobs. I built a new training platform called Defend the Org designed to teach you blue team skills that are actually used in the industry (Detection Engineering, Threat Hunting, Incident response, and more). I personally use it weekly help me stay sharp, continue upskilling, and itch that problem solving part of my brain.
Challenges
Even with the right tools and a skilled team, logging and monitoring isn’t as simple as flipping a switch.
There’s more to it than plugging different platforms into the SIEM, waving your magic wand, and suddenly you have valuable insights.
There are tradeoffs, tough choices, nuance, and plenty of considerations to be made along the way.
What do we collect?
Not all logs are created equal. You can’t collect everything — at least not realistically.
So a conscious decision must be made for every source.
At its simplest form, you need to determine what log sources are valuable by taking the time to spell out why.
Start by asking:
- What’s the actual value of this log source?
- Is it needed for real-time detection?
- Does it help with incident response?
- Does it enrich other logs through context?
- Is it required for compliance?
A shared understanding of what you’re collecting and why helps avoid wasted effort and bloated pipelines.
This is the foundation of a smart, sustainable strategy.
Where do we store it?
Storage is a constant balancing act between cost and capability. Budget is not infinite and log storage is expensive.
You’ll likely have two primary tiers:
- High-cost storage (e.g. your SIEM) for logs that support real-time detection use cases and require fast access.
- Low-cost storage (e.g. AWS S3) for logs that provide investigative context or are required for compliance retention.
There’s no one-size-fits-all solution. It’s no longer realistic nor cost-effective to store all log sources in a single source.
As a team you’ll need to understand what you prioritize — speed, budget, a single-pane-of-glass…
If you have the budget to keep all logs in one place — consider yourself lucky!
How long do we keep it?
It’s not always obvious what data you will need, or when you will need it.
The safest answer is often: “Keep everything, for as long as you can stomach it.”
But the reality is storage costs add up fast, especially for high-volume, high-cost platforms like SIEMs.
Many teams default to keeping logs for 12–15 months, which aligns with common compliance requirements.
But what happens if a threat has been lurking quietly for beyond then? What if a legal hold or regulatory inquiry suddenly requires access to old logs?
These are the kinds of scenarios that make retention strategy a critical part of your logging plan. The key is balancing cost, compliance, and risk — while also preparing for the unknown.
How do we drive action from our data?
With so many sources, fields, and values flooding your SIEM every day, separating noise from real signals can feel impossible.
But at the end of the day, that’s the job. Turning raw data into meaningful insight is what makes a security program proactive instead of reactive. And that takes skill.
You’ll need to write queries, look for patterns, understand business context, and recognize anomalies. It’s not just an analyst’s job — it’s a core skill for anyone working in cybersecurity — whether you’re red team, blue team, or somewhere in between.
The good news? Once you learn how to work with data, that skill travels with you.
The hard part? Getting there. But once you’re on the other side, it’s one of the most valuable tools for your career.
Architecture
The Traditional Approach
The go-to strategy for many cybersecurity teams has long been to send all logs to the SIEM.
The goal? A mythical “single pane of glass” — or one place to see everything. But in today’s landscape, is that even practical? Or smart?
Relying on a single platform can quickly lead to vendor lock-in. The more time and effort you invest into the one platform, the harder it becomes to leave.
Migrating your data, retraining your team, rebuilding your infrastructure, reconfiguring alerts — it’s a heavy lift.
And vendors know this. But at this point, you become a slave to their pricing because they know you’re stuck. There are a couple vendors that are notorious for insanely high cost (but I won’t put them on blast here).
Then there’s the issue of siloed data. Along with security specific data, security teams also often ingest some similar sources as other departments — leading to double ingestion costs and unnecessary complexity.
The truth is, the traditional model is showing its age. New players are entering the market with flexible, cost-effective approaches.
That “single pane” is cracking, and it might be time to rethink what centralized visibility should really look like.
Data is on the Move
Data lakes are rapidly becoming the backbone of modern security architectures.
Why? Because they’re not just cheaper, they’re smarter. A well-architected data lake allows you to store security-relevant data at scale, run advanced analytics, and break down silos between teams.
All while avoiding traditional vendor lock-in. You have the ability to:
- Centralize and unify data across departments.
- Lower storage and compute costs.
- Scale effortlessly.
- Support more complex detection and investigation workflows.
As this model continues to gain traction, SIEM vendors are being forced to adapt. They’re now figuring out how to work on top of your data lake — a major shift in power and flexibility.
The result? You take back ownership of your data. You control the architecture. And you can swap in and out tools as your needs evolve without feeling handcuffed to a single platform.
How Security Teams are Operationalizing Data
Statistics is the science of collecting, analyzing, interpreting, presenting, and organizing data.
The SIEM is a big data engine. It provides the tools to ingest, store, and visualize your security telemetry. But without the skills to analyze and operationalize the data, it’s like owning a library and not being able to read.
Security teams must develop strategies to act on their data at scale. Otherwise detection engineering, triage, hunting, and incident response all break down.
Detections
Detections are the heart beat of security operations.
Traditional detections often rely on black and white boolean logic to determine whether an event matches known bad behavior. But as threats grow more subtle and user behavior more dynamic, this approach starts to fall short.
That’s where statistical thinking steps in.
Behavioral detections, especially user-based ones, are notoriously tricky to get right. But by applying basic statistical analysis like mean and standard deviation to historical activity, you can begin to identify anomalies by searching for outliers.
These are specific activities that are statistically improbable.
This mindset shift allows you to go beyond simple pattern matching and to find signals that are truly anomalous.
Combine this with boolean logic, and you’ve got a powerful hybrid.
Alert Triage
Whether you’re manually triaging alerts or building automated SOAR workflows, statistical reasoning is a crucial skill.
Every alert is in a sense, a question: “Is this worth our time to investigate further?”
To answer it, you need to think like both a security analyst and a data analyst — you need to sift through raw telemetry, identify the relevant pieces, and organize them into a coherent story about a user, system, or behavior.
The goal is to contextualize the signal and assess the likelihood that it represents real risk. Sounds straightforward — but the challenge lies in variety and business context.
Different log sources, enrichment layers, and detection types all introduce complexity. And in these moments, environmental knowledge becomes just as important a technical skill.
Performance
The numbers don’t lie.
When you’re dealing with massive volumes of data, gut feelings won’t cut it — you need your metrics to prove your security function is performing.
Start collecting performance data across your operations as soon as possible: detection, response, and SOC workflows. These metrics provide an honest snapshot of where you stand today and how you’re trending over time.
Track the fidelity of your detections, the mean time to triage, and how long it takes to resolve incidents.
This data will quickly become your compass — pointing the way to efficiency and continuous improvement.
Threat Hunting
At its core, threat hunting is about finding what doesn’t belong.
It’s a manual process rooted in curiosity, intuition, and a methodical approach.
The best hunters don’t just stumble upon threats — they use structured techniques to interrogate data, spot anomalies, and test their hypotheses.
That means slicing through big datasets, surfacing patterns, and building a story based on evidence.
It takes a blend of technical skill and investigative mindset. The challenge? Knowing what to look for and how to get there without drowning in the noise.
Security Incident Response
Incident response thrives on precision, and your data is the foundation.
You’re not just collecting metrics to see how your team responds, you’re also building a full timeline of events based on historical data.
Attacks often sprawl. Your job is to trace them: sift through logs, correlate data sources, and identify the start and spread of an incident.
That means narrowing scope, identifying what’s relevant, and cutting the rest.
If you can compare current activity against historical baselines, even better. You’ll move faster, make stronger decisions, and resolve incidents with confidence.
The Narrative
By now, you’re probably noticing a theme: using data and statistical analysis to craft a narrative.
In cybersecurity, it’s not enough to just make sense of data — you need to translate it into something others can understand and act on.
That means making data actionable — the skill of filtering through massive amounts of telemetry, identifying what matters, and drawing conclusions that drive decisions.
Sure, if you’re communicating engineer to engineer, raw data might be enough.
But let’s be honest — that’s not how the real world works. Most of the time you’ll need to explain your findings to people who don’t live in the logs like you do.
Data is the evidence. The narrative is the conclusion.
This is exactly why statistical proficiency is so critical in cybersecurity. It’s the intersection of math and communication — taking something complex and making it understandable.
The professionals who can look at a wall of numbers and translate it into a compelling, security-relevant story are the ones who stand out. That skill of turning raw data into a clear and confident narrative is a superpower.
Cybersecurity is challenging for this exact reason. It’s not just one discipline — it’s many combined.
You need technical chops across a massive stack, data fluency, communication skills, and strategic thinking. All working in harmony.
But like anything else worth mastering, it takes practice. You won’t learn this overnight, but you will learn it if you show up, do the work, and build on the basics.
If you’re looking to improve this specific skillset, I’d highly recommend checking out these two articles next:
- My Log Source-Agnostic Methodology to Understanding Big Data
- Why Knowing How to Query is an Essential Cybersecurity Skill
—
Remember: The Cybersec Café gets articles first. Subscribe for free here.
Interested in getting into Cybersecurity? Defend the Org has learning tracks and micro-courses to help you go from zero-to-hero in cybersecurity, and gamified learningto help you stay hooked to the process.
I also have a free Discord community for established professionals, practioners, and people hoping to get into cybersecurity to connect. Come hang out!
Oh, and if you want even more content and updates, hop over to my website or follow me on Twitter/X. Can’t wait to keep sharing and learning together!
Cybersecurity is Data: Collect, Analyze, Interpret was originally published in OSINT Team on Medium, where people are continuing the conversation by highlighting and responding to this story.