Human-Guided Machine Learning: A Different Approach to Alert Fatigue
Chris Carlson explains why the Critical Start MDR platform favors determinism over autonomy at a time when cybersecurity vendors are rushing to bolt generative AI into every product layer.
In a recent interview with ISMG at Black Hat 2025, Carlson, Chief Product Officer at Critical Start, described how the company is tackling alert fatigue, reducing false positives, and supporting SOC analysts without handing over critical decisions to AI.
“The false positive problem, the alert fatigue problem was causing negative outcomes not only for our team and our SOC, but also customers themselves.”
That insight led to the creation of the Trusted Behavior Registry® (TBR®), a deterministic, ML-driven capability that scales across customers and environments to eliminate noise before it reaches an analyst.
A Deterministic Approach to Security Operations
While some vendors experiment with autonomous decision-making, Critical Start takes a different path. At the core of its SOC workflow is what Carlson calls a deterministic processing zone: a tightly controlled environment where outcomes are consistent, repeatable, and validated by humans.
“From start to finish is a deterministic processing zone with our SOC analysts. They guarantee repeatable, accurate, consistent results.”
This structure enables something few MDR providers can claim: complete alert review within contractual SLAs.
“This allows our team to look at every alert from their security tools within the industry’s only SLA.”
A More Nuanced Definition of “Response”
In MDR, “response” is often vague. Critical Start applies a structured model called response action typing, which categorizes actions into:
- Containment
- Eradication
- Prevention
Each response type aligns with the customer’s risk tolerance and operational resilience requirements.
“We have a very detailed rules of engagement... that line to the customer's risk, but more importantly their business operation resiliency requirements.”
This level of specificity stands in stark contrast to providers that merely forward alerts or deliver notifications without context.
Where AI Adds Value (and Where It Doesn’t)
AI plays a role in the Critical Start MDR platform, but not in decision-making. Instead, it accelerates analyst workflows by automating investigation tasks such as data aggregation and long-form summaries.
“Our SOC AI capability ... doesn’t make decisions on its own. It supports the human analyst.”
Carlson stressed that keeping the human in control is essential to trustworthy automation.
“The most important work ... is human decisioning if this is a real true attack.”
Responsible AI by Design
The Critical Start AI model is privately hosted, contractually governed, and fully audited. Every input and output is logged, with final validation always resting on an experienced analyst.
“We do not use public open source LLM models. We have private models. We do self-hosting. We have commercial contracts with the data protection agreements.”
“We log every input, we log every output, we validate it ... our SOC analyst team ... has the final eyes on glass.”
Translation for those outside the SOC: analysts — not algorithms — are the last line of judgment before action is taken.
Normalizing the Data That Fuels AI
Carlson also emphasized the importance of structured, high-quality data in effective AI. Critical Start uses the Open Cybersecurity Schema Framework (OCSF) to normalize inputs, even converting tools that don’t natively support the format.
“The use of AI is really a data problem ... Our platform would convert those tools ... into the OCSF standard.”
This normalization improves machine learning performance and strengthens the reliability of all AI-driven processes.
A Broader Vision for MDR
Looking ahead, Critical Start sees MDR evolving beyond alert triage into full-spectrum risk reduction. The roadmap includes:
- Normalized asset inventory with criticality scoring
- Vulnerability prioritization based on operational risk
- Executive-ready insights for board-level reporting
- Direct integration with emerging cloud and AI environments
“Our vision is how we can partner with our security leaders... not only on the security operations... but also how we can reduce risk across their security programs.”

You may also be interested in…
RESOURCE CATEGORIES
- Buyer's Guides (0)
- Consumer Education (0)
- Consumer Stories (0)
- Cybersecurity Consulting (0)
- Data Breaches (0)
- Data Privacy (0)
- Incident Response (0)
- Interview (0)
- MDR Services (0)
- MobileSOC (0)
- News (0)
- Press Release (0)
- Research Report (0)
- Security Assessments (0)
- Thought Leadership (0)
- Threat Hunting (0)
- Video (0)
- Vulnerability Disclosure (0)