How to Evaluate a Managed Detection and Response Provider

Where is the starting point?

Most Managed Detection and Response (MDR) evaluations run like software procurements, which is not ideal for such a specialised service.

Often a shortlist is built from industry contacts, web search results, and Gartner/Forrester guides. Providers are then invited to demonstrate their portals, SLAs, features, and clientele. The comparison narrows to integration counts and ATT&CK heatmaps until the decision ultimately lands on the price.

The buyer then discovers -usually during the first real incident-that what they purchased is just a notification service and a dashboard.

MDR is a service and an operational capability. Gartner’s current definition describes it as “remotely delivered, human-led, turnkey, modern SOC functions, ultimately delivering cyberattack disruption and containment.” Every part of that definition describes people and outcomes rather than software. In 2025, the global median dwell time rose to 14 days, while organisations first detected the activity internally in 52% of investigations, up from 43% the year before. What this mean is that detection is getting better, and intrusions are lasting longer at the same time. Evaluating a provider therefore needs a different question from the one used for a product.

If this provider were responsible for detecting and responding to a real attack against my organisation tonight, how would I know that they are capable of doing it?

Cutting through the acronym soup

Security vendor acronyms can be confusing, and the labels on a proposal offer little insight into what you are paying for.

Managed EDR, Managed SIEM, MSSP Monitoring, MXDR, MDR – they all look like simple step-up tiers on a sales slide, but each represents five different operational scopes. Without a close read, it is easy to assume you are buying full coverage when, in reality, you may only be buying basic log collection; it is very important to make this distinction.

That confusion isn’t always an accident. The true gap between these labels only becomes clear at 3:00 AM, when an alert fires and someone must make a real-time decision.

Figure 1 breaks down what the market looks like without the marketing gloss. At the end of the day, the acronym doesn’t matter. What matters are two simple questions: Who actually investigates the noise? And who is authorised to take action when things go sideways?

Even Gartner highlighted this in their 2024 guide, noting that many services label themselves “MDR” while delivering far less than customer expects. Many so-called “24/7 MDR” offerings do not have human analysts investigating threats overnight-they are simply automated pipelines that send an alert to your inbox at 3:00 AM and wait for you to respond.

If you’re evaluating providers, get the real boundaries in writing before signing anything:

• Triage & Investigation: Who filters out the noise, and who conducts deep investigations?
• Incident Declaration: Who decides when an event is actually a breach versus a false alarm?
• Response Authority: What specific actions can the provider take on your network without waking you up for permission?

(Quick tip: If you are operating in Singapore or Malaysia, check  licensing first. Managed SOC providers in both countries are required by national cybersecurity laws to hold a government license. This  gives you an immediate, verifiable baseline when evaluating providers.)

Figure 1: This chart clarifies which responsibilities are handled by the provider versus your team across common managed security service models.

Onboarding as a security control

Once you know who is handling your alerts, the next critical question is how your provider will access and interact with your environment.

Onboarding a security partner means granting a third party persistent, privileged access to your SIEM, SOAR, Identity Provider, EDR, cloud tenants, and email services.

Unfortunately, the standard industry approach to onboarding is often inadequate. Many providers request a global Domain Admin or privileged service account, store the credentials, and consider the process complete. This approach creates unnecessary risk. In contrast, modern security architectures provide more secure alternatives-such as Entra ID Access Packages, federated child identities, and scoped service principals-that provide stronger isolation, full auditability, and true least privilege.

When evaluating a prospective partner, go beyond simply asking if they can connect. Ask for specifics on how that integration is designed and managed:

Figure 2: Key differences between common provider connection practices and modern standards for secure onboarding.

The myth of “Status: connected”

Practitioners know that a log source shown as “onboarded” in a dashboard proves very little on its own. The tougher-and more telling-question is whether providers ensure telemetry continues to function correctly after go=live. Ensuring that it works after the pipe is open.

• Is the expected log volume coming through, or has a firewall started silently dropping traffic?
• Are the specific event types that your detection rules depend on present?
• Are timestamps synchronised, or are events arriving hours late?
• Is the security tool even configured to generate the right logs in the first place?

Joint guidance from CISA and Singapore’s CSA highlights this same point: log quality is about the types of rich events you collect, not just how clean the format looks. NIST’s draft log management guidelines (SP 800-92) say the same: you must continuously monitor logging health and pipeline failures, not just check a box at go-live.

Telemetry failures are silent (and deadly)

Why does this matter so much? Because when telemetry breaks, it doesn’t trigger a bright red error screen-it simply goes quiet.

Take Okta’s support system breach in late 2023 as an example. A logging gap-where different access methods generated different event types-meant unauthorized file downloads went completely unnoticed for weeks. The intrusion was ultimately flagged by a customer, not by Okta’s internal monitoring.

If a company whose core product is authentication logging can miss a blind spot in its own telemetry, an MSSP ingesting logs from dozens of enterprise clients is even more likely to miss yours-unless they have explicit automation to detect such gaps.

Ask your provider to demonstrate, in real time, how they detect when a log source silently stops sending data. Find out how their platform distinguishes between a quiet Sunday afternoon and a dead log forwarder, and whose phone is  alerted when a critical detection field disappears.

If their honest answer is “you’d probably notice it first and open a ticket,” congratulations: you’ve just found the edge of their service.

Detection engineering and detection quality

An MDR provider cannot make up for weak detection engineering indefinitely. When analysts are forced to work with noisy, brittle, or silently broken detections, the result is predictable: either they drown in alert fatigue, or they eventually stop looking altogether.

Forrester names detection engineering as one of the two most important trends shaping the MDR space, describing detection-as-code as “the only way to scale detection meaningfully as an MDR provider.” Yet, according to SANS, fewer than 45% of organisations have a reliable way to measure the effectiveness of their detection engineering. Assume your prospective provider falls into that majority-unless they can prove otherwise.

Prioritise transparency and proof. Inquire about how their detection engine is engineered and maintained:

• Dedicated Ownership: Is there a dedicated function or team responsible for rule creation, testing, tuning, regression testing against schema changes, version control, and retirement?
• Verifiable Efficacy: Request the opportunity to test detections directly. Ask if you can trigger a controlled, benign administrative action or a simple synthetic test (such as executing a harmless certutil decode command, spawning a suspicious parent-child process, or creating an impossible-travel API call) to observe how their pipeline responds.
• In-Environment Testing: How often are detections exercised against known malicious behaviour inside customer environments? Providers who tests rules only in their own lab are ultimately just validating their lab-not your network.

Identity and cloud: the EDR blind spot

Most traditional MDR providers built their operational models around Endpoint Detection and Response (EDR). However, in modern enterprise environments, initial access rarely starts with a malicious executable on a desktop. Instead, it often starts in identity providers and cloud platforms such as Entra ID, Okta, AWS, Google Cloud, or Microsoft 365, places where endpoint agents simply do not exist.

Evaluating a provider’s cloud and identity coverage requires looking beyond simple API ingestion and examining how they address non-endpoint threats:

• Identity Threat Detection (ITD): How do they detect session hijacking, token theft, or impossible travel when no malware is ever executed on a disk?
• SaaS & Cloud Sprawl: How do they monitor for rogue API keys, suspicious OAuth app consents, or overly permissive service principals created deep inside your cloud tenants?
• Cross-Domain Correlation: Can their platform correlate an anomalous Okta login from a foreign IP address with a subsequent PowerShell command executed on an endpoint ten minutes later?

If a provider’s primary answer to cloud security is simply “we ingest your audit logs into our SIEM,” ask for more detail. Log collection is not threat detection. Without true cross-domain correlation across identity and cloud, sophisticated adversaries can move through completely unnoticed.

Investigation, correlation, and the humans behind the screen

Reading an alert and investigating an attack are fundamentally different skill sets. Triage determines whether a signal deserves attention. Investigation uncovers what happened, assesses the blast radius, and identifies what remains unknown. Threat Hunting proactively searches for activity that detections may have missed, while Incident Response focuses on containing the damage. When evaluating providers, ask them to walk through each function-notice whether they clearly distinguish between these roles or blur the definitions together.

Modern attacks rarely stay confined to a single telemetry domain, making endpoint-only monitoring dangerously blind. Consider this standard identity-to-cloud attack path:

1. A user grants an OAuth app mailbox access after a consent-phishing lure.
2. The attacker uses stolen session tokens to log into your cloud tenant from an unverified IP.
3. They leverage a legacy app with elevated rights, drop a quiet mailbox rule, and scan SharePoint.
4. Days later, a remote management tool is deployed onto an endpoint.

Individually, each step can look like normal administrative noise. Connected together, they represent a critical breach.

The real test is whether an MDR provider’s analysts can correlate email, identity, cloud, SaaS, and endpoint events into a single, coherent narrative. Ask them to demonstate this capability using a scenario you provide.

What happens at 2:35 am?

Once you understand how a provider investigates incidents, take the next step and ask who is doing the work when things go wrong off-hours:

• Regional Intelligence: Is their threat intelligence tailored to adversaries and tactics specific to your region, or do they rely solely on generic global feeds?
• Analyst Autonomy: What percentage of alerts stay with Tier 1 analysts? Are analysts empowered to move beyond scripted playbooks to form and test their own hypotheses, or are they just processing tickets?
• The Dedicated Technical Lead: Will you have a named technical lead assigned to your account? A rotating SOC analyst at 2:00 AM may not understand your environment’s context. You need a dedicated resource who knows your tech stack, your crown-jewel assets, and what “normal” looks like for your business.

Pro-tip: Insist on interviewing the actual analysts and your assigned technical lead during evaluations, not just the sales engineers. The gap between sales pitch and operational reality is often where the truth emerges.

The handoff: does the provider own incident response?

Detecting an attack is only half the battle. One of the biggest pitfalls in MDR contracts is the vague line between basic containment (such as isolating a host or disabling a user) and full Incident Response (IR).

Many providers stop at basic containment, raise a high-severity ticket, and leave you to find your own Digital Forensics & Incident Response (DFIR) vendor.

Ask providers to explain in detail how the transition from monitoring to active IR is managed:

• DFIR integration: Is incident response handled by an integrated, in-house team, or outsourced to a third party?
• The In-House Advantage: An in-house IR team operates on the same platform as your SOC, inheriting all historical telemetry, investigation notes, triage data, and case files.  This results in a seamless handoff with no time lost deploying new tools or requesting access.
• Scope of Response: Does the provider perform root cause analysis, forensic artifact collection, and active threat eradication? Or do they simply escalate the incident and leave recovery entirely to you?

If a provider can’t deliver an integrated IR transition with zero loss of context, you’ll be  left bridging a major operational gap at the worst possible moment.

AI and automation: cutting through the hype

Almost every security provider is plastering “AI-powered” solutions on their sales decks and website. While true AI and automation can provide real benefits, the claims in this space are often exaggerated.

The industry is beginning to acknowledge this. Gartner’s 2026 Hype Cycle for Security Operations places autonomous “AI SOC Agents” at the Peak of Inflated Expectations, warning heavily against widespread “AI washing” and “agent washing”[5]. Meanwhile, SANS survey data shows that only 37% of security teams actually trust AI to independently detect or respond to threats[6].

When evaluating a provider, set aside the buzzwords and focus strictly at what the AI is authorised to do:

Figure 3: Summarises the concrete evidence you should request to validate vendor claims across key MDR service areas.

The structural trap

The primary risk of fully autonomous AI in security operations isn’t just the potential of hallucination, rather it’s structural.

In a SOC, raw telemetry is fundamentally controlled by the attacker. A malicious user-agent string, a crafted DNS query, an attempted username, or an HTTP POST payload can easily carry hidden prompt injection instructions targeting the underlying LLM model.

Recent academic security research demonstrated that indirect prompt injections embedded in a raw log content successfully suppressed 68% of malicious alerts against naive LLM classifiers. The researchers concluded that SOC AI architectures must treat incoming log streams as untrusted, adversarial input, not as benign analyst context. This aligns with industry warnings from OWASP, which lists Prompt Injection (LLM01) and Excessive Agency (LLM06) among the top security risks for LLM applications.

Key questions to clarify with your provider

If a provider claims their platform relies on autonomous AI or LLM agents, address governance and risk management before signing:

• Does the AI processing pipeline comply with cross-border data transfer laws and regional frameworks (such as Singapore’s PDPA or Malaysia’s RMiTI? f the model runs in a foreign region, who holds the legal jurisdiction over your ingested logs?
• Where are your security logs and prompts processed and stored? Is the underlying model hosted in an isolated Enterprise LLM instance (with zero-retention SLAs), self-hosted on a dedicated cloud tenant, or running on local edge infrastructure?
• What security decisions does the AI make independently, and what confidence thresholds govern those actions?
• Can the system show its step-by-step reasoning and raw evidence chain for every verdict, or is it a black box?
• How does their platform sanitise adversarial log content before feeding it into the model context window?
• How frequently do senior human analysts review AI-generated verdicts and closed tickets?
• What happened the last time the AI model issued a false negative or hallucinated a containment action, and how was it detected and addressed?

A provider that cannot clearly describe its AI’s failure modes has likely never tested them.

SLAs, pricing, and contract realities

Notification SLAs are easily the most quoted and least meaningful numbers in any MDR proposals. Two vendors can both promise a “15-minute response time” while measuring entirely different things. Gartner explicitly cautions buyers against overemphasizing traditional SLAs in detection and response services.

Before discussing numbers, insist on precise definitions. Require the provider to break down their SLAs across three distinct operational stages, clearly stating when the clock starts and stops for each:

Figure 4: Highlights of red flags and their significance when evaluating MDR providers.

Then, evaluate the health metrics that genuinely reflect operational quality-such as false positive ratios, escalation accuracy, log health monitoring, the creation of custom detections, and whether investigations consistently reach root-cause analysis. If a provider only reports notification speed, they are highlighting the one metric that’s easiest to manipulate.

Service and threat reporting

Don’t allow  providers to treat monthly reporting as a simple PDF export of closed ticket counts and raw alert volumes. Effective monthly service and threat reporting should deliver strategic value across three distinct operational layers:

Executive & Board Governance: Translates technical activity into insights on enterprise risk posture. Reports should cover macro-trends, SLA performance compliance, shifts in the threat landscape relevant to your organization and sector, and clear return on investment (ROI) metrics for your leadership.

Operational Health & Hygiene: Provides details on log ingestion health, telemetry coverage gaps, noise-reduction metrics (such as the reduction of millions of raw events to true-positive escalations), and analysis of recurring false-positive root causes.

Posture Hardening & Recommendations: Highlights near-misses, adversary tactics encountered (mapped to MITRE ATT&CK), and delivers actionable, prioritised advice for hardening your environment and  preventing repeat incidents.

If a provider cannot show you a sample monthly report that drives actionable security improvements rather than just proving they were awake, it’s a sign you are dealing with a passive log monitoring service, not a true MDR partner.

The true cost of “cheap” MDR

Pricing should inform your decision, not drive it. Always look beyond the baseline platform cost to understand what is excluded from the scope:

• Storage retention extensions, active response actions, DFIR hours, custom rule tuning, off-hours escalations, minimum volume commitments, and exit fees.
• The cheapest MDR option can quickly become the most expensive when your internal engineering team is forced to spend hundreds of hours building the log parsing, rule tuning, and investigation depth the provider failed to deliver.

Exit terms and the vendor as a target

Exit conditions are often only scrutinised after a relationship breaks down. Ensure your contract guarantees the right to retrieve all historical logs, case notes, and custom detection logic in open, usable formats, and that you can revoke the provider’s tenant access within hours.

Additionally, remember that your MDR provider is a high-value target for attackers, just as threat actors see them. Incidents like the Kaseya supply-chain breach and Okta’s support system intrusion show that adversaries actively target managed providers to gain downstream access to customer environments.

Before handing over the keys to your environment, demand full transparency into the provider’s internal identity security, analyst workstation hardening, customer tenant segregation, insider threat controls, and results from third-party penetration testing.

Cutting through the pitch

The primary responsibility of a security leader during procurement is to translate every marketing claim into a non-negotiable request for evidence:

• “24/7/365 SOC” should trigger a request for anonymised shift rosters, graveyard-shift escalation workflows, and sample overnight investigation notes.
• “Broad Detection Coverage” should be supported by a detection catalogue, including explicit log schema dependencies and proof of past regression testing.
• “AI-Powered Triage” warrants a live walkthrough demonstrating specific confidence thresholds, data boundary separation, and prompt injection mitigations.
•  “Proactive Threat Hunting” requires a documented methodology with concrete examples of past hunt hypotheses and findings.
•  “Rapid Response” should be validated by measured Time to Respond (TTR) metrics, validated by a controlled synthetic verification test.
•  “World-Class Analysts” becomes an unscripted, direct conversation with the actual analysts and named technical lead assigned to your account.

Tailoring your evaluation matrix

There is no one-size-fits-all weighting scheme for evaluating an MDR provider. The importance of each domain should be based entirely on your organisation’s internal maturity and operational requirements:

Figure 5: Illustration of how the priority of MDR evaluation criteria varies with your organisation’s security maturity and regulatory profile.

Ultimately, shift your procurement focus from “Which provider offers the most features for the lowest price?” to “Which provider can conclusively prove they have the people, engineering, telemetry, processes, and contractual authority to detect, investigate, and contain a real breach inside our environment?”

Pose this exact question of every vendor on your shortlist and never accept a marketing dashboard as the answer.

Test before you buy

Vendor presentations and dashboard walkthroughs do not constitute evidence. If a provider offers a Proof of Value/Concept (PoV/C) or pilot deployment, treat it as an opportunity to run controlled, scoped exercises with appropriate safety measures rather than simply observing logs as they roll in.

Execute tests across multiple telemetry domains to assess how the provider’s detection engine and SOC analysts respond under real-world conditions:

• Simulate credential compromise, OAuth consent grants, anomalous mailbox forwarding rules, and cloud account logins from an unverified geography.
• Trigger suspicious PowerShell execution, living-off-the-land (LotL) binary abuse, endpoint persistence mechanisms, and staged data exfiltration.
• Take a pragmatic approach to ransomware testing. Deploying a destructive encryption payload in production is rarely acceptable, and most providers will refuse it. Instead, emulate pre-encryption behaviours, such as shadow copy deletion, mass file renaming, or volume shadow service enumeration in an isolated segment to exercise the exact same detection path.
• Connect identity, cloud, and endpoint events into a single kill chain to test the provider’s ability to correlate cross-domains.

Evaluating the full response chain

Don’t stop at checking whether an alert appeared. Evaluate every link in the provider’s operational chain:

1.  Did raw telemetry arrive in full? Which specific detection fired? Were disparate identity, cloud, and host signals joined into a single coherent case narrative, or did they trigger three isolated tickets?
2. Who triaged the case and how quickly? Was their investigation accurate, or did they miss the lateral movement? Was containment executed correctly under agreed response authorities?
3.   Review the actual investigation report. A high-quality report should clearly detail what happened, which assets and identities were involved, the adversary’s actions, the supporting evidence, the analyst’s confidence level, what remains unknown, what containment actions were taken, and immediate next steps for your team.

If a notification simply reads “Suspicious activity detected on host X, please investigate,” it’s a warning sign of the kind of unhelpful noise you can expect to receive at 3:36 AM during a real incident.

Five questions that are worth more than a hundred

If procurement timelines force you to reduce your RFP down to just five essential questions, make them these. Each requires concrete evidence to answer and directly tests one of the four key capabilities an attack will expose:

1. How do you know when our visibility breaks, and how do you distinguish a quiet network from a broken log pipeline?
2. Show us a detection rule you proved to be ineffective in production. How did you identify the issue, and what specific changes did you make to fix it?
3. Demonstrate how alerts across identity, endpoint, cloud, and SaaS telemetry are correlated into a single investigation with a coherent threat narrative.
4. What specific actions can your analyst take at 02:00 AM in our local time zone without waiting for our sign-off, and under whose pre-authorized authority?
5. How will you prove, twelve months from today, that our security posture is measurably stronger than it is now?

The fifth question is the one most providers are least prepared to answer. It is the dividing line between a vendor that passively monitors your environment and a partner that actively matures it.

Beyond features: choosing the right MDR partner

Choosing a Managed Detection and Response provider is not simply about comparing features, dashboards, or price points. It is a strategic decision that shapes your organisation’s ability to withstand and respond to real cyberattacks. The difference between a vendor and a true partner becomes clear not in the heat of an incident-when evidence, operational depth, and trust matter most.

Resist the temptation to treat MDR as a commodity or to rely on superficial checklists. Demand proof, not promises. Prioritise providers who can demonstrate real-world capability, transparency, and willingness to be held accountable before you sign a contract. The right MDR partner will not only detect and respond, but will actively mature your security posture and stand with you when it matters most.

At Theos Cyber, our managed detection and response service is called Managed Threat Detection and Response (MTDR).

About Theos Cyber

Theos Cyber is a specialist MSSP that helps protect companies, making them harder to breach and faster to recover. From incident response and forensics to threat detection and offensive testing, our expert-led services are trusted by financial, tech, and critical infrastructure organisations across APAC, recognised by CrowdStrike as its 2025 Growth MSSP of the Year for APJ.

Appendix A. Proof over claims

Do not ask only what the provider says it can do. Ask how they prove they do it. For each vendor promise, request concrete evidence:

Appendix B. Warning signs

Appendix C. APAC regulatory obligations

Regional regulatory clocks do not pause for weekends, public holidays, or 3:00 AM local time. When an incident occurs outside business hours, your MDR provider’s escalation speed and forensic capabilities directly determine whether you meet your statutory obligations or face regulatory penalties.

Use the compliance requirements below to evaluate whether a prospective MDR vendor can support your jurisdiction’s legal clocks and evidentiary standards.

Licensing, outsourcing and third party obligations. These requirements shape what the contract must contain and in some jurisdictions, determine whether the provider may lawfully serve you at all.

Appendix D. The guide at a glance

This infographic summarises the guide in 5 checks: what you are buying, how a provider connects to you, what the AI is allowed to do, how response times are measured, and how to weight your evaluation.

Figure 6: How to evaluate an MDR provider, the guide at a glance.