Beyond the Alert: How External Breach Notifications Should Reshape Your Security

I notify many, many organizations that they have compromised systems.

Some people collect coins, and some collect stamps. I spend some of my free time each week looking for hacked systems on the Internet.

Hey… it's a hobby.

Last week, my wife and I were sitting at breakfast with our financial advisor for a quarterly meeting when my cell phone rang. The caller ID indicated an acronym that seemed familiar, so I excused myself and answered the call.

The caller quickly explained that he was calling in response to an email I'd sent the day before outlining some facts that led me to believe that someone had compromised their organization's website. In the email, I made recommendations for the next steps, but he was calling to request additional assistance—what my friend Ed Skoudis used to refer to as the vortex of free consulting. I did my best to give him some quick pointers without being sucked into indentured servitude.

After I hung up the call, I started thinking about providing something more—some kind of documentation for how to move forward when you learn you've been breached. This post is the result of that thinking.

One of my New Year's resolutions for 2025 is to identify and notify one organization each day that they are compromised. So far, I'm ahead of schedule—today is day 127 and I’ve notified 130. This experience highlights a more uncommon scenario: an organization first learns of a breach not from its sophisticated internal tools but from an external party.

I aim to explore what organizations should learn when someone without internal access (no logs, no monitoring dashboards) finds evidence of a compromise. How should this specific type of discovery prompt deeper changes to their security posture?

An external notification isn't just another security ticket. It signifies that all internal defenses and detection mechanisms failed to notice an active compromise. Unlike an internal alert (which shows your security monitoring worked), an external notification proves that:

1. Someone bypassed preventative controls (firewalls, patching, hardening).
2. Detection controls (IDS/IPS, log monitoring, file integrity checks, security scans) either weren't in place, were misconfigured, or didn't cover the specific attack vector or compromised asset.
3. Monitoring was insufficient to spot the symptoms of the breach (e.g., strange website behavior, blocklisting, unusual traffic).

The compromise might have existed for days, weeks, or even months before external detection. The longer the dwell time, the greater the potential damage.

Key Lessons Organizations Should Learn From An External Breach Notification

Lesson 1: Your Detection Capabilities Are Insufficient.

Problem: Your current security stack (WAF, IDS/IPS, AV, EDR, SIEM) didn't catch the active breach.

Actionable Insight: Don't just fix the specific vulnerability. Ask why it wasn't detected. Review tool configurations, rule sets, and coverage. Are you monitoring the right things? Are alerts being generated but ignored (alert fatigue)? Consider implementing or enhancing technologies like File Integrity Monitoring (FIM), external website scanning, and ensuring logs capture meaningful events.

Lesson 2: Monitoring Needs Broader Scope & Depth.

Problem: Both the entry into your system and the activity or outcome of the compromise (e.g., defaced page, malicious redirect, server sending spam) weren't flagged internally.

Actionable Insight: Expand monitoring beyond basic server health. Monitor website content changes, DNS records, SSL certificate validity, outbound traffic patterns, and public blocklists. Implement comprehensive logging across web servers, databases, OS, and applications, and ensure these logs are analyzed, not just stored.

Lesson 3: Prevention Strategies Need Re-evaluation.

Problem: Someone successfully exploited a vulnerability, or credentials were compromised.

Actionable Insight: This goes beyond just patching the exploited flaw. Review your entire vulnerability management program (scanning frequency, patching SLAs). Assess web application security practices (secure coding, input validation). Re-evaluate access controls, password policies, and the implementation of Multi-Factor Authentication (MFA) everywhere possible. Harden systems based on security benchmarks.

Lesson 4: Incident Response Plans Must Account for External Input.

Problem: How did the organization react to an external notification? Was there a straightforward process? Was it efficient?

Actionable Insight: Review or create an Incident Response (IR) plan. Does it include steps for validating and acting upon external reports? Who is responsible? How quickly can systems be isolated, analyzed, and restored? Practice the plan (e.g., tabletop exercises), including this scenario.

Lesson 5: The "Assumed Breach" Mentality is Non-Negotiable.

Problem: Relying solely on prevention and perimeter defense creates a false sense of security.

Actionable Insight: Shift towards an "assumed breach" mindset. This means proactively hunting for threats within the network, assuming attackers may already be present. It reinforces the need for robust detection, monitoring, and response capabilities, as you can't prevent every attack.

How External Notification Should Change Security Posture & System Security

1. From Reactive to Proactive: The notification should trigger a shift from merely fixing the immediate issue to proactively improving underlying weaknesses.
2. Invest in Visibility: Prioritize tools and processes that improve visibility into system activity and potential compromises (better logging, SIEM, EDR, FIM, external monitoring).
3. Strengthen the Foundations: Double down on security fundamentals, such as timely patching, secure configurations, strong access controls, network segmentation, and user awareness training.
4. Refine Incident Response: Ensure the IR plan is robust, tested, and includes clear steps for handling external notifications efficiently and respectfully.
5. Embrace External Reporting: Please (!) create clear, easy-to-find channels for security researchers to report vulnerabilities (e.g., a security.txt file with a dedicated security email address). Treat these reports seriously and thankfully. Lashing out at the person notifying you is truly bad form. Don't do it.

In Summary…

Finally, I acknowledge that receiving a notification from an unrelated third party that someone has compromised your organization is a humbling experience.

You missed something. You may have missed several things.

Rather than taking it as a defeat, using it as a learning experience is the best way to move forward and improve.

And isn't that really what we all want to do?

-TL
Tom Liston
Owner, Principal Consultant
Bad Wolf Security, LLC
Mastodon: @tliston@infosec.exchange
Twitter (yes, I know… X): @tliston
May 7, 2025

P.S.: I briefly considered adding a link to this blog post to my standard report language. Unfortunately, the uh… rather interesting name of this site might make it a bit more challenging to get my messages taken seriously.

Sigh… The compromises we must make for art.