The Erosion of Engineering Intuition in the Age of AI SREs

By Sylvain Kalache (Rewritten)

Back in 2012, during my tenure as an SRE at LinkedIn, I conceptualized a system capable of autonomous healing and learning from past failures. At the time, AI wasn't advanced enough to move beyond a prototype. Today, however, that vision is a reality.

Modern AI tools can now autonomously:

• Analyze incoming alerts.
• Formulate potential hypotheses.
• Query system telemetry.
• Link incidents to recent deployments.
• Execute the necessary fixes.

While this progress is impressive, it introduces a critical risk: we are losing our connection to the systems we build.

The Paradox of Automation

As these tools become more proficient at resolving "easy" or routine issues, human engineers lose the opportunity to practice. This creates a dangerous gap. When a high-severity, ambiguous incident occurs—one that defies automation—the engineers tasked with fixing it will be out of practice.

Essentially, AI handles everything AI handles the easy things, leaving humans with only the most grueling challenges.

The "AI SRE" Dilemma

While "AI SRE" is a term I find slightly distasteful, these tools are undeniably helpful. It feels like magic when an AI resolves a capacity issue at 3 AM, allowing you to sleep through the night. However, routine incidents are the "safe" training grounds where responders develop a gut feeling for system behavior.

This phenomenon was identified by human-factors researcher Lisanne Bainbridge in her 1983 seminal work:

"The Ironies of Automation" posits that automation reduces the operator's chance to practice routine tasks, yet still holds them responsible for managing abnormal or novel situations.

Consequently, Bainbridge argues that operators actually require more skill and more rigorous training than they did before automation existed.

The Impact on Metrics

I predict a divergence in our performance metrics. While the average Mean Time to Recovery ($\text{MTTR}$) will likely decrease, the time to resolve complex "black swan" events will spike.

$\text{Average MTTR} \downarrow \quad \text{vs.} \quad \text{Complex Incident Resolution Time} \uparrow$

Lessons from the Cockpit

The aviation industry provides a blueprint for solving this. Pilots rely heavily on automation, yet they are the final line of defense for catastrophic failures (e.g., stalls, engine failures, or instrument malfunctions).

Consider the rarity of engine failure: modern turbines might see fewer than one shutdown per 100,000 flight hours. A pilot could spend their entire career without seeing one in real life. Yet, they must react perfectly when it happens.

The Cost of Failure: In the case of TransAsia Airways Flight 235, a propeller autofeathered shortly after takeoff. Despite the plane's ability to fly on one engine, the crew misdiagnosed the situation. The result was a stall and a crash just 117 seconds after the initial warning.

To prevent this, the FAA mandates that captains undergo recurrent training or proficiency checks every six months, simulating emergencies like engine failure during takeoff.

Implementing "Incident Simulators" for Software

While a software crash rarely costs lives, the pursuit of mastery remains essential. At Rootly, we've partnered with Uptime Labs to bring aviation-style simulation to engineering.

In these simulations, engineers act as the Incident Commander during a fake e-commerce outage. They must:

1. Use observability tools to find the root cause.
2. Coordinate with stakeholders (simulated by LLMs) in Slack.
3. Manage pressure from the CEO and customer support teams.

This allows them to practice critical thinking under pressure, communication, and coordination.

Can AI be the Teacher?

Some suggest using AI to explain its reasoning—showing the signals it tracked and the evidence it used. However, observation is not the same as execution.

I learned this firsthand while running a software engineering school based on progressive education. When Dropbox noted that our graduates struggled with troubleshooting, I stopped giving lectures and started giving them broken infrastructure to fix. Hands-on experience beats passive instruction every time.

Combating "Comprehension Debt"

As LLMs take over more operational tasks, teams risk accumulating comprehension debt—the widening gap between the system's actual complexity and the engineer's understanding of it.

To stay ready, teams should adopt an On-Call Readiness Checklist:

• Regularly interact with the system manually.
• Intentionally trigger unfamiliar failures.
• Practice high-pressure coordination for SEV0 events.
• Conduct regular tabletop exercises and chaos engineering.

def check_readiness(engineer):
    if engineer.last_simulation_date > 6_months:
        return "Skill Decay Warning: Schedule Simulator Session"
    return "Ready for Incident Response"

The ultimate irony of automation is that the more successful it is, the less prepared we are for the moment it fails. We must fight this decay with intentional, hands-on practice.