Introduction
Thereâs a question the security industry still hasnât answered honestly: if organizations are spending more on cybersecurity than ever before, why are the attacks causing the most damage getting simpler? Not simpler in their consequences. Simpler in their method.
The incident that costs a company $100 million increasingly doesnât start with a zeroâday exploit. It starts with a phone call, a forged invoice, or a video meeting with people who donât exist. The industry spent two decades building better locks. Attackers learned to impersonate the locksmith. And now, with AI in the picture, they can do it at scale, at speed, and at a cost that would have seemed absurd five years ago.
The Shift That Changes Everything
Classical intrusion has a technical logic: find a vulnerability, exploit it, move laterally, achieve the objective. Defensive controls are built to interrupt those steps. Psychological exploitation works differently. The attacker isnât fighting your systems â theyâre moving through them. Legitimate processes, legitimate credentials, legitimate communication channels, all producing illegitimate outcomes. Thereâs no payload to catch and no obvious anomaly to escalate. At every step, it looks like normal operations.
None of this is conceptually new. Social engineering predates computing by centuries. Whatâs changed is its position in the stack. For serious threat actors, psychological manipulation is no longer a fallback when technical approaches fail. Itâs increasingly the cleaner path: cheaper, faster, harder to detect, and â critically â harder to prosecute. AI isnât introducing this shift, but it is accelerating it in ways that are easy to underestimate.
Three cases make the point:
- The Forged Document: When Routine Becomes the Vulnerability
In one wellâdocumented case, a fraudster spent years tricking Google and Facebook into paying forged invoices for work that was never performed â not by breaking into their systems, but by sending paperwork that looked exactly like routine business. The finance teams werenât careless. Thatâs the wrong lesson. They were following real procedures against someone who had studied those procedures closely enough to satisfy them. The real work happened before the first invoice was ever sent: understanding vendor relationships, invoice formats, approval thresholds, and the range of amounts that would feel routine enough not to trigger additional review. Thatâs what makes the attack interesting. It didnât bypass verification. It passed it.
Highâvolume routine processes donât get scrutinized evenly because they canât â the economics of operational workflows donât allow for it. Attackers find the range where extra verification feels unnecessary and stay comfortably inside it. The answer isnât âverify everything.â That collapses in the real world. The real challenge is identifying which actions are both highâvalue and hard to reverse, then forcing a confirmation path that exists entirely outside the transaction itself. A simple outâofâband callback to a preâestablished contact â not one introduced during the exchange â would likely have stopped this attack before the first payment cleared. - The Phone Call: When Your Own Processes Become the Attack Surface
Scattered Spider is worth studying carefully because they often do not begin by compromising systems. They begin by compromising the people whose job is to restore access when something goes wrong. The playbook is straightforward but devastating: identify an employee through public sources, assume their identity, and call the helpdesk with a plausible story under time pressure. The helpdesk, doing exactly what itâs supposed to do, resets the credentials. From there, access to identity systems can open everything else.
What makes this group especially instructive is that they donât succeed through a single well-crafted call. Reports like those from CISA describe a pattern of iterative social engineering, where attackers use early interactions with helpdesks or employees to understand internal processes, terminology, and verification flows. Each exchange increases their situational awareness and makes the next one more credible. By the time the real request arrives, they are no longer improvising â they are operating with enough context to sound like a legitimate employee working through a routine issue.
The uncomfortable truth is this: the attack often works not because the helpdesk agent made a bad decision, but because they made the decision the institution rewards. Resolution speed is the metric. Friction is penalized. An agent who slows down to verify more carefully can look worse â by the systemâs own measurements â than the one who resets access quickly and moves on. You cannot train your way out of that. If the incentive structure rewards speed over assurance, the failure is already designed in. The defense has to be structural. Highâprivilege actions need verification steps that canât be skipped, regardless of how normal the request sounds or how much pressure is applied. - The Deepfake Call: When Seeing is No Longer Believing
The Arup case marks a genuine inflection point. The mechanism is the same as in the previous examples â impersonation, manufactured authority, pressure toward an irreversible action. What changed is that AI removed one of the last intuitive friction points most people still trusted: visual confirmation. An employee received what appeared to be a request from a senior colleague. Skeptical â and doing what many organizations would still consider the right thing â they verified visually on a video call with people who looked exactly as they should. They approved the transfer. None of the faces on that call were real.
No systems were compromised, and no alerts fired. Everything unfolded through the same channels the organization uses every day, without producing a single signal that might raise suspicion. The attack didnât need to touch the infrastructure to move straight through it. Arupâs CIO called it âtechnologyâenhanced social engineering.â Accurate, but it undersells the bigger point.
The Cost of a Convincing Lie Just Collapsed
When the Arup attack happened, building a convincing realâtime deepfake still required meaningful resources and expertise. That constraint is gone â an attacker with a laptop and minimal effort can now impersonate almost any publicâfacing executive. Today, voiceâcloning tools can recreate someoneâs voice with less than thirty seconds of audio. That material is everywhere â conference talks, interviews, earnings calls. And the result isnât just âconvincingâ; itâs accurate enough to slip past the intuitive âdoes this sound like them?â filter most people still trust.
The visual side is moving just as quickly. Realâtime faceâswapping in video calls is no longer experimental; it runs on consumer hardware, and the barrier is no longer technical expertise so much as attention and intent. In 2024, a finance worker in Hong Kong transferred $25 million after a video conference with deepfake versions of multiple colleagues. In another case, the CEO of a British energy company approved a âŹ220,000 transfer after receiving a call from what sounded like his parent companyâs director. Both identities were entirely fabricated.
What changed wasnât just the volume of attacks. It was the quality floor. The uncomfortable implication is simple: the verification methods people instinctively trust â voice and visual confirmation â are no longer reliable for decisions that matter.
What AI Changes â And What it Doesnât
These cases largely predate the current wave of generative AI. That matters because it helps isolate what AI actually changes versus what was already broken. It doesnât change the core logic of the attack. Impersonation, manufactured authority, and social pressure arenât new. What it changes is the economics â and that shift ripples through everything else.
Voice cloning, deepfakes, and highly personalized phishing used to be expensive, slow, and technically demanding enough to limit their use. That constraint has largely disappeared. What once required a specialized team can now be done with a prompt and a bit of iteration. The pool of capable attackers is expanding, while the quality of what they can produce continues to improve.
The Preparation Problem Nobody Talks About
Organizations emit a constant public signal: earnings calls, job postings, press releases, executive interviews, social media activity. That signal reveals where pressure is concentrated, who holds access, and when timing matters. A model can absorb months of public communication, reconstruct internal patterns, and generate messages that sound as if they came from inside the organization. And it does this in minutes, not weeks, which completely changes the scale of the problem.
This is why even job postings have become part of the attack surface. A hiring push for identity engineers can signal exactly where systems are under strain â and where one more convincing request might slip through.
The Agent Layer: Whatâs Coming Next
Most attacks today still have a human in the loop â making calls, writing emails, steering the interaction. That is already starting to change.
Autonomous AI agents capable of researching, communicating, and executing multiâstep actions point toward a near future where entire attack chains can be automated and run in parallel. Weâre not fully there yet, but the direction is clear enough that ignoring it would be a mistake.
The Asymmetry That Defines the Problem
An attacker needs to get one interaction right â once, with the right person, at the right moment. A defender needs to get thousands of interactions right every day, under pressure, with limited attention and competing priorities.
AI is making that single convincing interaction dramatically cheaper. It is not making the rest any easier. And unlike traditional security gaps, this one doesnât close with better tools. It widens as those capabilities improve.
The Pattern Underneath it All
Strip away the details and the structure is consistent. Each attack starts with public information. Each builds a pretext designed to satisfy verification, not bypass it. Each exploits routine, trust, or operational pressure. And each ends with a single irreversible action.
These attacks donât look suspicious and succeed anyway. They look legitimate â by design. Defenses built to catch what looks wrong will keep missing attacks built to look exactly right.
What Effective Defense Actually Looks Like
Fixing the incentive structure matters more than refining the training. If speed is consistently rewarded over assurance, failure is a matter of time. For highâvalue or irreversible actions, verification needs to exist outside the request itself â not as a recommendation, but as a requirement that canât be bypassed by a convincing story or a familiar voice.
The mistake many organizations make is trying to add friction everywhere. That doesnât work; it gets worked around. The real challenge is knowing exactly where friction is nonânegotiable, and enforcing it there consistently. And perhaps most importantly, defenses need to reflect how these attacks actually unfold. Real incidents involve ambiguity, pressure, and incomplete information. If exercises donât include those conditions, theyâre preparing teams for a cleaner scenario than the one theyâll face.
The Uncomfortable Conclusion
Most organizations arenât being hacked. Theyâre being operated. And that distinction matters, because it points to a type of failure that canât be fixed with more tools or more dashboards. Itâs a failure of structure â of processes that work exactly as designed, just not under the conditions they now face.The most expensive incidents rarely begin with a technical exploit. They begin with workflows that assume trust is stable, predictable, and safe to lean on. The people who fall for these attacks are not careless; theyâre following procedures built for a world where a familiar voice or a routine request was enough to feel confident.
Awareness helps, but it doesnât change outcomes on its own. What does make a difference are the mechanisms that force verification when it matters, the incentives that reward caution instead of speed, and the processes that donât collapse the moment someone shows up with an urgent story and the right tone of authority.
Because the next major incident probably wonât arrive as malware. It will arrive as a normal request, in a trusted channel, from someone who looks â and sounds â exactly like the person youâve always trusted.






