Machine learning–aided scams

[harlan4096]

New technologies are clearly changing the world, but not the human psyche. As a result, evil geniuses are devising new technological innovations to target vulnerabilities in the human brain. One vivid example is the story of how scammers mimicked the voice of an international CEO to trick the head of a subsidiary into transferring money to shady accounts.

What happened?

The details of the attack are unknown, but the Wall Street Journal, citing insurance firm Euler Hermes Group SA, describes the incident as follows:

1. Answering a phone call, the CEO of a U.K.-based energy firm thought he was speaking with his boss, the chief executive of the firm’s German parent company, who asked him to send €220,000 to a (fictitious, as it later turned out) Hungarian supplier within an hour.

2. The British executive transferred the requested amount.

3. The attackers called again to say the parent company had transferred money to reimburse the U.K. firm.

4. They then made a third call later that day, again impersonating the CEO, and asked for a second payment.

5. Because the transfer reimbursing the funds hadn’t yet arrived and the third call was from an Austrian phone number, not a German one, the executive became suspicious. He didn’t make the second payment.

How was it done?

Insurers are considering two possibilities. Either the attackers sifted through a vast number of recordings of the CEO and manually pieced together the voice messages, or (more likely) they unleashed a machine-learning algorithm on the recordings. The first method is very time-consuming and unreliable — it is extremely difficult to assemble a cohesive sentence from separate words without jarring the ear. And according to the British victim, the speech was absolutely normal, with a clearly recognizable timbre and a slight German accent. So, the main suspect is AI. But the attack’s success had less to do with the use of new technologies than with cognitive distortion, in this case submission to authority.

Psychological postmortem

Social psychologists have conducted many experiments showing that even intelligent, experienced people are prone to obeying authority unquestioningly, even if doing so runs counter to personal convictions, common sense, or security considerations.

In his book The Lucifer Effect: Understanding How Good People Turn Evil, Philip Zimbardo describes this type of experiment, in which nurses got a phone call from a doctor asking them to inject a patient with a dose of medicine twice the maximum allowable amount. Out of 22 nurses, 21 filled the syringe as instructed. In fact, almost half of nurses surveyed had followed a doctor’s instructions that, in their opinions, could harm a patient. The obedient nurses believed they had less responsibility for the orders than a doctor with the legal authority to prescribe treatment to a patient.

Psychologist Stanley Milgram likewise explained the unquestioning obedience to authority using the theory of subjectivity, the essence of which is that if people perceive themselves as tools for fulfilling the wills of others, they do not feel responsible for their actions.
...

Continue Reading