The Three-Second Theft: Why AI Voice Fraud Outruns Every Defence

Article URL: https://smarterarticles.co.uk/the-three-second-theft-why-ai-voice-fraud-outruns-every-defence

Comments URL: https://news.ycombinator.com/item?id=48920432

Points: 135

# Comments: 171

Hacker News 讨论

136 points · 174 comments · 查看原帖

saltcured
Sad times are coming for a lot of families and individuals. It isn't just that technology is upending our naive ideas of trust and authenticity. This is, essentially, the broad class of "confused deputy" attacks. And the robust mitigation is to disempower the easily confused deputy, rather than to think you can block confusing signals. A looming problem with shifts in demographics and family structure is that many people will be slipping into cognitive decline without a formal transition to address their incompetence. Sadly, there is a point where the older person really needs to permanently delegate important decision-making to a trusted third party. They should no longer be legally empowered to authorize funds transfers, sign contracts, or even make medical decisions. We're not really setup to handle this well. Not at the systemic level of protecting people from themselves, and not at
offsign
Sounds like AI is just greasing the wheels of a long established 'grandparent scam'... goes something like this: 1) voice one: young adult calls, sobbing 2) grandparent inquires with a name... "Ben, is that you?" 3) voice one: "Yes grandma, it's me, Ben... I'm in trouble, please don't tell mom 4) voice two: "Hello, I'm attorney..." My grandmother fell victim to this almost 20 years ago, which only stopped when Western Union refused to let her continue sending wires... she was forced to call her daughter (at which point they just called my brother.) Our takeaway (at the time)... the voice doesn't even need to be terribly accurate, since the original interaction is brief / somewhat inaudible over the tears. Typically just requires an older vulnerable adult, a lucky strike with the initial setup (e.g. grandparent actually has a grandkid), and a lot of high pressure / duress salesmanship.
spenvo
This old post of mine may be of interest, where I point out: "After a bit of threat modeling, it becomes apparent that future spearphishing robocalls may not directly con you, but rather “farm” your voice data by asking you benign questions, and use that to train a voice model to penetrate more deeply into your network." A lot of this writing has been on the wall forever and many (I'm sure otherwise smart people in) mission critical industries like banking, ISPs, and more refused to even acknowledge the risks. 2021 - "Despite the prevalence of deepfake audio tech, banks and ISPs rush ahead with “voice print” authentication" https://keydiscussions.com/2021/12/07/despite-the-prevalence... ends with a section called "The next crisis: robocalls that spoof the voices of victims at scale"
m463
This makes me REALLY question "this call may be recorded for quality purposes" Now other businesses are starting to reference their privacy policy at the beginning of a call, which leads me to think there are many more uses popping up for our recorded voice. I'm sure a certain percentages of recordings of our voice "to stop fraud" might be used to start fraud.
imoverclocked
So, you answer your phone to the scam and… now they have your voice too. Talking on the phone is now an unmitigated liability.
skybrian
This blog is kind of an interesting hybrid: > Every article published on SmarterArticles is authored and editorially controlled by Tim Green. Artificial intelligence tools are used within a structured and supervised workflow as research and drafting instruments. All arguments, framing decisions, source selections, and final publication choices remain human-directed and under my full responsibility. There are references at the bottom, but I would have preferred direct links or footnotes within the article. Also, direct quotes are nice. I didn’t notice any glaring AI cliches.
wrs
Everyone suffers from this, not just the scam victims. I opened a bank account for a new business this year, and the friction for doing perfectly normal things was ridiculous due to the bank’s paranoia about scams. I couldn’t even make an initial deposit from my previous business, or transfer money to my personal account, without triggering a fraud alert and freezing the entire account (couldn’t even log into the bank website) until I could call and verify that it really was me on both ends of the transaction.
attila-lendvai
> It requires, second, regulating the supply of the weapon. [...] i guess even local models can do this now, especially in non-interactive mode. so, i have a hard time reading this part as mere naivitee, as opposed to enemy propaganda in support of mandatory digital ID's for everything. or for straight out criminalizing "unauthorized" compute altogether?