It’s become almost a cliché to say that cybercriminals are remarkably quick to latch onto the latest trends and technologies and exploit them for their own nefarious gains. The buzz around DeepSeek and its state-of-the-art AI models is no exception. In fact, the past few days have provided a stark reminder that while the tech world is evolving at a breakneck speed, the tactics of online scammers often remain strikingly familiar.

Since the R1 reasoning model of the little-known Chinese startup took the world by storm last week, security researchers have spotted a number of fraudulent attempts to capitalize on its meteoric rise to popularity. Alongside this, DeepSeek has faced intense scrutiny over its privacy and security practices, bringing to light several risks surrounding (not necessarily only DeepSeek’s) AI models.

Here’s a rundown of how fraudsters use DeepSeek’s popularity as a lure for scams and malware, as well as a short recap of some of the key privacy and security issues that have also thrown the spotlight on the company in the past few days.

Scams and malware

One example comes from a user on X who posted some details about a website that mimics the official one and urges visitors to download what poses as DeepSeek’s AI model. Instead, however, clicking it triggers the download of a malicious executable that ESET products detect as Win32/Packed.NSIS.A. 

While the website largely “looks the part”, a keen eye will spot at least one more giveaway beside the URL itself: unlike the “Start now” button on the official website, the fake one says “Download Now”. (DeepSeek has launched mobile apps for both iOS and Android with great success, but you can also use it directly in your desktop browser without needing to download anything.) To further bolster the ploy’s chances of success, the malware is digitally signed by “K.MY TRADING TRANSPORT COMPANY LIMITED”.

Others have also spotted a number of newly-created lookalike domains that aim to trick people into thinking that they have landed on the real thing, but are instead to part them from their data or hard-earned money, including by touting (non-existent) DeepSeek pre-IPO shares.

Another risk has to do with bogus DeepSeek crypto tokens that have surged on multiple blockchain networks, with some reaching market capitalizations of millions of dollars in short order. The company made it clear on X earlier in January that it has not issued any cryptocurrency.

Privacy and security concerns surrounding DeepSeek

Right on the heels of its rapid ascent, DeepSeek said it had itself been the target of “a large-scale cyberattack” that caused it to suspend new user signups.

Meanwhile, cloud cybersecurity company Wiz has found a database belonging to DeepSeek that inadvertently exposed API keys, system logs, user chat prompts and other sensitive information to the open internet. DeepSeek has since locked down the database.

Cybersecurity firms KELA and Palo Alto Networks have found that DeepSeek’s AI models are susceptible to so-called evil jailbreak attacks and their security guardrails can be subverted to generate malicious outputs, including ransomware, as well as fabricate content such as detailed instructions for creating toxins and explosives.

Much like has been the case with TikTok and other Chinese online services, DeepSeek’s data collection practices also garnered scrutiny almost immediately, including from regulatory authorities in the United States, Ireland, Italy and France.

Precautions

Whether it’s a viral new app, a juggernaut social media platform, or an AI tool, cybercriminals are highly adept at weaving thee latest fads and trends into their ploys, ultimately making the ruses more enticing and harder to spot.

To protect yourself from DeepSeek-themed scams, keep your eyes peeled for any email or social media messages that attempt to piggyback off its popularity and push you to click on suspicious links.

Indeed, as AI tools can be harnessed to create highly convincing phishing campaigns and other social engineering attacks, be skeptical of messages that arrive out of the blue, particularly if they offer something too good to be true such as investment opportunities or create a sense of urgency. You’re better off contacting the company or person mentioned in the messages directly via verified channels and navigating to the official website by typing it into your web browser.

Strengthen your online accounts with two-factor authentication (2FA) wherever possible so that it’s far harder for cybercriminals to access your accounts even if they obtain your credentials. Make sure to also use multilayered security software across all your devices that can go a long way towards keeping you safe.

More broadly, when interacting with DeepSeek or, indeed, any other AI model, be mindful of the data you’re entering into it, including names, email addresses and sensitive personal preferences. The same goes for corporate and other sensitive data; the US Navy, for example, has already banned use of DeepSeek among its ranks.

Vulnerability exploitation has long been a popular tactic for threat actors. But it’s becoming increasingly so – a fact that should alarm every network defender. Observed cases of vulnerability exploitation resulting in data breaches surged three-fold annually in 2023, according to one estimate. And attacks targeting security loopholes remain one of the top three ways threat actors start ransomware attacks.

As the number of CVEs continues to hit new record highs, organizations are struggling to cope. They need a more consistent, automated and risk-based approach to mitigating vulnerability-related threats.

Bug overload

Software vulnerabilities are inevitable. As long as humans create computer code, human error will creep in to the process, resulting in the bugs that bad actors have become so expert at exploiting. Yet doing so at speed and scale opens a door to not just ransomware and data theft, but sophisticated state-aligned espionage operations, destructive attacks and more.

Unfortunately, the number of CVEs being published each year is stubbornly high, thanks to several factors:

• New software development and continuous integration lead to increased complexity and frequent updates, expanding potential entry points for attackers and sometimes introducing new vulnerabilities. At the same time, companies adopt new tools that often rely on third-party components, open-source libraries and other dependencies that may contain undiscovered vulnerabilities.
• Speed is often prioritized over security, meaning software is being developed without adequate code checks. This allows bugs to creep into production code – sometimes coming from the open source components used by developers.
• Ethical researchers are upping their efforts, thanks in part to a proliferation of bug bounty programs run by organizations as diverse as the Pentagon and Meta. These are responsibly disclosed and patched by the vendors in question, but if customers don’t apply these patches, they’ll be exposed to exploits
• Commercial spyware vendors operate in a legal grey area, selling malware and exploits for their clients – often autocratic governments – to spy on their enemies. The UK’s National Cyber Security Centre (NCSC) estimates that the commercial “cyber-intrusion sector” doubles every ten years
• The cybercrime supply chain is increasingly professionalized, with initial access brokers (IABs) focusing exclusively on breaching victim organizations – often via vulnerability exploitation. One report from 2023 recorded a 45% increase in IABs on cybercrime forums, and a doubling of dark web IAB ads in 2022 versus the previous 12 months

What types of vulnerability are making waves?

The story of the vulnerability landscape is one of both change and continuity. Many of the usual suspects appear in MITRE’s top 25 list of the most common and dangerous software flaws seen between June 2023 and June 2024. They include commonly-seen vulnerability categories like cross-site scripting, SQL injection, use after free, out-of-bounds read, code injection and cross-site request forgery (CSRF). These should be familiar to most cyber-defenders, and may therefore require less effort to mitigate, either through improved hardening/protection of systems and/or enhanced DevSecOps practices.

However, other trends are perhaps even more concerning. The US Cybersecurity and Infrastructure Security Agency (CISA) claims in its list of 2023 Top Routinely Exploited Vulnerabilities that a majority of these flaws were initially exploited as a zero-day. This means, at the time of exploitation, there were no patches available, and organizations have to rely on other mechanisms to keep them safe or to minimize the impact. Elsewhere, bugs with low complexity and which require little or no user interaction are also often favored. An example is the zero-click exploits offered by commercial spyware vendors to deploy their malware.

Explore how ESET Vulnerability and Patch Management inside the ESET PROTECT platform provides a pathway to swift remediation, helping keep both disruption and costs down to a minimum.

Another trend is of targeting perimeter-based products with vulnerability exploitation. The National Cyber Security Centre (NCSC) has warned of an uptick in such attacks, often involving zero-day exploits targeting file transfer applications, firewalls, VPNs and mobile device management (MDM) offerings. It says:

“Attackers have realised that the majority of perimeter-exposed products aren’t ‘secure by design’, and so vulnerabilities can be found far more easily than in popular client software. Furthermore, these products typically don’t have decent logging (or can be easily forensically investigated), making perfect footholds in a network where every client device is likely to be running high-end detective capabilities.”

Making things worse

As if that weren’t enough to concern network defenders, their efforts are complicated further by:

• The sheer speed of vulnerability exploitation. Google Cloud research estimates an average time-to-exploit of just five days in 2023, down from a previous figure of 32 days
• The complexity of today’s enterprise IT and OT/IoT systems, which span hybrid and multi-cloud environments with often-siloed legacy technology
• Poor quality vendor patches and confusing communications, which leads defenders to duplicate effort and means they’re often unable to effectively gauge their risk exposure
• A NIST NVD backlog which has left many organizations without a critical source of up-to-date information on the latest CVEs

According to a Verizon analysis of CISA’s Known Exploited Vulnerabilities (KEV) catalog:

• At 30 days 85% of vulnerabilities went unremediated
• At 55 days, 50% of vulnerabilities went unremediated
• At 60 days 47% of vulnerabilities went unremediated

Time to patch

The truth is that there are simply too many CVEs published each month, across too many systems, for enterprise IT and security teams to patch them all. The focus should therefore be on prioritizing effectively according to risk appetite and severity. Consider the following features for any vulnerability and patch management solution:

• Automated scanning of enterprise environments for known CVEs
• Vulnerability prioritization based on severity
• Detailed reporting to identify vulnerable software and assets, relevant CVEs and patches etc
• Flexibility to select specific assets for patching according to enterprise needs
• Automated or manual patching options

For zero-day threats, consider advanced threat detection which automatically unpacks and scans possible exploits, executing in a cloud-based sandbox to check whether it’s malicious or not. Machine learning algorithms can be applied to the code to identify novel threats with a high degree of accuracy in minutes, automatically blocking them and providing a status of each sample.

Other tactics could include microsegmentation of networks, zero trust network access, network monitoring (for unusual behavior), and strong cybersecurity awareness programs.

As threat actors adopt AI tools of their own in ever-greater numbers, it will become easier for them to scan for vulnerable assets that are exposed to internet-facing attacks. In time, they may even be able to use GenAI to help find zero-day vulnerabilities. The best defense is to stay informed and keep a regular dialog going with your trusted security partners.

Artificial intelligence (AI) is transforming our world in ways both expected and unforeseen. For consumers, the technology means more accurately personalized digital content, better healthcare diagnostics, real-time language translation to help on holiday, and generative AI assistants to enhance productivity at work. But AI is also used to help cybercriminals be more productive, especially when it comes to identity fraud – the most common fraud type today.

Over a third of banking risk and innovation leaders in the UK, Spain and US cite their biggest challenge today as the rise of AI-generated fraud and deepfakes, making it the number one answer. So how does AI-powered fraud work and what can you do to stay safe?

How does AI-driven identity fraud work?

Identity fraud refers to the use of your personally identifiable information (PII) to commit a crime, such as running up credit card debt in your name, or accessing a bank or other account. According to one estimate, AI-driven fraud now accounts for over two-fifths (43%) of all fraud attempts recorded by the financial and payments sector. Nearly a third (29%) of those attempts are thought to be successful. So how is AI helping the cybercriminals?

There are several different tactics we can highlight:

• Deepfake account takeovers (ATOs) and account creation: Scammers are using deepfake audio and video likenesses of legitimate users to bypass the Know Your Customer (KYC) checks used by financial services companies to verify customers are who they say they are. An image or video of you is scraped from the web and fed into a deepfake tool or generative AI. It’s then inserted into the data stream between user and service provider in so-called injection attacks designed to fool the authentication systems. One report claims that deepfakes now account for a quarter (24%) of fraudulent attempts to pass motion-based biometrics checks and 5% of static selfie-based checks.
• Document forgeries: There was a time when fraudsters used physical document forgeries, such as faked passport pages, to open new accounts in the names of unassuming victims. However, they’re more likely today to do so digitally. According to this report, digital forgeries account for over 57% of all document fraud – a 244% annual increase. Scammers will typically access document templates online or download document images stolen in data breaches and then alter the details in Photoshop. Generative AI (GenAI) tools are helping them to do this at speed and scale.
• Synthetic fraud: This is where scammers either create new identities by combining real (stolen) and made-up PII to form a completely new (synthetic) identity, or create a new identity using just fabricated data. This is then used to open new accounts with banks and credit card firms, for example. Document forgeries and deepfakes can be combined with these identities to increase the fraudsters’ chances of success. According to one report, 76% of US fraud and risk professionals think their organization has synthetic customers. They estimate that this type of fraud has surged 17% annually.
• Deepfakes that trick friends and family: Sometimes, fake video or audio can be used in scams that trick even loved ones. One tactic is virtual kidnapping, where relatives receive a phone call from a threat actor claiming to have kidnapped you. They play a deepfake audio of your voice for proof and then demand a ransom. GenAI can also used in these efforts to help the scammers source a likely victim. ESET Global Security Advisor Jake Moore gave a taste of what is currently possible here and here.
• Credential stuffing (for ATO): Credential stuffing involves the use of stolen log-ins in automated attempts to access other accounts for which you may have used the same username and password. AI-powered tools could rapidly generate these credential lists from multiple sources of data, helping to scale attacks. And they could also be used to accurately mimic human behavior while logging in, in order to trick defensive filters.

What’s the impact of AI-based fraud?

Fraud is far from a victimless crime. In fact, AI-powered fraud can:

• Cause major emotional distress for the individual that’s defrauded. One report claims that 16% of victims contemplated suicide as a result of an identity crime
• Make scams more likely to succeed, eating into profits, which forces companies to put their prices up for everyone
• Impact the national economy. Lower profits mean lower tax receipts, which in turn mean less cash to spend on public services
• Undermine public confidence in the rule of law and even democracy
• Undermine business confidence, potentially leading to lower levels of investment into the country

How to keep your identity safe from AI-driven fraud

To combat the offensive use of AI against them, organizations are increasingly turning to defensive AI tools to spot the tell-tale signs of fraud. But what can you do? Perhaps the most effective strategy is to minimize opportunities for threat actors to obtain your PII and audio/video data in the first place. That means:

• Don’t overshare information on social media and restrict your privacy settings
• Be phishing aware: check sender domains, look for typos and grammatical mistakes, and never click on links or open attachments in unsolicited emails
• Turn on multifactor-authentication (MFA) on all accounts
• Always use strong, unique passwords stored in a password manager
• Keep software up to date on all laptops and mobile devices
• Keeping a close eye on bank and card accounts, regularly checking for suspicious activity and freezing accounts immediately if something doesn’t look right
• Install multi-layered security software from a reputable vendor on all devices

Also consider staying aware of the latest AI-powered fraud tactics and educating friends and family about deepfakes and AI fraud.

AI-driven fraud attacks will only continue to grow as the technology gets cheaper and more effective. As this new cyber-arms race plays out between corporate network defenders and their adversaries, it’s consumers that will be caught in the middle. Make sure you’re not next.

In his talk, Neil Lawrence, the Deep Mind Professor of Machine Learning at the University of Cambridge, tackles the aforementioned fundamental question head-on. With a career dedicated to understanding the intersection of technology and human potential, Mr. Lawrence explores how intelligent systems can complement, rather than replace, human capabilities. At the heart of his talk is the notion of the “atomic human” – a philosophical and technical perspective on what distinguishes us from machines.

Indeed, Mr. Lawrence goes on to examine how technological breakthroughs have forced us to reconsider the traits we hold as inherently human. Each time a machine did something we thought was uniquely human, it cut something away from us. And as this process continues, is there a final frontier – a moment where machines can no longer cut something away from us and take away our capabilities? And if we find what that moment is, does it tell us something about the essence of humanity?

By drawing parallels between AI’s rapid evolution and some of history’s greatest human achievements – from the Apollo missions to Amelia Earhart’s daring flights – Mr. Lawrence illustrates how technology has been a tool to augment human ingenuity, not a force to erase it. With AI reaching further into areas once thought unreachable, the challenge lies in ensuring that we shape it as a tool for empowerment, rather than let it redefine our identity.

ESET’s commitment to promoting scientific innovation and progress is seen in its ongoing efforts to foster a deep appreciation for science, celebrate the power of groundbreaking research, and connect with leading thinkers in technology and science. ESET recently partnered with Starmus, the global science communication festival, and brought its 7^th edition to Bratislava, Slovakia, in May 2024.

The festival featured a number of thought-provoking perspectives from some of the planet’s foremost thinkers. You can now relive the experience from the comfort of your home and get a taste of how the power of technology is being harnessed to tackle some of the most pressing challenges facing the world today.