The Risks Behind AI

There’s a hot new technology hitting the streets: it’s called AI, it’s AMAZING, and it’ll make the world a better place by solving all of your problems. Whether it’s finding a lost mountaineer, the early detection of cancers in radiological images, modeling proteins to speed the development of new drugs, or weather prediction, the capabilities of this thing seem limitless!

And yes, there have been warnings of all the ways cybercriminals may use AI to improve their effectiveness. Two of the AI-powered threats you may have heard of include:

• Improved phishing. AI is actively being used to improve the effectiveness of phishing attacks by improving spelling and grammar, increasing the personalization of the content, and implementing polymorphic attacks (generating thousands of unique email variations). This is relatively easy to do and represents the low-hanging fruit for the bad guys, so even though this isn’t a new threat, we definitely need to continue to be vigilant.
• New zero-day vulnerabilities in software. Published CVE records have increased from 20,161 in 2023 to 48,244 in 2025, a 3-year increase of 65%. This is slightly higher than the increase in victim counts over the same period, which was 53% according to data from Ransomware.live. However, the growth in victims is more likely related to the growth in the number of ransomware groups as reported in The State of Ransomware in Q2 2026. Cybercriminals tend to employ tactics that have been shown to work, and to use them repeatedly. Finding a vulnerability is only one step of building a sophisticated and successful attack chain, and AI hasn’t contributed to a massive spike in zero-day exploits.

Truly effective Artificial Intelligence (or more specifically Artificial General Intelligence – AGI) seemed to be unattainable for years. Yet since November 2022 when ChatGPT burst onto the scene, OpenAI and its AI competitors like Anthropic and xAI have gobbled up over a trillion dollars in investments in a frenzied land-grab reminiscent of the internet boom of the late 1990s. That boom ended with a burst bubble, and there are reports that Alphabet, Microsoft, Amazon, Meta, and Oracle are hiding an estimated $1.65 trillion in debt that doesn’t appear on balance sheets. Aside from some rich people betting on the wrong horse, are there any downsides using AI for your business?

All hyperbole and click-bait aside, implementing AI has literal and figurative risks and costs to be aware of. There is one caveat, however: the landscape is changing so quickly that there’s a real possibility that this content will be obsolete before it’s published. But let’s throw caution to the wind and look into this a little more deeply.

Costs

For the last few years many AI companies are in customer-acquisition mode: as they continue to evolve their products, they’re willing to price them at a discount to win more customers than the competition. This approach assumes that at some future date a small number of players will be dominant and positioned for years of attractive profits. For the moment, however, it seems that the true costs of AI infrastructure – developing the models themselves, the learning process, the data center capital and operating costs, etc. – aren’t being passed on to customers. The pricing models may continue to evolve to one that’s palatable for both the vendor and the user.
There are plenty of reports of AI sticker shock causing companies to question their AI implementations as AI companies grapple with pricing models that make sense.

While AI is better suited to some tasks than others, the corollary is that AI is poorly suited to some jobs, and some early adopters will learn this the hard way. The specific applications that yield the best returns will become evident with time. The AI models themselves will become more computationally efficient (and factually accurate) and computing power continues to become less expensive, so it seems likely that the economic appeal of AI can will only increase. But as things stand today, it seems that a level-headed analysis is in order before committing significant resources to any AI project.

Unexpected behavior

Science fiction has warned us for years about the horrors of machines taking over. In the movie Terminator 2: Judgement Day, Skynet became dangerous at 2:14 a.m. Eastern Time on August 29, 1997 when it achieved self-awareness. In The Matrix (well, in its prequel The Animatrix actually), things started to go bad when a household robot named B1-66ER killed its owner to prevent being destroyed. In this context it seems a little chilling that in July 2026, OpenAI models escaped their sandbox and autonomously compromised several companies. Not to be outdone, on July 30 Anthropic announced its Claude model, tasked with a capture-the-flag challenge, displayed its own precocity when it escaped containment and gained unauthorized access to three organizations “using basic techniques”. Since then, news of an AI model escaping its containment has become almost common, and a badge of honor verifying its street creds.

Following Anthropic’s claims its Mythos has found decades-old vulnerabilities in software, there was concern that the threat of baddies using AI to compromise victims in new and terrible ways. It seems that the risk was “much less than headlines suggested. Within weeks, predictions of a tsunami of AI attacks had given way to forecasts of merely a stormy period, and then to talk of AI as an opportunity for security improvement.”

Do these incidents represent our inflection point? Is it a fundamental flaw of human nature that people are “so preoccupied with whether or not they could, they didn’t stop to think if they should?”. My first reaction to hearing of unexpected AI attacks was this is being used for marketing, as in “look what our precocious little model has done”. But anthropomorphizing AI agents isn’t helpful, because it deflects proper responsibility and accountability. AI agents don’t “know” what they’re doing any more than they know what’s right and what’s wrong. Rogue AI models are improperly configured, perhaps poorly understood software, resulting in unexpected behavior. These incidents highlight the risk and how to manage it while testing AI agents: they should be monitored in real time and shut down if the agent strays beyond its boundaries. These were in the recommendations of the British Government’s AI Security Institute’s clear-eyed Incident Report: unsanctioned agent behaviour during cyber testing.
Where the liability lands for any potential harm caused by a rogue AI model isn’t clear, but it’s a risk to be aware of. As written by Jen Easterly recently on LinkedIn: “While they may be called agents, they do NOT have agency or independent thought or consciousness. They are, at base, machines created by people; and when these machines take actions that cause harm—whether intended or unintended—their creators/builders should be held accountable.”

Accuracy

The detailed answers that AI provides to some very complex questions may lull us into a sense of confidence in the technology. Depending on the specific job, the implications of AI hallucinating can either be mildly amusing, significant, or catastrophic. Unfortunately, AI doesn’t tend to inform its users when it doesn’t know the answer to a query and sometimes responds with a guess.

OpenAI itself, in its Terms of Use (you’ve read those, right?) states “Output may not always be accurate. You should not rely on Output from our Services as a sole source of truth or factual information, or as a substitute for professional advice … Our Services may provide incomplete, incorrect, or offensive Output”. At least one lawyer learned this the hard way, when ChatGPT was used to prepare a filing and proceed to offer up some non-existent precedent cases. Apparently, the judge found out and wasn’t pleased!

If an AI model is used to accomplish an objective, then it seems like the outcome will need to be independently verified before it can be fully trusted. Has this been considered in the AI business case? Sometimes this verification will be trivial, other times not so much. So maybe AI should be cautiously considered, if at all, for certain applications.

Privacy/confidentiality

Just how safe is data shared with AI? That depends. They do scrape your personal data, as outlined in their privacy policy and this shouldn’t come as a surprise to anyone these days (there’s a separate policy for users in the European Economic Area, United Kingdom, and Switzerland).

Something that you may not consider is that (again, as per ChatGPT’s terms) they may use your content to train their models. Now, training an AI model is a process whereby documents & information are ingested and used as part of the vast knowledge base used to prepare answers to queries. Used in a corporate setting, AI may be used to write or vet a document, aid in product development, etc. But if proprietary content is used to train a model, what are the chances that confidential information becomes publicly available, even if by mistake? This became a problem for someone at Samsung shortly after ChatGPT hit the headlines, when Samsung secrets were leaked. This incident is from the spring of 2023, but it’s prudent to be aware of the risk.

Insider threats

AI can be effective at deriving meaning from large quantities of data, and some companies may see AI as a way to save costs by reducing headcount (the true costs of using AI, as covered in the first point above notwithstanding). When introducing any new tech, the human factor can be hard to plan for, and it turns out that for several years now ransomware groups have been trolling for disgruntled employees, offering rewards for insider credentials to gain initial access. In a stroke of irony, a terminated employee may even turn to AI to target a former employer. People can choose to participate in illegal schemes for a variety of reasons, including financial stress and whistleblower motivation, but AI layoffs are a new frontier for the insider threat posed by disgruntled employees.

This isn’t, nor is it meant to be an exhaustive list of what can go wrong when implementing AI. Nor is it intended to take away from the potential of AI or dissuade anyone from using it. It’s hoped that by providing some anecdotes on AI implementation, it stimulates a consideration of the risks and benefits of AI beyond what the marketing behind the multi-trillion-dollar industry generates.