I Thought Prompt Injection Was A Vaccine Thing Until I Discovered AI

Information Week

The generative AI hype bubble definitely deflated significantly during the summer. Gartner calls it the "trough of disillusionment" in the "AI hype-cycle" and points to 'agentic AI' and 'AI-native software engineering' (aka 'vibe coding') as (somewhat) distinct new entrants. Why the disillusionment? Well, on top of earlier risk management warnings, we've heard lots about the fact that inaccuracy, bias and hallucination are features of generative AI, rather than bugs, as explained very well in the 'myth busting' post by The Guardian. But what we're really hearing about now are the security vulnerabilities, which seem even more problematic for agentic AI and vibe coding. In fact, the more applications that sit on top the worse the problem gets.

AI CEOs Get Cold Feet

Having greedily rushed to get their open generative AI services to market as 'minimum viable products' leaving all the shortcomings as 'externalities', the AI bosses spent this summer pretending to care as a way of demonstrating the 'true power' of what they'd foolishly unleashed. 

Altman signposted the fraudster's charter, and later found himself on the receiving end of a tragic wrongful death suit in connection with the death of a teenage user of Chat GPT between September 2024 and January 2025. This appears to have led the CEO of Microsoft AI to begin his own hand-wringing over the illusion that you seem to be having a 'conversation' with an AI, which he couldn't resist 'branding' as 'Seemingly Conscious AI'.

Humans replaced... then rehired

Meanwhile, overly enthusiastic adopters of chatbot functionality found themselves rowing back on their plans to nuke their customer service teams. Klarna performed such a volte 'farce', as did Australia's Commonwealth Bank, making it all the more bizarre that Microsoft should publish some, er, artificial research claiming to be able to 'predict' which jobs will be replaced.

Even if it were possible to run 'agentic AI' processes that do a lot of the mundane work "...before escalating..." more complex issues, to whom would they escalate those issues? Experienced senior managers? When they retire, who will have gained the experience to replace them? 

Advisory AI?

While the state of Illinois became the first to ban the use of AI to provide therapy, the UK government remain undeterred, announcing its decision to enable the unwitting British 'populace' to use agentic AI for everything from employment advice to obtaining driving licences...

Meanwhile, lawyers have had to be warned again about the fact that open generative AI tools produce fake law.

And before you start thinking of simple processes that AIs could fulfill, it's worth pointing out that ChatGPT-5 still fails at such seemingly straightforward tasks as creating an alphabet chart with each letter represented by an animal whose name starts with that letter; maps and decision trees, among other things. But a generative AI will still boast that it - or others - can do such things. For instance, when searching for an example of poor map making, Google 'AI Overview' produced the following slop (my emphasis):

The assertion that "AI can't do a map of Europe" is false, but it highlights the limitations of generative AI in producing accurate, detailed maps, which often contain errors like misplaced cities, incorrect country borders, and inaccurate iconography. While AI has access to a vast amount of data and can create maps that look plausible, it struggles with the precision and reliability required for a complex geographical representation.

AI Insecurity

But by far the worst issues are to do with security, including 'poisoned calendar invites' containing malicious prompt injections. This is a grave issue that 'better prompting' cannot fix, and the open architecture of open generative AI militates against a 'zero trust' approach which is unlikely to be commercially viable in any event. 

Rogue AIs can only be shut down

An 'agentic AI world' would be wide open to malicious prompt injections, hallucination, bias and error. So what, you might say. We could just shut it down. Yet researchers have found that some AIs can resist shutdown and find ways to keep working on their latest tasks. 

And if you've replaced your 'traditional' staff, systems and business processes with an AI, what then? 

Further reading:

For the AI sceptic's view, I follow Professor Barry O'Sullivan, Denis O., Axel C., Simon Au-Young and Georg Zoeller. Your mileage may differ ;-)

Koup Aid: Trump's Lethal Brand Of Soft Drink Has Killed Populism

The murder-suicide of Jim Jones' 900 cult followers in 1978 was "the largest single incident of intentional civilian death in American history" - until Donald J Trump began wreaking havoc on society with his own lethal brand of soft drink: 'Koup Aid'. Those of his MAGA cult followers who managed to avoid injecting themselves with bleach now face unemployment, wealth & pension evaporation, incarceration, deportation, bankruptcy and/or starvation, thanks to his mindless, lawless public cost-cutting and destructive tariffs. And you can add to that list the many politicians around the globe who'd pinned their electoral hopes on populism as a route to power. That's over now. A new political strategy is required. 

Trump's global distribution of Koup Aid has been undermining populist regimes the world over since Brazil's Bolsonaro lost in 2018. Argentina's very own chainsaw-wielding maniac faces his own net disapproval ratings. And right 'whinge' leadership hopefuls have just lost their national elections - and their own seats - in Canada and Australia.

While Britain's own Brexidiot populist provocateur, Nigel Farage, continues to enjoy modest electoral success, that's only in a few of his country's predominantly white constituencies who actually suffer little from the 'channel crossings in dinghies' that he ironically clings to for his own political survival (we fear the unknown, after all). Last year's Labour landslide shows that the rest of the country isn't fooled on that front. And the Australian populist parties' own doomed electioneering demonstrates that directly copying Trump's DOGE approach to government efficiency, the "Make [your country's name here] Great Again" slogan and the promise of 'border control' do not carry you into the nation's top political job.

Nope, the populists must find a new route to political power. Gone are the days when the blithering idiots in the Conservative Party, for example, could try to 'out-Nigel' Nigel. And they can only go so far right, anyway, before they meet the blithering idiots on the far left, as Corbynites revealed. 

Such is the nature of what I like to call the Political Opportunity Donut. The Trump experiment in America - and recent electoral victories everywhere else - highlight the political vacuum that has emerged in the 'centre' of western democracies. And 'nature abhors a vacuum', as Aristotle observed, so every aspiring political leader worth their salt is now rushing to fill it.

Of course, the political Centre is also a tough place to be, as Tony 'Bliar', 'Wavy' Dave Cameron and Nick 'Tuition Fee' Clegg all found to their eventual cost in the UK. It's only so long before populists with their phoney issues and respective lethal cocktails emerge on the left and right to try to reclaim the ensuing vacuums elsewhere on the Donut. 

So it always goes. We are where we are.

I must say that I enjoy this Centrist phase. It's when genuine problems get identified and solved. The decent political leader need only focus on that process and demonstrate progress, because it's hard to argue with actual solutions. People even generally enjoy helping. Morale is boosted, which brings its own tailwind.

Of course, there are inevitably heated arguments about which socio-economic problems to solve first, their root causes and potential solutions; and which get more resources than others. But those are political arguments worth having, instead of washing down meaningless slogans with Koup Aid.

Our mistake is to allow politicians to distract us from the problems that remain.

What To Do About The Coup

It's clear that Trump 2.0 is a coup: an illegal and overt attempt to seize control of the United States government. Rather than operating as President (other than in name), Trump's plans involve him running the US government like a private corporation, with himself as chairman and Musk as CEO. Yet while we are in this 'move fast and break things' phase of American politics, Trump and his co-conspirators have opened the way for Congress to turn the tables on them in relation to each of their three key tactics, and it must do so swiftly. 

First, as Vance himself recommended, Congress, the courts and all 'paper protections' are being ignored. This means that Congress should be free to act against Trump, Vance, Musk and the other co-conspirators as it sees fit. Trump (then Vance) could be impeached under the 25th Amendment. Solitary confinement in Guantanamo Bay awaits, pending their trial under Chapter 115 of the US Code.

Second, in their efforts to purge the democracy and dismantle institutions every civil servant in the administrative state is either being fired or replaced by Trump's people. Congress can therefore treat every person who agrees to replace a civil servant as a conspirator in the coup.

Third, in the course of seizing control of government media and information to maintain power, all Government IT and payment systems are being expropriated and modified or replaced using private software and systems without recompense. This may be very difficult to undo, so Congress could simply nationalise all the replacement software and systems and service providers, also without recompense.

The consequences for a coup must be swift and severe. These people knew the risks...

Hey, WTF's Going On?

Well may you ask! I'm assuming you're referring to Trump 2.0: Revenge of the Musk? 

Well I've learned that this is a coup planned among Musk, Thiel and Sacks (Trump's new "Crypto Czar"), the South African members of  the PayPal mafia, with input from Thiel lieutenant, Vice President J.D. Vance. You can read a summary of The Plan by Gil Duran of the Nerd Reich, who has understandably experienced a surge in subscribers since January 20.

“Trump himself will not be the brain …He will not be the CEO. He will be the chairman of the board—he will select the CEO (an experienced executive). This process, which obviously has to be televised, will be complete by his inauguration—at which the transition to the next regime will start immediately.”

With Musk as the “CEO”, they are systematically rebuilding the US government as if it were post-war Japan. They are replacing federal employees with their own “ninjas” - and extending this to academia and the media. They’re ignoring the courts and “paper protections”, as Vance told them to: 

I think that what Trump should do, if I was giving him one piece of advice: Fire every single midlevel bureaucrat, every civil servant in the administrative state, replace them with our people. 

This is an illegal and overt attempt to seize control of the US government. A coup.

You can spot the growth of the techno aspect of their "New Reality" in Google's 2014 declaration of war on the human race and the creed of the techno-optimists. And much I've observed on these pages in between.

Much was made of their $1m 'donation' to attend the Orange Leader's inauguration, but you have to wonder whether Zuckerberg and the other tech oligarchs who were then featured in the front row were also in on the plan - or just presented as if they were.

Can America's institutions hold out? Or will the Republican Party remain complicit in the plan - wittingly or unwittingly?

Or is it too late?

Also bear in mind the leaders of this coup also have the world in their sights, riding the rails of their own borderless technology - AI, crypto meme coins and bubbling bitcoin and the social media - and they're attacking copyright and AI regulation worldwide in an attempt to free-ride on our privacy and creative content and make it their own. 

Now that Musk and the nerds from DOGE control the federal government IT and payment systems, you could replace Mad Marjorie Taylor Greene's "jewish space laser" geoengineering conspiracy theory with a Tesla-Starlink cyborg network fueled by $TRUMP meme coins bought with US government money.  

It's in our power to simply stop using their tech, and there are plenty of independent providers of social media, search, email and so on. 

Or is it too late?

Open Agentic AI And True Personalisation

Sixteen years on from my own initial posts on the subject of a personal assistant that can privately find and buy stuff for you on the web, and we have 'open agentic AI'. But are you really any closer to the automated selection and purchase of your own personalised products without needlessly surrendering your privacy or otherwise becoming the victim? Should this latest iteration of open generative AI be autonomously making and executing decisions on your behalf? 

What is Agentic AI?

An 'agentic' AI is an evolution of generative AI beyond a chatbot. It receives your data and relies on pattern matching to generate, select and execute one of a number of potential pre-programmed actions without human guidance, then 'learns' from the result (as NVIDIA, the leading AI chip maker, explains). 

A 'virtual assistant' that can find, buy and play music, for example, is a form of agentic AI (since it uses AI in its processes), but the ambition involves a wider range of tasks and more process automation and autonomy (if not end-to-end). 

You'll see a sleight-of-hand in the marketing language (like NVIDA's) as developers start projecting 'perception', 'understanding' and 'reasoning' on their agentic AIs, but computers don't actually do any of those human things. 

It's certainly a compelling idea to apply this to automating various highly complex, tedious consumer 'workflows' that have lots of different parameters - like buying a car, perhaps (or booking a bloody rail ticket in the UK!). 

Wearing my legal hat, I also see myriad interesting challenges (which I'd be delighted to discuss, of course!), some of which are mentioned here, but not all...

Some challenges

The main problem with using an 'agentic AI' in a consumer context is the involvement of a large language model and generative AI where there is a significant (e.g. economic, medical and/or legal) consequence for the user (as opposed to a chatbot or information-only scenario (though that can also be problematic). Currently, the household or device based virtual assistants are carrying out fairly mundane tasks, and you could probably get a refund if it bought you the wrong song, for example, if that really bothered you. Buying the wrong car would likely be a different matter.

There may also be confusion about the concept of 'agency' here. The word 'agentic' is used to mean that the AI has 'agency' in the sense it can operate without human guidance. That AI is not necessarily anyone's legal 'agent' (more below) and is trained on generic training data (subject to privacy, copyright consents/licensing), which these days is itself synthetic - generated by an AI. So, agentic AIs are not hosted exclusively by or on behalf of the specific consumer and do not specifically cater to a single end-customer's personalised needs in terms of the data it holds/processes and how it deals with suppliers. It does not 'know' you or 'understand' anyone, let alone you.  

Of course, that is consistent with how consumer markets work: products have generally been developed to suit the supplier's requirements in terms of profitability and so on, rather than any individual customer's needs. Having assembled what the supplier believes to be a profitable product by reference to an ideal customer profile in a given context, the supplier's systems and marketing/advertising arrangements seek out customers for the product who are 'scored' on the extent to which they fit that 'profile' and context. This also preserves 'information asymmetry' in favour of the supplier, who knows far more about its product and customers than you know about the supplier or the product. In an insurance context, for example, that will mean an ideal customer will pay a high premium but find it unnecessary, too hard or impossible to make a claim on the policy. For a loan, the lender will be looking for a higher risk customer who will end up paying more in additional interest and default fees than lower risk customers. But all this is only probabilistic, since human physiology may be 'normally distributed' but human behaviour is not.

So using an agentic AI in this context would not improve your position or relationship with your suppliers, particularly if the supplier is the owner/operator of the agentic AI. The fact that Open AI has offered its 'Operator' agentic AI to its pro-customers (who already pay a subscription of $200 a month!) begs the question whether Open AI really intends rocking this boat, or whether it's really a platform for suppliers like Facebook or Google search in the online advertising world. 

It's also an open question - and a matter for contract or regulation - as to whether the AI is anyone's legal 'agent' (which it could be if the AI were deployed by an actual agent or fiduciary of the customer, such as a consumer credit broker). 

Generative AI also has a set of inherent risks. Not only do they fail to 'understand' data, but to a greater or lesser degree they are also inaccurate, biased and randomly hallucinate rubbish (not to mention the enormous costs in energy/water, capital and computing; the opportunity cost of diverting such resources from other service/infrastructure requirements; and other the 'externalities' or socioeconomic consequences that are being ignored and not factored into soaring Big Tech stock prices - a bubble likely to burst soon). It may also not be possible to explain how the AI arrives at its conclusions (or, in the case of an agentic AI, why it selected a particular product, or executed a specific task, rather than another). Simply overlaying a right to human intervention by either customer or supplier would not guarantee a better outcome on theses issues (due to lack of explainability, in particular). A human should be able to explain why and how the AI's decision was reached and be able to re-take the decision. And, unfortunately, we are seeing less and less improvement in each of these inherent risk areas with each version of generative AIs.

All this means that agentic AI should not be used to fully automate decisions or choices that have any significant impact on an individual consumer (such as buying a car or obtaining a loan or a pension product).  

An Alternative... Your Own Personal Agent

What feels like a century ago, in 2009, I wondered whether the 'semantic web' would spell the end of price comparison websites. I was tired of seeing their expensive TV ads - paid for out of the intermediary's huge share of the gross price of the product. I thought: "If suppliers would only publish their product data in semantic format, a 'widget' on my own computer could scan their datafeeds and identify the product that's right for me, based on my personal profile and other parameters I specify". 

By 2013, I was calling that 'widget' an Open Data Spider and attempted to explain it further in an article for SCL on the wider tech themes of Midata, Open Data and Big Data tools (and elsewhere with the concept of 'monetising you'). I thought then - and still think now - that: 

"a combination of Midata, Open Data and Big Data tools seems likely to liberate us from the tyranny of the 'customer profile' and reputational 'scores', and allow us instead to establish direct connections with trusted products and suppliers based on much deeper knowledge of our own circumstances."

Personalised assistants are evolving to some degree, in the form of 'personal [online] data stores' (like MyDex or Solid); as well as 'digital wallets' or payment apps that sit on smartphones and other digital devices and can be used to store transaction data, tickets, boarding passes and other evidence of actual purchases. The former are being integrated in specific scenarios like recruitment and healthcare; while the latter tend to be usable only within checkout processes. None seems to be playing a more extensive role in pre-evaluating your personal requirements, then seeking, selecting and purchasing a suitable product for you from a range of potential suppliers (as opposed to a product that a supplier has created for its version of an 'ideal' customer that you seem to fit to some degree). 

Whether the providers of existing personal data stores and digital wallets will be prepared to extend their functionality to include more process automation for consumers may also depend on the willingness of suppliers to surrender some of their information advantage and adapt their systems (or AIs) to respond to and adapt products according to actual consumer requests/demand.

Equally, the digital 'gatekeepers' such as search providers and social media platforms will want to protect their own advertising revenue and other fees currently paid by suppliers who rely on them for targeting 'ideal' customers. Whether they can 'switch sides' to act for consumers and preserve/grow this revenue flow remains to be seen.

Overall, if I were a betting man, I'd wager that open agentic AI won't really change the fundamental relationship between suppliers, intermediaries and consumers, and that consumers will remain the targets (victims) for whatever suppliers and intermediaries dream up for them next...

I'd love to be corrected!