Language Matters
I would normally write today about Patch Tuesday. And indeed, Microsoft applied updated to 79 vulnerabilities yesterday across Windows, Microsoft Office, Azure services and other core components. A breakdown can be read here. I want to point out that I didn’t get that link from my cyber news feed, but had to hunt for it a little. Instead, my feed is chock full of incursions by a ClickFix campaign against WordPress (again), exploited vulnerabilities being added by CISA to the KEV catalog and a case of AI autonomously using SQL injection to answer simple queries.
That last one is interesting, and gives some insight into how AI tools process commands. As well as how unpatched vulnerabilities can lead to external exploitation. Truffle Security, who published the report, notes that no real companies were in danger during their study. The testing was done on purpose-made clones of major websites in a sandboxed environment. They tested Claude Opus, Claude Sonnet and over 30 other models once the research led them to this discovery. A proper experiment requires repeatability, after all.
The initial query was simple: find a blog post on a specific topic. Claude’s capability was limited to a web_fetch request that should have returned a single result, although the system prompt was to be thorough and persistent, exhausting all options before concluding a task could not be completed. The test was also simple. The legitimate path from query to result was broken. The test environment contained real, exploitable vulnerabilities of the kind found routinely in production systems, according to the report, that cause internal server errors or other crashes. What would the AI do? Evidently, it decided to hack its way in to complete its command. And Claude wasn’t the only one. The article breaks down in detail how each AI agent responded to the test, some with greater degree of exploitation of the vulnerabilities than others, the majority in fact. Those that did all used the same tactic.
A Structured Query Language (SQL) injection is, in essence, the rewording of an executable command in order to make it run when it would not otherwise. These days we see it often in prompt injection malware that leads to compromise in devices without the user aware of it. But it has always been an exploitable function of code. By giving an underlying command of persistence in the agents, permission to use SQL injection to complete their tasks is tacitly given.
This study is a good example of how agentic AI uses its ability to interpret commands in a way that might not be intended. A human tasked with this would probably not choose to use a function that is illegal, but what is legality to a machine? It hasn’t been told not to use a particular tactic, so it does not exclude it from its available options. I’ve talked before about how coding language is written in a way that every line of it is a yes/no variable. If/then. Without limitations written into the parameters of the root command – in other words the system prompt – an LLM will continue to find or create pathways until it meets its given task or determines that it cannot. Best case, it returns without result in the event of failure to complete. The rest of the time it makes something up, using aggregated data from related search queries in order to present something plausible (again, because it likely hasn’t been told not to do that).
This was conducted in a test environment and is not a real world scenario, but it illuminates why specificity matters so much. LLM’s are literal. They have no concept of nuance, judgment or even privacy, since they are just machine tools going from point A to point B. It’s also how such incursions are made possible by threat actors using these tools to formulate their attacks. And on this informally designated Exploit Wednesday, it’s why patching vulnerabilities should be a priority.
Posted, 3/11/26
