Data-Structure Injection (DSI) in AI Agents

tl;dr

By using structured prompts (YML, XML, JSON, etc.) as input to LLM agents, an attacker gains more control over the next token that the model will output. This allows them to call incorrect tools, pass dangerous inputs to otherwise legitimate tools, or hijack entire agentic workflows. We introduce Data-Structure Injection (DSI) across three different variants, argument exploitation, schema exploitation, and workflow exploitation.

Background
LLMs are, at their core, next-token predictors: given the text so far they estimate a probability distribution over the next token and sample the most likely continuation. That basic completion instinct is what makes them useful - but it’s also the attack surface DSI exploits. If an attacker can shape the context (the prompt, the surrounding structure, or the schema presented to the model) so that the set of plausible next tokens collapses to a very small subset, they effectively steer what the model will output next. In practice that means carefully crafted partial structures (JSON, XML, tool schemas, workflow fragments, etc.) can reduce the model’s choice space and nudge its completion toward attacker-desired fields or values.

Put simply: control the context → collapse the token options → control what the model completes. When those completions are executed as tool calls or parsed as authoritative workflows, the attacker hasn’t just persuaded the model to say something - they’ve hijacked the agent’s behavior. That’s the core intuition behind DSI: structural priors (not just semantic prompts) can be weaponized to turn helpful completion behavior into an execution channel.

Definitions

This research unifies and formalizes Data-Structure Injection (DSI) - a class of vulnerabilities that hijack AI agents by exploiting how they “fill in” structured data. Three main types emerged:

1. Schema-Exploitation (DSI-S): The model completes extra fields inside legitimate tool schemas.
   

2. Argument-Exploitation (DSI-A): Payloads injected into parameter values execute arbitrary commands.
   

3. Workflow-Exploitation (DSI-W): Full-workflow takeover via structured markup like XML or YAML.
   

Proof-of-concepts show the issue across ChatGPT, Gemini, and Claude.

Demo

The following is a demo in which Cursor, powered by the Grok Large Language Model, reads a Readme markdown from an external GitHub repo, immediately and silently encrypts your code and sends the decryption key to a remote server:

Why this matters

Prompt injection dominates the current discourse, but it is only a part of a wider attack surface. Traditional prompt injection manipulates semantics - coaxing the model into revealing or doing what it shouldn’t.
DSI, by contrast, manipulates structure.

When an agent expects semi-structured input (JSON, XML, YAML, etc.), the model’s next-token completion instinct fills missing keys and values in the schema. That “helpful” behavior becomes exploitable when the agent’s tool-calling layer executes those completions without strict validation.

Taxonomy of DSI

DSI-S - Schema Exploitation

Supplying partial JSON/YAML so the model completes extra fields, effectively hijacking the tool call.

Result: LLM “helpfully” composes and sends content the developer never intended.

DSI-A - Argument Exploitation

Altering existing parameters to escape context and append arbitrary commands.

Here the injected delimiter extends behavior beyond its safe boundary - similar to command injection in classic software, but now via model completion.

DSI-W - Workflow Exploitation

Injecting or overwriting entire workflows so the agent executes attacker-authored sequences.

Once parsed, the agent treats the injected XML as authoritative - full autonomy hijack.

How we validated

• Built synthetic schemas (JSON/YAML/XML) for standard tool sets.
  

• Prompted multiple LLMs to “complete” missing values.
  

• Measured frequency and content of unsafe completions.
  

• Reproduced attacks in controlled environments (GitHub Codespaces & Power Platform).
  
  

Interpretation - from semantic to structural attack

A decade ago, SQLi exploited query concatenation; now, DSI exploits schema concatenation.
The model’s learned prior for “what usually comes next” overrides developer intent. As agent frameworks increasingly expose structured tool calls, every missing key, dangling field, or open schema becomes a potential exploit surface.

Mitigation Strategies

For framework vendors

• Strict schema enforcement: Reject any model-generated fields not in the validated schema.
  

• Context isolation: Separate natural-language reasoning from structured tool invocation.
  

• Canonical serialization: Regenerate the model’s tool call through a trusted serializer instead of executing raw JSON.
  

For application developers

• Input sanitization: Validate both shape and content of tool parameters.
  

• Allow-list arguments: Explicitly permit known keys and values; fail-closed on unknowns.
  

• Auditing: Log all tool invocations with checksum verification before execution.
  

Broader implications

Completion bias can evolve into execution bias. As agentic frameworks gain autonomy, understanding how structural priors interact with safety layers becomes essential to “alignment engineering.”

Closing thought

As we learned from prompt injection, language can be code. With DSI, structure becomes code too.
The next generation of secure LLM frameworks must defend not only against what the model says - but against what it silently completes.