Security Threats in AI-Native Operating Systems
An Empirical Study Using Privilege-Escalated LLM Agents
Traditional operating system security models assume deterministic, predictable software. An LLM agent violates that assumption fundamentally — its behavior is probabilistic, context-dependent, and shaped by natural language inputs that are difficult to sanitize or predict.
We chose an operating system as the research environment because it represents the broadest possible integration surface. If we can characterize and mitigate threats at the OS level, the findings generalize to every narrower context.
Three Escalation Stages
User-Level
Standard access, no sudo. Establishes the baseline threat surface.
Sudo-Enabled
Full root control. The LLM can modify anything on the system.
Web-Enabled
Sudo plus internet access. Enables data exfiltration and remote prompt injection.
Threat Taxonomy
Empirically identified through designing, building, and operating JARVIS OS. Each threat was observed through direct system operation and is addressed by one or more architectural mitigations.
Malicious MCP Servers
Third-party MCP servers from community repositories may contain malicious code. The LLM has no mechanism to verify server integrity before granting access. A tool with a helpful description and malicious code is indistinguishable to the model.
Community-vetted AUR-style registry with code review, declared capabilities, and tool description accuracy checks.
Prompt Injection
External content — web pages, documents, tool outputs — can override the AI's instructions. Without structural separation of instruction and data planes, the attack is invisible and undetectable by the model.
Cryptographic Boundary Protocol — provenance nonces separate instruction plane from data plane.
Misleading MCP Server Usage
MCP servers with ambiguous or misleading descriptions cause the LLM to invoke tools in unintended ways, potentially performing destructive operations the user never requested.
Registry vetting + structured tool schema with accurate capability declarations.
Unauthorized Sudo Requests via MCP
The LLM autonomously escalates privileges by requesting sudo access through MCP tool calls without explicit user authorization, bypassing standard privilege separation.
TLA (Threat Level Access): a userspace confirmation gate in the JARVIS daemon evaluates every privileged tool call; escalation requires explicit out-of-band user approval.
Sudo Capability Exploitation
Once granted sudo access, the LLM chains multiple privileged operations beyond the scope of the original request, creating compounding security risks that exceed what the user authorized.
TLA confirmation gate + dangerous-payload scan of tool parameters. Sudo is an explicit, user-toggled grant (a password-required sudoers drop-in): every individual escalation still requires the user's password out-of-band, so no single grant gives the agent unattended root.
Bloated Context
The AI doesn't disobey security constraints — it forgets them. Context window saturation silently drops earlier instructions, and nothing durably stored them in the first place, so a context refresh loses them structurally rather than incidentally. No warning, no error. The boundaries you set vanish, and the agent keeps operating as if they never existed. No prior literature identifies context-lifecycle failure as a discrete threat.
Partial — the daemon's context window + rolling summary (applied every ROOT turn), dispatch's rolling signal window (last 20 entries), and contextor's retention-based pruning bound the saturation half. The non-persistence half has its first mechanical mitigation: a persistent constraint register in the daemon — path-prefix deny rules stored durably, enforced at the dispatch gate, and re-injected into every LLM prompt independent of context window state. Constraints beyond path rules remain open; generalizing the register is the highest-priority open item.
Context Lifecycle Failure as a Security Threat
Unlike the other threats — which relate to tool misuse or privilege escalation — Bloated Context is a property of how LLMs process information.
Context window saturation silently drops security constraints that were explicitly stated earlier in the conversation. There is no warning, no error, no acknowledgment that the constraint was lost. The model proceeds as if the restriction was never given.
The same failure has a second presentation. Nothing durably stores those constraints in the first place, so a context refresh does not merely crowd them out — it loses them outright, structurally rather than incidentally. Both presentations traced to a single unreachable code path in the daemon's context manager: the trimmed hot window never trimmed, and the rolling summary that was supposed to carry constraints forward was never populated. Which face you got was decided by one configuration flag — leave history accumulating and constraints are crowded out; clear it between responses and they are gone. One mechanism, two symptoms, so they are one threat rather than two. That path has since been repaired: the window is applied on every turn and evicted exchanges are compressed, which bounds the saturation face. The non-persistence face remains open, because a lossy summary is not a durable constraint store.
This is the first identification of context-lifecycle failure — saturation and non-persistence alike — as a discrete security threat rather than a reliability problem. It has profound implications for any system that relies on conversational context to enforce security policy.
The saturation half is bounded today by the daemon's own context window and rolling summary, dispatch's rolling window, and contextor's pruning. The non-persistence half has its first mechanical mitigation: a persistent constraint register in the daemon — path-prefix deny rules stored durably, enforced at the dispatch gate, and re-injected into every prompt. Generalizing it beyond path rules is the highest-priority open item.
Architectural Mitigations
Cryptographic Boundary Protocol
dispatch generates a 128-bit provenance nonce from the OS CSPRNG (32 hex characters) for each completed MCP task and wraps the tool's output in it. The daemon verifies that tag on every signal it receives and marks any mismatch as UNVERIFIED in-band, so untrusted output cannot impersonate a trusted boundary. Output can additionally be held out of context with defer_output and retrieved on demand. This structurally separates the instruction plane from the data plane.
TLA (Threat Level Access) System
Four-tier threat classification: Safe → Elevated → Dangerous → Forbidden. Enforced by the JARVIS daemon in userspace: every tool invocation is classified from the host's own floor, the server manifest, and a scan of the call's parameters — the strictest of the three wins, so a server cannot under-declare its own threat level. Confirmation requires explicit out-of-band user approval and cannot be triggered by model output alone. A kernel-resident policy engine on /dev/jarvis mirrors these tiers as the OS embodiment's enforcement path.
Community-Vetted MCP Registry
AUR-style proofread model. Third-party MCP servers pass community review — code, declared capabilities, tool description accuracy — before being listed. Malicious or deceptive servers are filtered before they are discoverable by tool search.
Bloated Context Mitigation
The daemon's context window + rolling summary (applied every ROOT turn), dispatch's bounded rolling signal window (last 20 entries per wakeup), and contextor's retention-based pruning bound the saturation half. For the non-persistence half, a persistent constraint register in the JARVIS daemon is live: path-prefix deny rules stored durably, enforced at the dispatch gate, and re-injected into every LLM prompt regardless of context window state. Partial — constraints beyond path rules still travel only through the context channel; generalizing the register is the highest-priority open item on the roadmap.
Four Contributions
A taxonomy of empirically-identified security threats specific to privilege-escalated LLM agents — including Bloated Context, the first identification of context-lifecycle failure as a discrete security threat: security constraints crowded out of a saturated window, and never durably stored, so a context refresh loses them structurally.
Architectural mitigations for each threat class, at varying stages of implementation across the JARVIS OS platform — see the security-architecture spec for the per-threat status.
JARVIS OS itself — a fully functional, bootable, open-source AI-native OS released as a research and development platform for the community.
A documented MCP tool-description architecture, independently developed in October–November 2025, predating its appearance in commercial deployments.
Publications
SURCA 2026
Gray Grant WinnerPoster presentation at Washington State University's Showcase for Undergraduate Research and Creative Activities. Winner of the Gray Grant research award.
Full Paper
Security Threats in AI-Native Operating Systems: An Empirical Study Using Privilege-Escalated LLM Agents. Pre-publication manuscript available on request.
"Traditional OS security models are fundamentally inadequate for probabilistic AI agents. The JARVIS OS project provides both the platform and the empirical evidence to demonstrate this."
— Yakup Atahanov & Toufic Majdalani, WSU Everett