Publications

Conventional vulnerability analysis relies on source-level or binary-level access, or dynamic interaction, all of which may be unavailable to third-party analysts auditing closed-source, remotely hosted, or commercially gated software. Therefore, we propose a new paradigm of no-box vulnerability analysis in which neither source code nor runtime interaction is accessible, and only functionality metadata is available. Such metadata defines the intended behavior of the system, including its inputs, outputs, and side effects, while constraining the space of implementations consistent with that behavior. The intended behavior implies irreducible data flows, the minimal source-to-sink data flow skeletons shared by all conforming implementations. By reasoning over irreducible data flows, an analyst can formulate vulnerability hypotheses without observing or interacting with the target system. The analyst can later validate these hypotheses when additional access becomes available. We showcase the feasibility of no-box vulnerability analysis through implementing a pipeline called MCPSEC, which audits Model Context Protocol (MCP) servers for indirect prompt injection vulnerabilities using only the tool metadata exposed at server registration time. We evaluate MCPSEC on 20 widely deployed MCP servers comprising 177 tools, among which human evaluators confirm 95 vulnerable tools. MCPSEC identified 143 tools as vulnerable, and for each vulnerable tool, it produces Theory of Concepts (ToCs), a hypothesized attack scenario for later analyst validation. Using metadata alone, MCPSEC recovers 94 (98.9% recall) real verified vulnerabilities, compared with 80 (84.2% recall) for the LLM baseline. Overall, our results introduce no-box vulnerability analysis as a new analysis paradigm and demonstrate its practical feasibility in realistic systems.

Shows that as LLMs get better at reasoning they become more susceptible to novel jailbreaks. Introduces ACE and LACE — attacks that encode malicious queries with custom and layered ciphers — and CipherBench to measure cipher-decoding ability.