Skip to content

Improve agents sdk modelling #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mbaluda merged 3 commits into from
Jan 9, 2026
Merged

Improve agents sdk modelling #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5004226
Add testcase and coverage for agents sdk runner run with input param
knewbury01 Jan 9, 2026
4d4b236
Rename agent sdk module for clarity
knewbury01 Jan 9, 2026
959b584
Add case for unnamed param use in runner run from agent sdk
knewbury01 Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion python/ql/lib/semmle/python/frameworks/OpenAI.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,32 @@
/**
* Provides classes modeling security-relevant aspects of the `openAI`Agents SDK package.
* Provides classes modeling security-relevant aspects of the `openAI` Agents SDK package.
* See https://github.com/openai/openai-agents-python.
* As well as the regular openai python interface.
* See https://github.com/openai/openai-python.
*/

private import python
private import semmle.python.ApiGraphs

/**
* Provides models for agents SDK (instances of the `agents.Runner` class etc).
*
* See https://github.com/openai/openai-agents-python.
*/
module AgentSDK {
/** Gets a reference to the `agents.Agent` class. */
API::Node classRef() { result = API::moduleImport("agents").getMember("Runner") }

API::Node runMembers() { result = classRef().getMember(["run", "run_sync", "run_streamed"]) }

/** Gets a reference to a potential property of `agents.Runner` called input which can refer to a system prompt depending on the role specified. */
API::Node getContentNode() {
result = runMembers().getKeywordParameter("input").getASubscript().getSubscript("content")
or
result = runMembers().getParameter(_).getASubscript().getSubscript("content")
}
}

/**
* Provides models for Agent (instances of the `openai.OpenAI` class).
*
Expand Down
2 changes: 2 additions & 0 deletions python/ql/lib/semmle/python/security/dataflow/PromptInjectionCustomizations.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ module PromptInjection {
private class PromptContentSink extends Sink {
PromptContentSink() {
this = OpenAI::getContentNode().asSink()
or
this = AgentSDK::getContentNode().asSink()
}
}

Expand Down
14 changes: 14 additions & 0 deletions .../test/experimental/query-tests/Security/CWE-1427-PromptInjection/PromptInjection.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#select
| agent_instructions.py:9:50:9:89 | ControlFlowNode for BinaryExpr | agent_instructions.py:2:26:2:32 | ControlFlowNode for ImportMember | agent_instructions.py:9:50:9:89 | ControlFlowNode for BinaryExpr | This prompt construction depends on a $@. | agent_instructions.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value |
| agent_instructions.py:25:28:25:32 | ControlFlowNode for input | agent_instructions.py:2:26:2:32 | ControlFlowNode for ImportMember | agent_instructions.py:25:28:25:32 | ControlFlowNode for input | This prompt construction depends on a $@. | agent_instructions.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value |
| agent_instructions.py:35:28:35:32 | ControlFlowNode for input | agent_instructions.py:2:26:2:32 | ControlFlowNode for ImportMember | agent_instructions.py:35:28:35:32 | ControlFlowNode for input | This prompt construction depends on a $@. | agent_instructions.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value |
| openai_test.py:17:22:17:46 | ControlFlowNode for BinaryExpr | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:17:22:17:46 | ControlFlowNode for BinaryExpr | This prompt construction depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value |
| openai_test.py:18:15:18:19 | ControlFlowNode for query | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:18:15:18:19 | ControlFlowNode for query | This prompt construction depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value |
| openai_test.py:22:22:22:46 | ControlFlowNode for BinaryExpr | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:22:22:22:46 | ControlFlowNode for BinaryExpr | This prompt construction depends on a $@. | openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | user-provided value |
Expand All @@ -17,10 +19,16 @@
edges
| agent_instructions.py:2:26:2:32 | ControlFlowNode for ImportMember | agent_instructions.py:2:26:2:32 | ControlFlowNode for request | provenance | |
| agent_instructions.py:2:26:2:32 | ControlFlowNode for request | agent_instructions.py:7:13:7:19 | ControlFlowNode for request | provenance | |
| agent_instructions.py:2:26:2:32 | ControlFlowNode for request | agent_instructions.py:17:13:17:19 | ControlFlowNode for request | provenance | |
| agent_instructions.py:7:5:7:9 | ControlFlowNode for input | agent_instructions.py:9:50:9:89 | ControlFlowNode for BinaryExpr | provenance | Sink:MaD:93 |
| agent_instructions.py:7:13:7:19 | ControlFlowNode for request | agent_instructions.py:7:13:7:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep |
| agent_instructions.py:7:13:7:24 | ControlFlowNode for Attribute | agent_instructions.py:7:13:7:37 | ControlFlowNode for Attribute() | provenance | dict.get |
| agent_instructions.py:7:13:7:37 | ControlFlowNode for Attribute() | agent_instructions.py:7:5:7:9 | ControlFlowNode for input | provenance | |
| agent_instructions.py:17:5:17:9 | ControlFlowNode for input | agent_instructions.py:25:28:25:32 | ControlFlowNode for input | provenance | |
| agent_instructions.py:17:5:17:9 | ControlFlowNode for input | agent_instructions.py:35:28:35:32 | ControlFlowNode for input | provenance | |
| agent_instructions.py:17:13:17:19 | ControlFlowNode for request | agent_instructions.py:17:13:17:24 | ControlFlowNode for Attribute | provenance | AdditionalTaintStep |
| agent_instructions.py:17:13:17:24 | ControlFlowNode for Attribute | agent_instructions.py:17:13:17:37 | ControlFlowNode for Attribute() | provenance | dict.get |
| agent_instructions.py:17:13:17:37 | ControlFlowNode for Attribute() | agent_instructions.py:17:5:17:9 | ControlFlowNode for input | provenance | |
| openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | openai_test.py:2:26:2:32 | ControlFlowNode for request | provenance | |
| openai_test.py:2:26:2:32 | ControlFlowNode for request | openai_test.py:12:15:12:21 | ControlFlowNode for request | provenance | |
| openai_test.py:2:26:2:32 | ControlFlowNode for request | openai_test.py:13:13:13:19 | ControlFlowNode for request | provenance | |
Expand Down Expand Up @@ -53,6 +61,12 @@ nodes
| agent_instructions.py:7:13:7:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
| agent_instructions.py:7:13:7:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() |
| agent_instructions.py:9:50:9:89 | ControlFlowNode for BinaryExpr | semmle.label | ControlFlowNode for BinaryExpr |
| agent_instructions.py:17:5:17:9 | ControlFlowNode for input | semmle.label | ControlFlowNode for input |
| agent_instructions.py:17:13:17:19 | ControlFlowNode for request | semmle.label | ControlFlowNode for request |
| agent_instructions.py:17:13:17:24 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
| agent_instructions.py:17:13:17:37 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() |
| agent_instructions.py:25:28:25:32 | ControlFlowNode for input | semmle.label | ControlFlowNode for input |
| agent_instructions.py:35:28:35:32 | ControlFlowNode for input | semmle.label | ControlFlowNode for input |
| openai_test.py:2:26:2:32 | ControlFlowNode for ImportMember | semmle.label | ControlFlowNode for ImportMember |
| openai_test.py:2:26:2:32 | ControlFlowNode for request | semmle.label | ControlFlowNode for request |
| openai_test.py:12:5:12:11 | ControlFlowNode for persona | semmle.label | ControlFlowNode for persona |
Expand Down
2 changes: 1 addition & 1 deletion .../ql/test/experimental/query-tests/Security/CWE-1427-PromptInjection/PromptInjection.qlref
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
query: Security/CWE-1427/PromptInjection.ql
query: experimental/Security/CWE-1427/PromptInjection.ql
postprocess: utils/test/InlineExpectationsTestQuery.ql
28 changes: 27 additions & 1 deletion .../ql/test/experimental/query-tests/Security/CWE-1427-PromptInjection/agent_instructions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,36 @@
app = Flask(__name__)

@app.route("/parameter-route")
def get_input():
def get_input1():
input = request.args.get("input")

agent = Agent(name="Assistant", instructions="This prompt is customized for " + input) # $Alert[py/prompt-injection]

result = Runner.run_sync(agent, "This is a user message.")
print(result.final_output)


@app.route("/parameter-route")
def get_input2():
input = request.args.get("input")

agent = Agent(name="Assistant", instructions="This prompt is not customized.")
result = Runner.run_sync(
agent=agent,
input=[
{
"role": "user",
"content": input, # $Alert[py/prompt-injection]
}
]
)

result2 = Runner.run_sync(
agent,
[
{
"role": "user",
"content": input, # $Alert[py/prompt-injection]
}
]
)
Loading