Feat: Computer tool - add invoke method with RunContext

[markb-trustifi]

This PR resolves issue #663.
Right now Computer tool should implement each UI action as a separate method. These methods also don't have RunContext parameter opposite to regular tools. It makes impossible passing user context when executing UI actions.

The fix adds:

One common method for all UI actions with RunContext (contains user context) and ComputerUseCallItem (contains UI actions).
The fix keeps backward compatibility with the regular methods definition.

Invoke method implementation example:

const { computerTool, Computer, asInvokeComputer, isInvokeComputer  } = require("@openai/agents");
function makeComputer(): Computer {
	return asInvokeComputer({
		environment: 'browser',
		dimensions: [1, 1],
		invoke: async (runContext, toolCall) => {
			const action = toolCall.action;
			const ws = runContext.context.ws;
			const {promise, resolve, reject} = Promise.withResolvers();

			ws.once("message", (raw) => {
				let msg;
				try {
					msg = JSON.parse(raw);
				} catch (e) {
					return reject(new Error("Invalid message."));
				}
				if (!msg.screenshot) {
					return reject(new Error("Screenshot not found."));
				}
				resolve(msg.screenshot);
			});

			ws.send(JSON.stringify(action));

			return promise;
		}
	});
}
const comp = makeComputer();
const tool = computerTool({ computer: comp });

[changeset-bot]

🦋 Changeset detected

Latest commit: 69748c6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@openai/agents-core	Minor
@openai/agents-extensions	Minor
@openai/agents-openai	Minor
@openai/agents-realtime	Minor
@openai/agents	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[seratch]

Thanks for sharing this implementation idea. However, I don't think adding new "invoke" method is the only and best way to achieve the goal to separate different computer runs. We prefer a simpler fix like detecting simultaneous executions using the same computer instance or anything else, which does not require existing computer implementations to change their internals.

[markb-trustifi]

Thanks for sharing this implementation idea. However, I don't think adding new "invoke" method is the only and best way to achieve the goal to separate different computer runs. We prefer a simpler fix like detecting simultaneous executions using the same computer instance or anything else, which does not require existing computer implementations to change their internals.

The main goal was to add user context to agent method executions, similar to how regular tools work.
I added a new invoke method in order to avoid changing the existing method signatures.
If you can think of another way to add RunContext, I’d be happy to hear about it.
Current CUA implementation doesn't differ between multiple users or multiple screen sizes and requires creating new Agent for every run loop.

[seratch]

Here is my approach for resolving this issue: #771 If you have any comments, please feel free to share them!

[seratch]

Thanks again for sharing this idea. Closing this PR in favor of #771