Skip to content

Comments

fix(IMP-003): move generateCustomJobToolDefinition to safe_outputs_config_generation.go#17770

Merged
pelikhan merged 2 commits intomainfrom
copilot/add-generate-custom-job-tool-definition
Feb 22, 2026
Merged

fix(IMP-003): move generateCustomJobToolDefinition to safe_outputs_config_generation.go#17770
pelikhan merged 2 commits intomainfrom
copilot/add-generate-custom-job-tool-definition

Conversation

Copy link
Contributor

Copilot AI commented Feb 22, 2026

The IMP-003 conformance check greps for generateCustomJobToolDefinition in safe_outputs_config_generation.go, but the function was placed in safe_outputs_tools_generation.go, causing the check to fail.

Changes

  • Moved generateCustomJobToolDefinition from safe_outputs_tools_generation.gosafe_outputs_config_generation.go (no logic changes)
  • Added fmt and sort imports to safe_outputs_config_generation.go to support the moved function
  • Added unit tests in safe_outputs_config_generation_test.go covering string/boolean/number/choice inputs, empty inputs, default description fallback, and JSON serializability

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE 64/bin/go git rev-�� --show-toplevel go /usr/bin/git H1qo/6K-Oxms-mc7git GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha -json GO111MODULE 2154935/b124/vet.cfg GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env 0403-31997/test-815431192/.github/workflows GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha vaScript547976447/001/test-complex-frontmatter-with-tools.md GO111MODULE 0/x64/bin/node GOINSECURE GOMOD GOMODCACHE go -has�� SameOutput1819461009/001/stability-test.md GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env runs/20260222-190403-31997/test-974324181/.github/workflows GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha -b test-branch /usr/bin/git ./cmd/gh-aw GOPROXY 64/bin/go git rev-�� --show-toplevel node /usr/bin/git --check scripts/**/*.js 64/bin/go git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha /tmp/go-build2732154935/b001/_pkg_.a (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha 64/bin/go sh /usr/bin/git "prettier" --chegit sh 64/bin/go git rev-�� --git-dir go /usr/bin/git -json GO111MODULE 64/bin/go /usr/bin/git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE erignore env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha ck 'scripts/**/*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE _9LLtYheNxUI env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x-test.short=true GOINSECURE GOMOD GOMODCACHE go env runs/20260222-190403-31997/test-434847911/.github/workflows GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go l GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha brave.md GO111MODULE /opt/hostedtoolcache/go/1.25.0/x-buildmode=exe GOINSECURE GOMOD GOMODCACHE go env 0403-31997/test-47053892 GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go l GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go estl�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go estl�� -json GO111MODULE 64/bin/go Action pins syngit GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path om/charmbraceletGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 5026500/b389/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/envu.github/aw/actions-lock.json GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env Uqc0/F5v_X3OP52TGOSUMDB GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 5026500/b406/importcfg (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 1407720430/.github/workflows GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 5026500/b397/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 5026500/b386/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build2732154935/b381/cli.test /tmp/go-build2732154935/b381/cli.test -test.testlogfile=/tmp/go-build2732154935/b381/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 5026500/b383/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE bagabas/go-udiffenv GOMODCACHE go (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[Safe Outputs Conformance] IMP-003: generateCustomJobToolDefinition function missing from safe_outputs_config_generation.go</issue_title>
<issue_description>### Conformance Check Failure

Check ID: IMP-003
Severity: MEDIUM
Category: Implementation

Problem Description

The IMP-003 conformance check verifies that pkg/workflow/safe_outputs_config_generation.go contains a generateCustomJobToolDefinition function for dynamic MCP tool schema generation. This function is absent from the file.

The file currently exports only two functions:

  • populateDispatchWorkflowFiles
  • generateSafeOutputsConfig

The specification requires that safe output types support dynamic schema generation (i.e., the ability to generate MCP tool definitions at runtime based on configuration, rather than relying solely on the static schema in pkg/workflow/js/safe_outputs_tools.json). Without generateCustomJobToolDefinition, any custom job tool types cannot have their schemas generated dynamically.

Affected Components

  • File: pkg/workflow/safe_outputs_config_generation.go
  • Expected function: generateCustomJobToolDefinition
  • Related static schema: pkg/workflow/js/safe_outputs_tools.json (present and contains inputSchema)
  • Related config: pkg/workflow/safe_outputs_config.go (has Schema Generation Architecture documentation)

Current Behavior

pkg/workflow/safe_outputs_config_generation.go does not define generateCustomJobToolDefinition. Custom job tool definition generation is either unimplemented or handled elsewhere without the expected function name.

Expected Behavior

Per the IMP-003 check, a generateCustomJobToolDefinition function should exist in pkg/workflow/safe_outputs_config_generation.go. This function should accept a custom job tool configuration and return an MCP-compatible tool definition with a populated inputSchema.

Remediation Steps

This task can be assigned to a Copilot coding agent with the following steps:

  1. Review pkg/workflow/safe_outputs_config.go (which contains Schema Generation Architecture documentation) to understand the intended design for dynamic schema generation.
  2. Review pkg/workflow/js/safe_outputs_tools.json to understand the static schema format that generateCustomJobToolDefinition should produce dynamically.
  3. Review generateSafeOutputsConfig in pkg/workflow/safe_outputs_config_generation.go to understand the existing generation patterns.
  4. Implement generateCustomJobToolDefinition in pkg/workflow/safe_outputs_config_generation.go that generates an MCP tool definition (with inputSchema) for a given custom job safe output type.
  5. Add appropriate unit tests.

Verification

After remediation, verify the fix by running:

bash scripts/check-safe-outputs-conformance.sh

IMP-003 should pass: [PASS] IMP-003: Schema generation is implemented.

References

  • Safe Outputs Specification: docs/src/content/docs/reference/safe-outputs-specification.md
  • Conformance Checker: scripts/check-safe-outputs-conformance.sh (lines 358–396, check_schema_consistency)
  • Config Generation: pkg/workflow/safe_outputs_config_generation.go
  • Static Schema: pkg/workflow/js/safe_outputs_tools.json
  • Architecture Docs: pkg/workflow/safe_outputs_config.go
  • Run ID: §22281286232
  • Date: 2026-02-22

Generated by Daily Safe Outputs Conformance Checker

  • expires on Feb 23, 2026, 5:01 PM UTC

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

…nfig_generation.go

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add generateCustomJobToolDefinition function for conformance fix(IMP-003): move generateCustomJobToolDefinition to safe_outputs_config_generation.go Feb 22, 2026
@pelikhan pelikhan marked this pull request as ready for review February 22, 2026 20:03
Copilot AI review requested due to automatic review settings February 22, 2026 20:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes IMP-003 conformance by relocating generateCustomJobToolDefinition into the file the checker expects, and adds tests around custom job tool schema generation.

Changes:

  • Moved generateCustomJobToolDefinition from safe_outputs_tools_generation.go to safe_outputs_config_generation.go.
  • Updated imports in safe_outputs_config_generation.go to support the moved function (fmt, sort).
  • Added unit tests for generateCustomJobToolDefinition, including JSON serializability.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
pkg/workflow/safe_outputs_tools_generation.go Removes the custom job tool definition generator from the tools-generation file (now sourced from config-generation).
pkg/workflow/safe_outputs_config_generation.go Adds generateCustomJobToolDefinition (and its dependencies) in the location required by IMP-003.
pkg/workflow/safe_outputs_config_generation_test.go Adds test coverage for MCP tool schema generation for custom safe-jobs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.


return tool
}

Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc comment for populateDispatchWorkflowFiles appears to have been removed when generateCustomJobToolDefinition was inserted above it. Since this function has non-obvious ordering requirements (must run before generateSafeOutputsConfig), please restore the previous comment or otherwise document the requirement here to avoid regressions.

Suggested change
// populateDispatchWorkflowFiles resolves dispatch workflow names to concrete workflow
// file extensions (e.g., ".lock.yml" or ".yml") and populates the
// SafeOutputs.DispatchWorkflow.WorkflowFiles map accordingly.
//
// NOTE: This function must be called before generateSafeOutputsConfig, which relies
// on WorkflowFiles being fully populated. Do not change the call ordering without
// updating both functions and their documentation.

Copilot uses AI. Check for mistakes.
Comment on lines +169 to +176
// TestGenerateCustomJobToolDefinition tests that generateCustomJobToolDefinition produces
// valid MCP tool definitions from SafeJobConfig input definitions.
func TestGenerateCustomJobToolDefinition(t *testing.T) {
tests := []struct {
name string
jobName string
jobConfig *SafeJobConfig
check func(t *testing.T, result map[string]any)
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new tests for generateCustomJobToolDefinition largely overlap with existing coverage in safe_outputs_tools_generation_test.go (e.g., basic structure, default description, boolean input). Consider consolidating to a single test file or trimming redundant cases to keep the test suite smaller and easier to maintain.

Copilot uses AI. Check for mistakes.
@pelikhan pelikhan merged commit f6f36d2 into main Feb 22, 2026
120 checks passed
@pelikhan pelikhan deleted the copilot/add-generate-custom-job-tool-definition branch February 22, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Safe Outputs Conformance] IMP-003: generateCustomJobToolDefinition function missing from safe_outputs_config_generation.go

2 participants