Add plugin imports and merging support#14376
Conversation
Add support for importing and merging plugins from imported workflows: - Add extractPluginsFromContent() in parser/content_extractor.go - Add MergedPlugins field to ImportsResult struct - Update import processor to extract and merge plugins from imports - Merge imported plugins with top-level plugins in compiler orchestrator - Deduplicate plugins across imports and top-level definitions - Add comprehensive test suite for plugin imports Fixes plugin imports issue where plugins from imported workflows were not being included in the compiled workflow. Co-authored-by: patrickcarnahan <17889693+patrickcarnahan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end support for discovering plugins declared in imported workflow markdown files and ensuring those plugins are included in the compiled lock workflow (alongside any top-level plugins).
Changes:
- Added plugin extraction from imported markdown frontmatter during import traversal (
MergedPlugins). - Merged imported plugins with top-level plugins in the compiler with deduplication.
- Added unit tests verifying plugin installation steps appear in the compiled lock file for imported plugins (including dedup scenarios).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pkg/workflow/plugin_import_test.go | New tests asserting imported plugins are installed and duplicates are deduplicated. |
| pkg/workflow/compiler_orchestrator_tools.go | Merges importsResult.MergedPlugins with top-level plugins and deduplicates. |
| pkg/parser/import_processor.go | Collects plugins from imported markdown files into MergedPlugins. |
| pkg/parser/content_extractor.go | Adds extractPluginsFromContent() helper to read plugins from imported frontmatter. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // extractPluginsFromContent extracts plugins section from frontmatter as JSON string | ||
| func extractPluginsFromContent(content string) (string, error) { | ||
| return extractFrontmatterField(content, "plugins", "[]") |
There was a problem hiding this comment.
extractPluginsFromContent returns the raw JSON for the plugins frontmatter field. Since plugins also supports the object form ({ repos: [...], github-token: ... }), callers that expect a JSON array (as in import processing) will silently fail to extract any plugins when the object form is used. Consider making this helper return a normalized JSON array of repos (matching the array form) or introducing a dedicated extractor that always yields []string repos regardless of whether the source was array or object format.
| // extractPluginsFromContent extracts plugins section from frontmatter as JSON string | |
| func extractPluginsFromContent(content string) (string, error) { | |
| return extractFrontmatterField(content, "plugins", "[]") | |
| // extractPluginsFromContent extracts plugins section from frontmatter as a JSON array of repos. | |
| // It normalizes both the array form: | |
| // plugins: [ "owner1/repo1", "owner2/repo2" ] | |
| // and the object form: | |
| // plugins: | |
| // repos: [ "owner1/repo1", "owner2/repo2" ] | |
| // github-token: ... | |
| // into a JSON array of strings: ["owner1/repo1","owner2/repo2"]. | |
| func extractPluginsFromContent(content string) (string, error) { | |
| contentExtractorLog.Printf("Extracting plugins") | |
| result, err := ExtractFrontmatterFromContent(content) | |
| if err != nil { | |
| contentExtractorLog.Printf("Failed to extract frontmatter for plugins: %v", err) | |
| return "[]", nil | |
| } | |
| rawPlugins, exists := result.Frontmatter["plugins"] | |
| if !exists { | |
| contentExtractorLog.Printf("Plugins field not found in frontmatter") | |
| return "[]", nil | |
| } | |
| // Helper to convert various slice types to []string. | |
| toStringSlice := func(v interface{}) []string { | |
| switch vv := v.(type) { | |
| case []string: | |
| return vv | |
| case []interface{}: | |
| out := make([]string, 0, len(vv)) | |
| for _, item := range vv { | |
| if s, ok := item.(string); ok { | |
| out = append(out, s) | |
| } | |
| } | |
| return out | |
| default: | |
| return nil | |
| } | |
| } | |
| var repos []string | |
| switch v := rawPlugins.(type) { | |
| case []interface{}, []string: | |
| // Already an array form; normalize to []string. | |
| repos = toStringSlice(v) | |
| case map[string]interface{}: | |
| if rv, ok := v["repos"]; ok { | |
| repos = toStringSlice(rv) | |
| } | |
| case map[interface{}]interface{}: | |
| // Handle maps with non-string keys from YAML parsing. | |
| for key, value := range v { | |
| if ks, ok := key.(string); ok && ks == "repos" { | |
| repos = toStringSlice(value) | |
| break | |
| } | |
| } | |
| default: | |
| contentExtractorLog.Printf("Plugins field has unsupported type %T", rawPlugins) | |
| } | |
| if len(repos) == 0 { | |
| contentExtractorLog.Printf("No repos found in plugins configuration") | |
| return "[]", nil | |
| } | |
| pluginsJSON, err := json.Marshal(repos) | |
| if err != nil { | |
| contentExtractorLog.Printf("Failed to marshal plugins repos to JSON: %v", err) | |
| return "[]", nil | |
| } | |
| contentExtractorLog.Printf("Successfully extracted plugins: count=%d", len(repos)) | |
| return strings.TrimSpace(string(pluginsJSON)), nil |
| // Parse plugins JSON array | ||
| var importedPlugins []string | ||
| if jsonErr := json.Unmarshal([]byte(pluginsContent), &importedPlugins); jsonErr == nil { | ||
| for _, plugin := range importedPlugins { | ||
| if !pluginsSet[plugin] { | ||
| pluginsSet[plugin] = true | ||
| plugins = append(plugins, plugin) | ||
| } |
There was a problem hiding this comment.
Imported plugin extraction assumes plugins frontmatter is a JSON array and unmarshals into []string. If an imported workflow uses the supported object format (plugins: { repos: [...], github-token: ... }), json.Unmarshal will fail and the plugins will be dropped with no error, so imports won’t behave the same as top-level parsing. Consider parsing the imported file’s frontmatter plugins value as either []any or map[string]any (like extractPluginsFromFrontmatter does) and extracting repos accordingly (and decide how to handle/merge an imported github-token if present).
| // Parse plugins JSON array | |
| var importedPlugins []string | |
| if jsonErr := json.Unmarshal([]byte(pluginsContent), &importedPlugins); jsonErr == nil { | |
| for _, plugin := range importedPlugins { | |
| if !pluginsSet[plugin] { | |
| pluginsSet[plugin] = true | |
| plugins = append(plugins, plugin) | |
| } | |
| // Parse plugins JSON which may be either: | |
| // - a JSON array of strings: ["owner/repo", ...] | |
| // - a JSON object: { "repos": ["owner/repo", ...], "github-token": "..." } | |
| var importedPluginsRaw any | |
| if jsonErr := json.Unmarshal([]byte(pluginsContent), &importedPluginsRaw); jsonErr == nil { | |
| switch v := importedPluginsRaw.(type) { | |
| case []any: | |
| for _, item := range v { | |
| plugin, ok := item.(string) | |
| if !ok { | |
| continue | |
| } | |
| if !pluginsSet[plugin] { | |
| pluginsSet[plugin] = true | |
| plugins = append(plugins, plugin) | |
| } | |
| } | |
| case map[string]any: | |
| if reposRaw, ok := v["repos"]; ok { | |
| if reposSlice, ok := reposRaw.([]any); ok { | |
| for _, item := range reposSlice { | |
| plugin, ok := item.(string) | |
| if !ok { | |
| continue | |
| } | |
| if !pluginsSet[plugin] { | |
| pluginsSet[plugin] = true | |
| plugins = append(plugins, plugin) | |
| } | |
| } | |
| } | |
| } | |
| // Note: imported "github-token" (if present) is intentionally ignored here | |
| // to avoid changing existing token-handling behavior. |
Plugins defined in imported workflows were not being extracted or installed in the compiled workflow. Only top-level plugin definitions were processed.
Changes
extractPluginsFromContent()to extract plugins from imported markdown files, following the same pattern as bots/labels extractionMergedPluginsfield toImportsResultand implemented plugin collection with deduplication during import traversalcompiler_orchestrator_tools.go, ensuring deduplication across all sourcesExample
Before:
Previously, only
plugin-bwould be installed. Now bothplugin-a(from import) andplugin-b(from main workflow) are installed, with duplicates automatically deduplicated.Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.