Use importlib.resources to resolve tool name to a file location

.github/workflows/smoketest.yaml

@@ -49,19 +49,19 @@ jobs:
 
         run: |
           source .venv/bin/activate
-          python main.py -p GitHubSecurityLab/seclab-taskflow-agent/personalities/assistant 'explain modems to me please'
-          python main.py -p GitHubSecurityLab/seclab-taskflow-agent/personalities/c_auditer 'explain modems to me please'
-          python main.py -p GitHubSecurityLab/seclab-taskflow-agent/personalities/examples/echo 'explain modems to me please'
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/CVE-2023-2283/CVE-2023-2283
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/echo
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_globals
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_inputs
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_large_list_result_iter
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_repeat_prompt
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_repeat_prompt_async
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_repeat_prompt_dictionary
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_reusable_prompt
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_reusable_taskflows
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/example_triage_taskflow
-          python main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/examples/single_step_taskflow
+          python main.py -p personalities.assistant 'explain modems to me please'
+          python main.py -p personalities.c_auditer 'explain modems to me please'
+          python main.py -p personalities.examples.echo 'explain modems to me please'
+          python main.py -t taskflows.CVE-2023-2283.CVE-2023-2283
+          python main.py -t taskflows.examples.echo
+          python main.py -t taskflows.examples.example
+          python main.py -t taskflows.examples.example_globals
+          python main.py -t taskflows.examples.example_inputs
+          python main.py -t taskflows.examples.example_large_list_result_iter
+          python main.py -t taskflows.examples.example_repeat_prompt
+          python main.py -t taskflows.examples.example_repeat_prompt_async
+          python main.py -t taskflows.examples.example_repeat_prompt_dictionary
+          python main.py -t taskflows.examples.example_reusable_prompt
+          python main.py -t taskflows.examples.example_reusable_taskflows
+          python main.py -t taskflows.examples.example_triage_taskflow
+          python main.py -t taskflows.examples.single_step_taskflow

README.md

@@ -147,43 +147,23 @@ Every YAML files used by the Seclab Taskflow Agent must include a header like th
 seclab-taskflow-agent:
   version: 1
   filetype: taskflow
-  filekey: GitHubSecurityLab/seclab-taskflow-agent/taskflows/CVE-2023-2283/CVE-2023-2283
 ```
 
-The `filetype` determines whether the file defines a personality, toolbox, or
-taskflow. This means that different types of files can be stored in the same directory.
-A `filetype` can be one of the followings:
-  - taskflow
-  - personality
-  - toolbox
-  - prompt
-  - model_config
-
-We'll explain these file types in more detail in the following sections.
-
-The `filekey` is a unique name for the file. It is used to allow
-cross-referencing between files. For example, a taskflow can reference
-a personality by its filekey. Because filekeys are used for
-cross-referencing (rather than file paths), it means that you can move
-a file to a different directory without breaking the links. This also
-means that you can easily import new files by dropping them into a sub-directory.
-We recommend including something like your
-GitHub `<username>/<reponame>` in your filekeys to make them globally unique.
-
-In the above example, it is a `taskflow` file with `filekey` `GitHubSecurityLab/seclab-taskflow-agent/taskflows/CVE-2023-2283/CVE-2023-2283`. The `filekey` is needed to run the taskflow from command line, e.g.:
-
-```
-python3 main.py -t GitHubSecurityLab/seclab-taskflow-agent/taskflows/CVE-2023-2283/CVE-2023-2283
-```
-
-will run the taskflow.
-
 The `version` number in the header should always be 1. It means that the
 file uses version 1 of the seclab-taskflow-agent syntax. If we ever need
 to make a major change to the syntax, then we'll update the version number.
 This will hopefully enable us to make changes without breaking backwards
 compatibility.
 
+The `filetype` determines whether the file defines a personality, toolbox, etc.
+This means that different types of files can be stored in the same directory.
+A `filetype` can be one of the following:
+  - taskflow
+  - personality
+  - toolbox
+  - prompt
+  - model_config
+
 We'll now explain the role of different types of files and functionalities available to them.
 
 ## Personalities
@@ -199,7 +179,6 @@ Example:
 seclab-taskflow-agent:
   version: 1
   filetype: personality
-  filekey: GitHubSecurityLab/seclab-taskflow-agent/personalities/examples/echo
 
 personality: |
   You are a simple echo bot. You use echo tools to echo things.
@@ -209,7 +188,7 @@ task: |
 
 # personality toolboxes map to mcp servers made available to this Agent
 toolboxes:
-  - GitHubSecurityLab/seclab-taskflow-agent/toolboxes/echo
+  - toolboxes.echo
 ```
 
 In the above, the `personality` and `task` field specifies the system prompt to be used whenever this `personality` is used.
@@ -219,7 +198,7 @@ files of the `filetype` `toolbox`.
 Personalities can be used in two ways. First it can be used standalone with a prompt input from the command line:
 
 ```
-python3 main.py -p GitHubSecurityLab/seclab-taskflow-agent/personalities/examples/echo "echo this message"
+python3 main.py -p personalities.examples.echo "echo this message"
 ```
 
 In this case, `personality` and `task` from `GitHubSecurityLab/seclab-taskflow-agent/personalities/examples/echo` are used as the 
@@ -233,29 +212,29 @@ taskflow:
   - task:
       ...
       agents:
-        - GitHubSecurityLab/seclab-taskflow-agent/personalities/assistant
+        - personalities.assistant
       user_prompt: |
         Fetch all the open pull requests from `github/codeql` github repository. 
         You do not need to provide a summary of the results.
       toolboxes:
-        - GitHubSecurityLab/seclab-taskflow-agent/toolboxes/github_official
+        - toolboxes.github_official
 ```
 
-In this case, the `personality` specified in `agents` provides the system prompt and the user prompt is specified in `user_prompt` field of the task. A big difference in this case is that the `toolboxes` specified in the `task` will overwrite the `toolboxes` that the `personality` has access to. So in the above example, the `GitHubSecurityLab/seclab-taskflow-agent/personalities/assistant` will have access to the `GitHubSecurityLab/seclab-taskflow-agent/toolboxes/github_official` toolbox instead of its own toolbox. It is important to note that the `personalities` toolboxes get *overwritten* in this case, so whenever a `toolboxes` field is provided in a `task`, it'll use the provided toolboxes and `personality` loses access to its own toolboxes. e.g.
+In this case, the `personality` specified in `agents` provides the system prompt and the user prompt is specified in `user_prompt` field of the task. A big difference in this case is that the `toolboxes` specified in the `task` will overwrite the `toolboxes` that the `personality` has access to. So in the above example, the `personalities.assistant` will have access to the `toolboxes.github_official` toolbox instead of its own toolbox. It is important to note that the `personalities` toolboxes get *overwritten* in this case, so whenever a `toolboxes` field is provided in a `task`, it'll use the provided toolboxes and `personality` loses access to its own toolboxes. e.g.
 
 ```yaml
 taskflow:
   - task:
       ...
       agents:
-        - GitHubSecurityLab/seclab-taskflow-agent/personalities/examples/echo
+        - personalities.examples.echo
       user_prompt: |
         echo this
       toolboxes:
-        - GitHubSecurityLab/seclab-taskflow-agent/toolboxes/github_official
+        - toolboxes.github_official
 ```
 
-In the above `task`, `GitHubSecurityLab/seclab-taskflow-agent/personalities/examples/echo` will only have access to the `GitHubSecurityLab/seclab-taskflow-agent/toolboxes/github_official` and can no longer access the `GitHubSecurityLab/seclab-taskflow-agent/toolboxes/echo` `toolbox`. (Unless it is added also in the `task` `toolboxes`)
+In the above `task`, `personalities.examples.echo` will only have access to the `toolboxes.github_official` and can no longer access the `toolboxes.echo` `toolbox`. (Unless it is added also in the `task` `toolboxes`)
 
 ## Toolboxes
 
@@ -269,7 +248,6 @@ For example, to start a stdio MCP server that are implemented in a python file:
 seclab-taskflow-agent:
   version: 1
   filetype: toolbox
-  filekey: toolboxes/echo
 
 server_params:
   kind: stdio
@@ -319,7 +297,6 @@ Example:
 seclab-taskflow-agent:
   version: 1
   filetype: taskflow
-  filekey: taskflows/examples/example.yaml
 
 taskflow:
   - task:
@@ -391,6 +368,13 @@ Taskflows support [Agent handoffs](https://openai.github.io/openai-agents-python
 
 See the [taskflow examples](taskflows/examples) for other useful Taskflow patterns such as repeatable and asynchronous templated prompts.
 
+
+You can run a taskflow from the command line like this:
+
+```
+python3 main.py -t taskflows.CVE-2023-2283.CVE-2023-2283
+```
+
 ## Prompt
 
 Prompts are configured through YAML files of `filetype` `prompt`. They define a reusable prompt that can be referenced in `taskflow` files.
@@ -401,13 +385,12 @@ They contain only one field, the `prompt` field, which is used to replace any `{
 seclab-taskflow-agent:
   version: 1
   filetype: prompt
-  filekey: GitHubSecurityLab/seclab-taskflow-agent/prompts/examples/example_prompt
 
 prompt: |
   Tell me more about bananas as well.
 ```
 
-would replace any `{{ PROMPT_GitHubSecurityLab/seclab-taskflow-agent/prompts/examples/example_prompt }}` template parameter found in the `user_prompt` section in a taskflow:
+would replace any `{{ PROMPT_prompts.examples.example_prompt }}` template parameter found in the `user_prompt` section in a taskflow:
 
 ```yaml
   - task:
@@ -416,7 +399,7 @@ would replace any `{{ PROMPT_GitHubSecurityLab/seclab-taskflow-agent/prompts/exa
       user_prompt: |
         Tell me more about apples.
 
-        {{ PROMPTS_GitHubSecurityLab/seclab-taskflow-agent/prompts/examples/example_prompt }}
+        {{ PROMPTS_prompts.examples.example_prompt }}
 ```
 
 becomes:
@@ -439,15 +422,14 @@ Model configs are configured through YAML files of `filetype` `model_config`. Th
 seclab-taskflow-agent:
   version: 1
   filetype: model_config
-  filekey: GitHubSecurityLab/seclab-taskflow-agent/configs/model_config
 models:
    gpt_latest: gpt-5
 ```
 
 A `model_config` file can be used in a `taskflow` and the values defined in `models` can then be used throughout.
 
 ```yaml
-model_config: GitHubSecurityLab/seclab-taskflow-agent/configs/model_config
+model_config: configs.model_config
 
 taskflow:
   - task:
@@ -478,7 +460,7 @@ taskflow:
   - task:
       must_complete: true
       agents:
-        - GitHubSecurityLab/seclab-taskflow-agent/personalities/assistant
+        - personalities.assistant
       user_prompt: |
         Store the json array ["apples", "oranges", "bananas"] in the `fruits` memory key.
       env:

available_tools.py

@@ -1,64 +1,88 @@
+from enum import Enum
 import logging
+import importlib.resources
+import yaml
 
-class VersionException(Exception):
+class BadToolNameError(Exception):
     pass
 
-class FileIDException(Exception):
+class VersionException(Exception):
     pass
 
 class FileTypeException(Exception):
     pass
 
-def add_yaml_to_dict(table, key, yaml):
-    """Add the yaml to the table, but raise an error if the id isn't unique """
-    if key in table:
-        raise FileIDException(str(key))
-    table.update({key: yaml})
+class AvailableToolType(Enum):
+    Personality = "personality"
+    Taskflow = "taskflow"
+    Prompt = "prompt"
+    Toolbox = "toolbox"
+    ModelConfig = "model_config"
 
 class AvailableTools:
     """
     This class is used for storing dictionaries of all the available
     personalities, taskflows, and prompts.
     """
-    def __init__(self, yamls: dict):
-        self.personalities = {}
-        self.taskflows = {}
-        self.prompts = {}
-        self.toolboxes = {}
-        self.model_config = {}
+    def __init__(self):
+        self.__yamlcache = {}
+
+    def get_personality(self, name: str):
+        return self.get_tool(AvailableToolType.Personality, name)
+
+    def get_taskflow(self, name: str):
+        return self.get_tool(AvailableToolType.Taskflow, name)
+
+    def get_prompt(self, name: str):
+        return self.get_tool(AvailableToolType.Prompt, name)
+
+    def get_toolbox(self, name: str):
+        return self.get_tool(AvailableToolType.Toolbox, name)
+
+    def get_model_config(self, name: str):
+        return self.get_tool(AvailableToolType.ModelConfig, name)
 
-        # Iterate through all the yaml files and divide them into categories.
-        # Each file should contain a header like this:
-        #
-        #   seclab-taskflow-agent:
-        #     type: taskflow
-        #     version: 1
-        #
-        for path, yaml in yamls.items():
-            try:
-                header = yaml['seclab-taskflow-agent']
+    def get_tool(self, tooltype: AvailableToolType, toolname: str):
+        """for example: available_tools.get_tool("personality", "personalities/fruit_expert")
+        This method first checks whether the tool has already been loaded. If not, it
+        finds the yaml file and parses it. It also checks that the filetype in the header
+        matches the expected tooltype.
+        """
+        try:
+            return self.__yamlcache[tooltype][toolname]
+        except KeyError:
+            pass
+        # Split the string to get the path and filename.
+        components = toolname.rsplit('.', 1)
+        if len(components) == 2:
+            path = components[0]
+            filename = components[1]
+        else:
+            path = ''
+            filename = toolname
+        try:
+            d = importlib.resources.files(path)
+            if not d.is_dir():
+                raise BadToolNameError(f'Cannot load {toolname} because {d} is not a valid directory.')
+            f = d.joinpath(filename + ".yaml")
+            with open(f) as s:
+                y = yaml.safe_load(s)
+                header = y['seclab-taskflow-agent']
                 version = header['version']
                 if version != 1:
                     raise VersionException(str(version))
-                filekey = header['filekey']
-                filetype = header['filetype']
-                if filetype == 'personality':
-                    add_yaml_to_dict(self.personalities, filekey, yaml)
-                elif filetype == 'taskflow':
-                    add_yaml_to_dict(self.taskflows, filekey, yaml)
-                elif filetype == 'prompt':
-                    add_yaml_to_dict(self.prompts, filekey, yaml)
-                elif filetype == 'toolbox':
-                    add_yaml_to_dict(self.toolboxes, filekey, yaml)
-                elif filetype == 'model_config':
-                    add_yaml_to_dict(self.model_config, filekey, yaml)
-                else:
-                    raise FileTypeException(str(filetype))
-            except KeyError as err:
-                logging.error(f'{path} does not contain the key {err.args[0]}')
-            except VersionException as err:
-                logging.error(f'{path}: seclab-taskflow-agent version {err.args[0]} is not supported')
-            except FileIDException as err:
-                logging.error(f'{path}: file ID {err.args[0]} is not unique')
-            except FileTypeException as err:
-                logging.error(f'{path}: seclab-taskflow-agent file type {err.args[0]} is not supported')
+                filetype = header['filetype'] 
+                if filetype != tooltype.value:
+                    raise FileTypeException(
+                        f'Error in {f}: expected filetype to be {tooltype}, but it\'s {filetype}.')
+                if tooltype not in self.__yamlcache:
+                    self.__yamlcache[tooltype] = {}
+                self.__yamlcache[tooltype][toolname] = y
+                return y
+        except ModuleNotFoundError as e:
+            raise BadToolNameError(f'Cannot load {toolname}: {e}')
+        except FileNotFoundError:
+            # deal with editor temp files etc. that might have disappeared
+            raise BadToolNameError(f'Cannot load {toolname} because {f} is not a valid file.')
+        except ValueError as e:
+            raise BadToolNameError(f'Cannot load {toolname}: {e}')

configs/model_config.yaml

@@ -1,7 +1,6 @@
 seclab-taskflow-agent:
   version: 1
   filetype: model_config
-  filekey: GitHubSecurityLab/seclab-taskflow-agent/configs/model_config
 models:
    sonnet_default: claude-sonnet-4
    sonnet_latest: claude-sonnet-4.5