From 85158a385cb9ea611f3edd60f452012672fda5e2 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Mon, 17 Oct 2016 20:59:51 -0400 Subject: [PATCH 01/22] Beta support for configurable dependency resolution & Biocontainers. Consider the included tool ``seqtk_seq.cwl``. It includes the following SoftwareRequirement hint: ``` hints: SoftwareRequirement: packages: - package: seqtk version: - 1.2 ``` I'm not happy that ``version`` is a list - but I can live with it for now I guess. If cwltool is executed with the hidden ``--beta-conda-dependencies`` flag, this requirement will be processed by galaxy-lib, Conda will be installed, and seqtk will be installed, and a Conda environment including seqtk will be setup for the job. ``` virtualenv .venv . .venv/bin/activate python setup.py install pip install galaxy-lib cwltool --beta-conda-dependencies tests/seqtk_seq.cwl tests/seqtk_seq_job.json ``` Additional flags are available to configure dependency resolution in a more fine grained way - using Conda however has a number of advantages that make it particularily well suited to CWL. Conda packages are distributed as binaries that work across Mac and Linux and work on relatively old version of Linux (great for HPC). Conda also doesn't require root and supports installation of multiple different versions of a package - again these factors make it great for HPC and non-Docker targets. The Biocontainers project (previously Biodocker) dovetails nicely with this. Every version of every Bioconda package has a corresponding best-practice (very lightweight, very small) Docker container on quay.io (assembled by @bgruening and colleagues). There are over 1800 such containers currently. Continuing with the example above, the new ``--beta-use-biocontainers`` flag instructs cwltool to fetch the corresponding Biocontainers container from quay.io automatically or build one to use locally (required for instance for tools with multiple software requirements - fat tools). ``` cwltool --beta-use-biocontainers tests/seqtk_seq.cwl tests/seqtk_seq_job.json ``` These containers contain the same binaries that the package would use locally (outside of Docker). Therefore this technique allows cross platform reproducibility/remixability across CWL, Galaxy, and CLI - both inside and outside of Docker. My sincerest hope is that we move away from CWL-specific Dockerfiles. For less effort, a community bioconda package can be made and the result can be used in many more contexts. The Docker image will then be maintained by the community Biocontainer project. Rebased with correct spelling of DependenciesConfiguration thanks to @tetron. --- cwltool/builder.py | 2 + cwltool/draft2tool.py | 12 +- cwltool/job.py | 11 +- cwltool/main.py | 133 +++++++++++++++++- cwltool/process.py | 6 + tests/2.fasta | 11 ++ tests/2.fastq | 12 ++ tests/seqtk_seq.cwl | 24 ++++ tests/seqtk_seq_job.json | 6 + typeshed/2.7/galaxy/__init__.pyi | 13 ++ typeshed/2.7/galaxy/tools/__init__.pyi | 4 + typeshed/2.7/galaxy/tools/deps/__init__.pyi | 32 +++++ typeshed/2.7/galaxy/tools/deps/brew_exts.pyi | 74 ++++++++++ typeshed/2.7/galaxy/tools/deps/commands.pyi | 23 +++ typeshed/2.7/galaxy/tools/deps/conda_util.pyi | 61 ++++++++ .../deps/container_resolvers/__init__.pyi | 15 ++ .../deps/container_resolvers/explicit.pyi | 12 ++ .../tools/deps/container_resolvers/mulled.pyi | 30 ++++ typeshed/2.7/galaxy/tools/deps/containers.pyi | 86 +++++++++++ .../2.7/galaxy/tools/deps/docker_util.pyi | 41 ++++++ .../2.7/galaxy/tools/deps/requirements.pyi | 35 +++++ .../galaxy/tools/deps/resolvers/__init__.pyi | 40 ++++++ .../2.7/galaxy/tools/deps/resolvers/conda.pyi | 40 ++++++ .../tools/deps/resolvers/galaxy_packages.pyi | 33 +++++ .../tools/deps/resolvers/resolver_mixins.pyi | 18 +++ .../deps/resolvers/tool_shed_packages.pyi | 13 ++ typeshed/2.7/galaxy/util/__init__.pyi | 131 +++++++++++++++++ typeshed/2.7/galaxy/util/dictifiable.pyi | 8 ++ typeshed/2.7/galaxy/util/inflection.pyi | 46 ++++++ 29 files changed, 961 insertions(+), 11 deletions(-) create mode 100644 tests/2.fasta create mode 100644 tests/2.fastq create mode 100644 tests/seqtk_seq.cwl create mode 100644 tests/seqtk_seq_job.json create mode 100644 typeshed/2.7/galaxy/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/brew_exts.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/commands.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/conda_util.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/containers.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/docker_util.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/requirements.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi create mode 100644 typeshed/2.7/galaxy/util/__init__.pyi create mode 100644 typeshed/2.7/galaxy/util/dictifiable.pyi create mode 100644 typeshed/2.7/galaxy/util/inflection.pyi diff --git a/cwltool/builder.py b/cwltool/builder.py index 52693dbcd..76a46a146 100644 --- a/cwltool/builder.py +++ b/cwltool/builder.py @@ -50,6 +50,8 @@ def __init__(self): # type: () -> None # Will be default "no_listing" for CWL v1.1 self.loadListing = "deep_listing" # type: Union[None, str] + self.find_default_container = None # type: Callable[[], Text] + def bind_input(self, schema, datum, lead_pos=None, tail_pos=None): # type: (Dict[Text, Any], Any, Union[int, List[int]], List[int]) -> List[Dict[Text, Any]] if tail_pos is None: diff --git a/cwltool/draft2tool.py b/cwltool/draft2tool.py index 2aa9ce388..1a1e4d160 100644 --- a/cwltool/draft2tool.py +++ b/cwltool/draft2tool.py @@ -174,9 +174,19 @@ class CommandLineTool(Process): def __init__(self, toolpath_object, **kwargs): # type: (Dict[Text, Any], **Any) -> None super(CommandLineTool, self).__init__(toolpath_object, **kwargs) + self.find_default_container = kwargs["find_default_container"] def makeJobRunner(self, use_container=True): # type: (Optional[bool]) -> JobBase dockerReq, _ = self.get_requirement("DockerRequirement") + if not dockerReq and use_container: + default_container = self.find_default_container(self) + if default_container: + self.requirements.insert(0, { + "class": "DockerRequirement", + "dockerPull": default_container + }) + dockerReq = self.requirements[0] + if dockerReq and use_container: return DockerCommandLineJob() else: @@ -283,7 +293,7 @@ def rm_pending_output_callback(output_callbacks, jobcachepending, reffiles = copy.deepcopy(builder.files) - j = self.makeJobRunner(kwargs.get("use_container")) + j = self.makeJobRunner(kwargs.get("find_default_container")) j.builder = builder j.joborder = builder.job j.stdin = None diff --git a/cwltool/job.py b/cwltool/job.py index 60573cbad..647803d2b 100644 --- a/cwltool/job.py +++ b/cwltool/job.py @@ -33,6 +33,7 @@ PYTHON_RUN_SCRIPT = """ import json +import os import sys import subprocess @@ -41,6 +42,7 @@ commands = popen_description["commands"] cwd = popen_description["cwd"] env = popen_description["env"] + env["PATH"] = os.environ.get("PATH") stdin_path = popen_description["stdin_path"] stdout_path = popen_description["stdout_path"] stderr_path = popen_description["stderr_path"] @@ -145,7 +147,6 @@ def _setup(self): # type: () -> None _logger.debug(u"[job %s] initial work dir %s", self.name, json.dumps({p: self.generatemapper.mapper(p) for p in self.generatemapper.files()}, indent=4)) - def _execute(self, runtime, env, rm_tmpdir=True, move_outputs="move"): # type: (List[Text], MutableMapping[Text, Text], bool, Text) -> None @@ -328,8 +329,12 @@ def run(self, pull_image=True, rm_container=True, env = cast(MutableMapping[Text, Text], os.environ) if docker_req and kwargs.get("use_container") is not False: img_id = docker.get_from_requirements(docker_req, True, pull_image) - elif kwargs.get("default_container", None) is not None: - img_id = kwargs.get("default_container") + if img_id is None: + find_default_container = self.builder.find_default_container + default_container = find_default_container and find_default_container() + if default_container: + img_id = default_container + env = os.environ if docker_req and img_id is None and kwargs.get("use_container"): raise Exception("Docker image not available") diff --git a/cwltool/main.py b/cwltool/main.py index b7ba1e24e..ad3fded0c 100755 --- a/cwltool/main.py +++ b/cwltool/main.py @@ -13,6 +13,7 @@ import pkg_resources # part of setuptools import requests +import string import ruamel.yaml as yaml import schema_salad.validate as validate @@ -33,6 +34,14 @@ from .resolver import ga4gh_tool_registries, tool_resolver from .stdfsaccess import StdFsAccess from .update import ALLUPDATES, UPDATES +from .utils import get_feature +try: + from galaxy.tools.deps.requirements import ToolRequirement, ToolRequirements + from galaxy.tools import deps +except ImportError: + ToolRequirement = None # type: ignore + ToolRequirements = None # type: ignore + deps = None _logger = logging.getLogger("cwltool") @@ -149,6 +158,15 @@ def arg_parser(): # type: () -> argparse.ArgumentParser exgroup.add_argument("--quiet", action="store_true", help="Only print warnings and errors.") exgroup.add_argument("--debug", action="store_true", help="Print even more logging") + # help="Dependency resolver configuration file describing how to adapt 'SoftwareRequirement' packages to current system." + parser.add_argument("--beta-dependency-resolvers-configuration", default=None, help=argparse.SUPPRESS) + # help="Defaut root directory used by dependency resolvers configuration." + parser.add_argument("--beta-dependencies-directory", default=None, help=argparse.SUPPRESS) + # help="Use biocontainers for tools without an explicitly annotated Docker container." + parser.add_argument("--beta-use-biocontainers", default=None, help=argparse.SUPPRESS, action="store_true") + # help="Short cut to use Conda to resolve 'SoftwareRequirement' packages." + parser.add_argument("--beta-conda-dependencies", default=None, help=argparse.SUPPRESS, action="store_true") + parser.add_argument("--tool-help", action="store_true", help="Print command line help for tool") parser.add_argument("--relative-deps", choices=['primary', 'cwd'], @@ -236,12 +254,6 @@ def output_callback(out, processStatus): for req in jobReqs: t.requirements.append(req) - if kwargs.get("default_container"): - t.requirements.insert(0, { - "class": "DockerRequirement", - "dockerPull": kwargs["default_container"] - }) - jobiter = t.job(job_order_object, output_callback, **kwargs) @@ -716,8 +728,20 @@ def main(argsl=None, # type: List[str] stdout.write(json.dumps(processobj, indent=4)) return 0 + conf_file = getattr(args, "beta_dependency_resolvers_configuration", None) # Text + use_conda_dependencies = getattr(args, "beta_conda_dependencies", None) # Text + + make_tool_kwds = vars(args) + + build_job_script = None # type: Callable[[Any, List[str]], Text] + if conf_file or use_conda_dependencies: + dependencies_configuration = DependenciesConfiguration(args) # type: DependenciesConfiguration + make_tool_kwds["build_job_script"] = dependencies_configuration.build_job_script + + make_tool_kwds["find_default_container"] = functools.partial(find_default_container, args) + tool = make_tool(document_loader, avsc_names, metadata, uri, - makeTool, vars(args)) + makeTool, make_tool_kwds) if args.validate: return 0 @@ -838,5 +862,100 @@ def locToPath(p): _logger.addHandler(defaultStreamHandler) +COMMAND_WITH_DEPENDENCIES_TEMPLATE = string.Template("""#!/bin/bash +$handle_dependencies +python "run_job.py" "job.json" +""") + + +def find_default_container(args, builder): + if args.default_container: + return args.default_container + elif args.beta_use_biocontainers: + try: + from galaxy.tools.deps.containers import ContainerRegistry, AppInfo, ToolInfo, DOCKER_CONTAINER_TYPE + except ImportError: + raise Exception("galaxy-lib not found") + + app_info = AppInfo( + involucro_auto_init=True, + enable_beta_mulled_containers=True, + container_image_cache_path=".", + ) # type: AppInfo + container_registry = ContainerRegistry(app_info) # type: ContainerRegistry + requirements = _get_dependencies(builder) + tool_info = ToolInfo(requirements=requirements) # type: ToolInfo + container_description = container_registry.find_best_container_description([DOCKER_CONTAINER_TYPE], tool_info) + if container_description: + return container_description.identifier + + return None + + +class DependenciesConfiguration(object): + + def __init__(self, args): + # type: (argparse.Namespace) -> None + conf_file = getattr(args, "beta_dependency_resolvers_configuration", None) + tool_dependency_dir = getattr(args, "beta_dependencies_directory", None) + conda_dependencies = getattr(args, "beta_conda_dependencies", None) + if conf_file is not None and os.path.exists(conf_file): + self.use_tool_dependencies = True + if not tool_dependency_dir: + tool_dependency_dir = os.path.abspath(os.path.dirname(conf_file)) + self.tool_dependency_dir = tool_dependency_dir + self.dependency_resolvers_config_file = conf_file + elif conda_dependencies: + if not tool_dependency_dir: + tool_dependency_dir = os.path.abspath("./cwltool_deps") + self.tool_dependency_dir = tool_dependency_dir + self.use_tool_dependencies = True + self.dependency_resolvers_config_file = None + else: + self.use_tool_dependencies = False + + @property + def config_dict(self): + return { + 'conda_auto_install': True, + 'conda_auto_init': True, + } + + def build_job_script(self, builder, command): + # type: (Any, List[str]) -> Text + if deps is None: + raise Exception("galaxy-lib not found") + tool_dependency_manager = deps.build_dependency_manager(self) # type: deps.DependencyManager + dependencies = _get_dependencies(builder) + handle_dependencies = "" # str + if dependencies: + handle_dependencies = "\n".join(tool_dependency_manager.dependency_shell_commands(dependencies, job_directory=builder.tmpdir)) + + template_kwds = dict(handle_dependencies=handle_dependencies) # type: Dict[str, str] + job_script = COMMAND_WITH_DEPENDENCIES_TEMPLATE.substitute(template_kwds) + return job_script + + +def _get_dependencies(builder): + # type: (Any) -> List[ToolRequirement] + (software_requirement, _) = get_feature(builder, "SoftwareRequirement") + dependencies = [] # type: List[ToolRequirement] + if software_requirement and software_requirement.get("packages"): + packages = software_requirement.get("packages") + for package in packages: + version = package.get("version", None) + if isinstance(version, list): + if version: + version = version[0] + else: + version = None + dependencies.append(ToolRequirement.from_dict(dict( + name=package["package"].split("#")[-1], + version=version, + type="package", + ))) + return ToolRequirements.from_list(dependencies) + + if __name__ == "__main__": sys.exit(main(sys.argv[1:])) diff --git a/cwltool/process.py b/cwltool/process.py index fcf78615a..7fdd99d3d 100644 --- a/cwltool/process.py +++ b/cwltool/process.py @@ -598,6 +598,12 @@ def _init_job(self, joborder, **kwargs): builder.resources = self.evalResources(builder, kwargs) + build_job_script = kwargs.get("build_job_script", None) # type: Callable[[Builder, List[str]], Text] + curried_build_job_script = None # type: Callable[[List[str]], Text] + if build_job_script: + curried_build_job_script = lambda commands: build_job_script(builder, commands) + builder.build_job_script = curried_build_job_script + return builder def evalResources(self, builder, kwargs): diff --git a/tests/2.fasta b/tests/2.fasta new file mode 100644 index 000000000..3bfe7d3d3 --- /dev/null +++ b/tests/2.fasta @@ -0,0 +1,11 @@ +>Sequence 561 BP; 135 A; 106 C; 98 G; 222 T; 0 other; +gttcgatgcc taaaatacct tcttttgtcc ctacacagac cacagttttc ctaatggctt +tacaccgact agaaattctt gtgcaagcac taattgaaag cggttggcct agagtgttac +cggtttgtat agctgagcgc gtctcttgcc ctgatcaaag gttcattttc tctactttgg +aagacgttgt ggaagaatac aacaagtacg agtctctccc ccctggtttg ctgattactg +gatacagttg taataccctt cgcaacaccg cgtaactatc tatatgaatt attttccctt +tattatatgt agtaggttcg tctttaatct tcctttagca agtcttttac tgttttcgac +ctcaatgttc atgttcttag gttgttttgg ataatatgcg gtcagtttaa tcttcgttgt +ttcttcttaa aatatttatt catggtttaa tttttggttt gtacttgttc aggggccagt +tcattattta ctctgtttgt atacagcagt tcttttattt ttagtatgat tttaatttaa +aacaattcta atggtcaaaa a \ No newline at end of file diff --git a/tests/2.fastq b/tests/2.fastq new file mode 100644 index 000000000..436c05ff2 --- /dev/null +++ b/tests/2.fastq @@ -0,0 +1,12 @@ +@EAS54_6_R1_2_1_413_324 +CCCTTCTTGTCTTCAGCGTTTCTCC ++ +;;3;;;;;;;;;;;;7;;;;;;;88 +@EAS54_6_R1_2_1_540_792 +TTGGCAGGCCAAGGCCGATGGATCA ++ +;;;;;;;;;;;7;;;;;-;;;3;83 +@EAS54_6_R1_2_1_443_348 +GTTGCTTCTGGCGTGGGTGGGGGGG ++EAS54_6_R1_2_1_443_348 +;;;;;;;;;;;9;7;;.7;393333 \ No newline at end of file diff --git a/tests/seqtk_seq.cwl b/tests/seqtk_seq.cwl new file mode 100644 index 000000000..b97d6c25e --- /dev/null +++ b/tests/seqtk_seq.cwl @@ -0,0 +1,24 @@ +cwlVersion: v1.0 +class: CommandLineTool +id: "seqtk_seq" +doc: "Convert to FASTA (seqtk)" +inputs: + - id: input1 + type: File + inputBinding: + position: 1 + prefix: "-a" +outputs: + - id: output1 + type: File + outputBinding: + glob: out +baseCommand: ["seqtk", "seq"] +arguments: [] +stdout: out +hints: + SoftwareRequirement: + packages: + - package: seqtk + version: + - r93 diff --git a/tests/seqtk_seq_job.json b/tests/seqtk_seq_job.json new file mode 100644 index 000000000..79ea46c37 --- /dev/null +++ b/tests/seqtk_seq_job.json @@ -0,0 +1,6 @@ +{ + "input1": { + "class": "File", + "location": "2.fastq" + } +} diff --git a/typeshed/2.7/galaxy/__init__.pyi b/typeshed/2.7/galaxy/__init__.pyi new file mode 100644 index 000000000..78ce81a70 --- /dev/null +++ b/typeshed/2.7/galaxy/__init__.pyi @@ -0,0 +1,13 @@ +# Stubs for galaxy (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +PROJECT_NAME = ... # type: str +PROJECT_OWNER = ... # type: str +PROJECT_USERAME = ... # type: str +PROJECT_URL = ... # type: str +PROJECT_AUTHOR = ... # type: str +PROJECT_EMAIL = ... # type: str +RAW_CONTENT_URL = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/__init__.pyi b/typeshed/2.7/galaxy/tools/__init__.pyi new file mode 100644 index 000000000..af6879982 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/__init__.pyi @@ -0,0 +1,4 @@ +# Stubs for galaxy.tools (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + diff --git a/typeshed/2.7/galaxy/tools/deps/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/__init__.pyi new file mode 100644 index 000000000..b3bd75ccf --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/__init__.pyi @@ -0,0 +1,32 @@ +# Stubs for galaxy.tools.deps (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from .resolvers import NullDependency as NullDependency +from .resolvers.conda import CondaDependencyResolver as CondaDependencyResolver +from .resolvers.galaxy_packages import GalaxyPackageDependencyResolver as GalaxyPackageDependencyResolver +from .resolvers.tool_shed_packages import ToolShedPackageDependencyResolver as ToolShedPackageDependencyResolver + +log = ... # type: Any +EXTRA_CONFIG_KWDS = ... # type: Any +CONFIG_VAL_NOT_FOUND = ... # type: Any + +def build_dependency_manager(config: Any): ... # type: DependencyManager + +class NullDependencyManager: + dependency_resolvers = ... # type: Any + def uses_tool_shed_dependencies(self): ... + def dependency_shell_commands(self, requirements: Any, **kwds) -> List[str]: ... + def find_dep(self, name, version: Optional[Any] = ..., type: str = ..., **kwds): ... + +class DependencyManager: + extra_config = ... # type: Any + default_base_path = ... # type: Any + resolver_classes = ... # type: Any + dependency_resolvers = ... # type: Any + def __init__(self, default_base_path, conf_file: Optional[Any] = ..., **extra_config) -> None: ... + def dependency_shell_commands(self, requirements: Any, **kwds) -> List[str]: ... + def requirements_to_dependencies(self, requirements, **kwds): ... + def uses_tool_shed_dependencies(self): ... + def find_dep(self, name, version: Optional[Any] = ..., type: str = ..., **kwds): ... diff --git a/typeshed/2.7/galaxy/tools/deps/brew_exts.pyi b/typeshed/2.7/galaxy/tools/deps/brew_exts.pyi new file mode 100644 index 000000000..f2074d55d --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/brew_exts.pyi @@ -0,0 +1,74 @@ +# Stubs for galaxy.tools.deps.brew_exts (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +WHITESPACE_PATTERN = ... # type: Any +DESCRIPTION = ... # type: str +DEFAULT_HOMEBREW_ROOT = ... # type: str +NO_BREW_ERROR_MESSAGE = ... # type: str +CANNOT_DETERMINE_TAP_ERROR_MESSAGE = ... # type: str +VERBOSE = ... # type: bool +RELAXED = ... # type: bool +BREW_ARGS = ... # type: Any + +class BrewContext: + homebrew_prefix = ... # type: Any + homebrew_cellar = ... # type: Any + def __init__(self, args: Optional[Any] = ...) -> None: ... + +class RecipeContext: + @staticmethod + def from_args(args, brew_context: Optional[Any] = ...): ... + recipe = ... # type: Any + version = ... # type: Any + brew_context = ... # type: Any + def __init__(self, recipe, version, brew_context: Optional[Any] = ...) -> None: ... + @property + def cellar_path(self): ... + @property + def tap_path(self): ... + +def main(): ... + +class CommandLineException(Exception): + command = ... # type: Any + stdout = ... # type: Any + stderr = ... # type: Any + message = ... # type: Any + def __init__(self, command, stdout, stderr) -> None: ... + +def versioned_install(recipe_context, package: Optional[Any] = ..., version: Optional[Any] = ..., installed_deps: Any = ...): ... +def commit_for_version(recipe_context, package, version): ... +def print_versioned_deps(recipe_context, recipe, version): ... +def load_versioned_deps(cellar_path, relaxed: Optional[Any] = ...): ... +def unversioned_install(package): ... +def attempt_unlink_all(package, deps): ... +def attempt_unlink(package): ... +def brew_execute(args, env: Optional[Any] = ...): ... +def build_env_statements_from_recipe_context(recipe_context, **kwds): ... +def build_env_statements(cellar_root, cellar_path, relaxed: Optional[Any] = ..., custom_only: bool = ...): ... +def build_env_actions(deps, cellar_root, cellar_path, relaxed: Optional[Any] = ..., custom_only: bool = ...): ... + +class EnvAction: + variable = ... # type: Any + action = ... # type: Any + value = ... # type: Any + def __init__(self, keg_root, action_description) -> None: ... + @staticmethod + def build_env(env_actions): ... + def modify_environ(self, environ): ... + def to_statements(self): ... + +def brew_head_at_version(recipe_context, package, version): ... +def brew_head_at_commit(commit, tap_path): ... +def git_execute(args): ... +def execute(cmds, env: Optional[Any] = ...): ... +def brew_deps(package): ... +def brew_info(recipe): ... +def extended_brew_info(recipe): ... +def brew_versions_info(package, tap_path): ... +def recipe_cellar_path(cellar_path, recipe, version): ... +def ensure_brew_on_path(args): ... +def which(file): ... diff --git a/typeshed/2.7/galaxy/tools/deps/commands.pyi b/typeshed/2.7/galaxy/tools/deps/commands.pyi new file mode 100644 index 000000000..aad9cae42 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/commands.pyi @@ -0,0 +1,23 @@ +# Stubs for galaxy.tools.deps.commands (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from galaxy.util import which as which +def shell_quote(args: Any = ...): ... + +def redirecting_io(sys: Any = ...): ... +def redirect_aware_commmunicate(p, sys: Any = ...): ... +def shell(cmds, env: Optional[Any] = ..., **kwds): ... +def shell_process(cmds, env: Optional[Any] = ..., **kwds): ... +def execute(cmds): ... +def argv_to_str(command_argv, quote: bool = ...): ... +def download_command(url, to: Any = ..., quote_url: bool = ...): ... + +class CommandLineException(Exception): + command = ... # type: Any + stdout = ... # type: Any + stderr = ... # type: Any + message = ... # type: Any + def __init__(self, command, stdout, stderr) -> None: ... + diff --git a/typeshed/2.7/galaxy/tools/deps/conda_util.pyi b/typeshed/2.7/galaxy/tools/deps/conda_util.pyi new file mode 100644 index 000000000..55082a96a --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/conda_util.pyi @@ -0,0 +1,61 @@ +# Stubs for galaxy.tools.deps.conda_util (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +import installable +from sys import platform as _platform + +class CondaContext(installable.InstallableContext): + installable_description = ... # type: str + condarc_override = ... # type: Any + conda_exec = ... # type: Any + debug = ... # type: Any + shell_exec = ... # type: Any + conda_prefix = ... # type: Any + ensure_channels = ... # type: Any + ensured_channels = ... # type: bool + def __init__(self, conda_prefix: Optional[Any] = ..., conda_exec: Optional[Any] = ..., shell_exec: Optional[Any] = ..., debug: bool = ..., ensure_channels: str = ..., condarc_override: Optional[Any] = ..., use_path_exec: Any = ...) -> None: ... + def ensure_channels_configured(self): ... + def conda_info(self): ... + def is_conda_installed(self): ... + def can_install_conda(self): ... + def load_condarc(self): ... + def save_condarc(self, conf): ... + @property + def condarc(self): ... + def command(self, operation, args): ... + def exec_command(self, operation, args): ... + def exec_create(self, args): ... + def exec_remove(self, args): ... + def exec_install(self, args): ... + def export_list(self, name, path): ... + def env_path(self, env_name): ... + @property + def envs_path(self): ... + def has_env(self, env_name): ... + @property + def deactivate(self): ... + @property + def activate(self): ... + def is_installed(self): ... + def can_install(self): ... + @property + def parent_path(self): ... + +class CondaTarget: + package = ... # type: Any + version = ... # type: Any + channel = ... # type: Any + def __init__(self, package, version: Optional[Any] = ..., channel: Optional[Any] = ...) -> None: ... + @property + def package_specifier(self): ... + @property + def install_environment(self): ... + def __hash__(self): ... + def __eq__(self, other): ... + def __ne__(self, other): ... + +def install_conda(conda_context: Optional[Any] = ...): ... +def install_conda_target(conda_target, conda_context: Optional[Any] = ...): ... +def requirements_to_conda_targets(requirements, conda_context: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi new file mode 100644 index 000000000..045ccdb39 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi @@ -0,0 +1,15 @@ +# Stubs for galaxy.tools.deps.container_resolvers (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from galaxy.util.dictifiable import Dictifiable + +class ContainerResolver(Dictifiable): + dict_collection_visible_keys = ... # type: Any + __metaclass__ = ... # type: Any + app_info = ... # type: Any + resolver_kwds = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., **kwds) -> None: ... + def resolve(self, tool_info): ... + def resolver_type(self): ... diff --git a/typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi b/typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi new file mode 100644 index 000000000..511140171 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi @@ -0,0 +1,12 @@ +# Stubs for galaxy.tools.deps.container_resolvers.explicit (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from ..container_resolvers import ContainerResolver as ContainerResolver + +log = ... # type: Any + +class ExplicitContainerResolver(ContainerResolver): + resolver_type = ... # type: str + def resolve(self, enabled_container_types, tool_info): ... diff --git a/typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi b/typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi new file mode 100644 index 000000000..0c04bd898 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi @@ -0,0 +1,30 @@ +# Stubs for galaxy.tools.deps.container_resolvers.mulled (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..container_resolvers import ContainerResolver +from collections import namedtuple + +CachedMulledImageSingleTarget = namedtuple('CachedMulledImageSingleTarget', ['package_name', 'version', 'build', 'image_identifier']) + +CachedMulledImageMultiTarget = namedtuple('CachedMulledImageMultiTarget', ['hash', 'image_identifier']) + +class CachedMulledContainerResolver(ContainerResolver): + resolver_type = ... # type: str + namespace = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., namespace: Optional[Any] = ...) -> None: ... + def resolve(self, enabled_container_types, tool_info): ... + +class MulledContainerResolver(ContainerResolver): + resolver_type = ... # type: str + namespace = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., namespace: str = ...) -> None: ... + def resolve(self, enabled_container_types, tool_info): ... + +class BuildMulledContainerResolver(ContainerResolver): + resolver_type = ... # type: str + namespace = ... # type: Any + auto_init = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., namespace: str = ..., **kwds) -> None: ... + def resolve(self, enabled_container_types, tool_info): ... diff --git a/typeshed/2.7/galaxy/tools/deps/containers.pyi b/typeshed/2.7/galaxy/tools/deps/containers.pyi new file mode 100644 index 000000000..4eecb5440 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/containers.pyi @@ -0,0 +1,86 @@ +# Stubs for galaxy.tools.deps.containers (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from .container_resolvers.explicit import ExplicitContainerResolver as ExplicitContainerResolver +from .container_resolvers.mulled import BuildMulledContainerResolver as BuildMulledContainerResolver, CachedMulledContainerResolver as CachedMulledContainerResolver, MulledContainerResolver as MulledContainerResolver +from .requirements import ContainerDescription as ContainerDescription +from .requirements import DEFAULT_CONTAINER_RESOLVE_DEPENDENCIES as DEFAULT_CONTAINER_RESOLVE_DEPENDENCIES, DEFAULT_CONTAINER_SHELL as DEFAULT_CONTAINER_SHELL +from ..deps import docker_util as docker_util + +log = ... # type: Any +DOCKER_CONTAINER_TYPE = ... # type: str +DEFAULT_CONTAINER_TYPE = ... # type: Any +ALL_CONTAINER_TYPES = ... # type: Any +LOAD_CACHED_IMAGE_COMMAND_TEMPLATE = ... # type: str + +class ContainerFinder: + app_info = ... # type: Any + container_registry = ... # type: Any + def __init__(self, app_info) -> None: ... + def find_container(self, tool_info, destination_info, job_info): ... + +class NullContainerFinder: + def find_container(self, tool_info, destination_info, job_info): ... + +class ContainerRegistry: + resolver_classes = ... # type: Any + enable_beta_mulled_containers = ... # type: Any + app_info = ... # type: Any + container_resolvers = ... # type: Any + def __init__(self, app_info) -> None: ... + def find_best_container_description(self, enabled_container_types, tool_info): ... + +class AppInfo: + galaxy_root_dir = ... # type: Any + default_file_path = ... # type: Any + outputs_to_working_directory = ... # type: Any + container_image_cache_path = ... # type: Any + library_import_dir = ... # type: Any + enable_beta_mulled_containers = ... # type: Any + containers_resolvers_config_file = ... # type: Any + involucro_path = ... # type: Any + involucro_auto_init = ... # type: Any + def __init__(self, galaxy_root_dir: Optional[Any] = ..., default_file_path: Optional[Any] = ..., outputs_to_working_directory: bool = ..., container_image_cache_path: Optional[Any] = ..., library_import_dir: Optional[Any] = ..., enable_beta_mulled_containers: bool = ..., containers_resolvers_config_file: Optional[Any] = ..., involucro_path: Optional[Any] = ..., involucro_auto_init: bool = ...) -> None: ... + +class ToolInfo: + container_descriptions = ... # type: Any + requirements = ... # type: Any + env_pass_through = ... # type: Any + def __init__(self, container_descriptions: Any = ..., requirements: Any = ...) -> None: ... + +class JobInfo: + working_directory = ... # type: Any + job_directory = ... # type: Any + tool_directory = ... # type: Any + job_directory_type = ... # type: Any + def __init__(self, working_directory, tool_directory, job_directory, job_directory_type) -> None: ... + +class Container: + __metaclass__ = ... # type: Any + container_id = ... # type: Any + app_info = ... # type: Any + tool_info = ... # type: Any + destination_info = ... # type: Any + job_info = ... # type: Any + container_description = ... # type: Any + def __init__(self, container_id, app_info, tool_info, destination_info, job_info, container_description) -> None: ... + @property + def resolve_dependencies(self): ... + @property + def shell(self): ... + def containerize_command(self, command): ... + +class DockerContainer(Container): + def containerize_command(self, command): ... + +def docker_cache_path(cache_directory, container_id): ... + +CONTAINER_CLASSES = ... # type: Any + +class NullContainer: + def __init__(self) -> None: ... + def __nonzero__(self): ... + +NULL_CONTAINER = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/deps/docker_util.pyi b/typeshed/2.7/galaxy/tools/deps/docker_util.pyi new file mode 100644 index 000000000..c6ea13d51 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/docker_util.pyi @@ -0,0 +1,41 @@ +# Stubs for galaxy.tools.deps.docker_util (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from .commands import argv_to_str as argv_to_str, shell_quote as shell_quote + +DEFAULT_DOCKER_COMMAND = ... # type: str +DEFAULT_SUDO = ... # type: bool +DEFAULT_SUDO_COMMAND = ... # type: str +DEFAULT_HOST = ... # type: Any +DEFAULT_VOLUME_MOUNT_TYPE = ... # type: str +DEFAULT_WORKING_DIRECTORY = ... # type: Any +DEFAULT_NET = ... # type: Any +DEFAULT_MEMORY = ... # type: Any +DEFAULT_VOLUMES_FROM = ... # type: Any +DEFAULT_AUTO_REMOVE = ... # type: bool +DEFAULT_SET_USER = ... # type: str +DEFAULT_RUN_EXTRA_ARGUMENTS = ... # type: Any + +class DockerVolume: + from_path = ... # type: Any + to_path = ... # type: Any + how = ... # type: Any + def __init__(self, path, to_path: Optional[Any] = ..., how: Any = ...) -> None: ... + @staticmethod + def volumes_from_str(volumes_as_str): ... + @staticmethod + def volume_from_str(as_str): ... + +def kill_command(container, signal: Optional[Any] = ..., **kwds): ... +def logs_command(container, **kwds): ... +def build_command(image, docker_build_path, **kwds): ... +def build_save_image_command(image, destination, **kwds): ... +def build_pull_command(tag, **kwds): ... +def build_docker_cache_command(image, **kwds): ... +def build_docker_images_command(truncate: bool = ..., **kwds): ... +def build_docker_load_command(**kwds): ... +def build_docker_run_command(container_command, image, interactive: bool = ..., terminal: bool = ..., tag: Optional[Any] = ..., volumes: Any = ..., volumes_from: Any = ..., memory: Any = ..., env_directives: Any = ..., working_directory: Any = ..., name: Optional[Any] = ..., net: Any = ..., run_extra_arguments: Any = ..., docker_cmd: Any = ..., sudo: Any = ..., sudo_cmd: Any = ..., auto_rm: Any = ..., set_user: Any = ..., host: Any = ...): ... +def command_list(command, command_args: Any = ..., **kwds): ... +def command_shell(command, command_args: Any = ..., **kwds): ... diff --git a/typeshed/2.7/galaxy/tools/deps/requirements.pyi b/typeshed/2.7/galaxy/tools/deps/requirements.pyi new file mode 100644 index 000000000..2764d6430 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/requirements.pyi @@ -0,0 +1,35 @@ +# Stubs for galaxy.tools.deps.requirements (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +DEFAULT_REQUIREMENT_TYPE = ... # type: str +DEFAULT_REQUIREMENT_VERSION = ... # type: Any + +class ToolRequirement: + name = ... # type: Any + type = ... # type: Any + version = ... # type: Any + def __init__(self, name: Optional[Any] = ..., type: Optional[Any] = ..., version: Optional[Any] = ...) -> None: ... + def to_dict(self): ... + @staticmethod + def from_dict(dict: Dict[Any, Any]): ... # type: ToolRequirement + +DEFAULT_CONTAINER_TYPE = ... # type: str +DEFAULT_CONTAINER_RESOLVE_DEPENDENCIES = ... # type: bool +DEFAULT_CONTAINER_SHELL = ... # type: str + +class ContainerDescription: + identifier = ... # type: Any + type = ... # type: Any + resolve_dependencies = ... # type: Any + shell = ... # type: Any + def __init__(self, identifier: Optional[Any] = ..., type: Any = ..., resolve_dependencies: Any = ..., shell: Any = ...) -> None: ... + def to_dict(self): ... + @staticmethod + def from_dict(dict): ... + +def parse_requirements_from_dict(root_dict): ... +def parse_requirements_from_xml(xml_root): ... +def container_from_element(container_elem): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi new file mode 100644 index 000000000..057e51c0f --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi @@ -0,0 +1,40 @@ +# Stubs for galaxy.tools.deps.resolvers (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from galaxy.util.dictifiable import Dictifiable +from ..requirements import ToolRequirement as ToolRequirement + +class DependencyResolver(Dictifiable): + dict_collection_visible_keys = ... # type: Any + disabled = ... # type: bool + resolves_simple_dependencies = ... # type: bool + __metaclass__ = ... # type: Any + def resolve(self, name, version, type, **kwds): ... + +class ListableDependencyResolver: + __metaclass__ = ... # type: Any + def list_dependencies(self): ... + +class InstallableDependencyResolver: + __metaclass__ = ... # type: Any + def install_dependency(self, name, version, type, **kwds): ... + +class Dependency(Dictifiable): + dict_collection_visible_keys = ... # type: Any + __metaclass__ = ... # type: Any + def shell_commands(self, requirement): ... + def exact(self): ... + @property + def resolver_msg(self): ... + +class NullDependency(Dependency): + dependency_type = ... # type: Any + exact = ... # type: bool + version = ... # type: Any + name = ... # type: Any + def __init__(self, version: Optional[Any] = ..., name: Optional[Any] = ...) -> None: ... + @property + def resolver_msg(self): ... + def shell_commands(self, requirement): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi new file mode 100644 index 000000000..8308d36f2 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi @@ -0,0 +1,40 @@ +# Stubs for galaxy.tools.deps.resolvers.conda (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..resolvers import Dependency, DependencyResolver, InstallableDependencyResolver, ListableDependencyResolver + +class CondaDependencyResolver(DependencyResolver, ListableDependencyResolver, InstallableDependencyResolver): + dict_collection_visible_keys = ... # type: Any + resolver_type = ... # type: str + versionless = ... # type: Any + dependency_manager = ... # type: Any + conda_prefix_parent = ... # type: Any + ensure_channels = ... # type: Any + auto_init = ... # type: Any + conda_context = ... # type: Any + disabled = ... # type: Any + auto_install = ... # type: Any + copy_dependencies = ... # type: Any + verbose_install_check = ... # type: Any + def __init__(self, dependency_manager, **kwds) -> None: ... + def resolve(self, name, version, type, **kwds): ... + def list_dependencies(self): ... + def install_dependency(self, name, version, type, **kwds): ... + @property + def prefix(self): ... + +class CondaDependency(Dependency): + dict_collection_visible_keys = ... # type: Any + dependency_type = ... # type: str + activate = ... # type: Any + environment_path = ... # type: Any + def __init__(self, activate, environment_path, exact, name: Optional[Any] = ..., version: Optional[Any] = ...) -> None: ... + @property + def exact(self): ... + @property + def name(self): ... + @property + def version(self): ... + def shell_commands(self, requirement): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi new file mode 100644 index 000000000..c83a8a7b1 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi @@ -0,0 +1,33 @@ +# Stubs for galaxy.tools.deps.resolvers.galaxy_packages (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from .resolver_mixins import UsesToolDependencyDirMixin +from ..resolvers import Dependency, DependencyResolver, ListableDependencyResolver + +class GalaxyPackageDependency(Dependency): + dict_collection_visible_keys = ... # type: Any + dependency_type = ... # type: str + script = ... # type: Any + path = ... # type: Any + version = ... # type: Any + name = ... # type: Any + def __init__(self, script, path, version, name, exact: bool = ...) -> None: ... + @property + def exact(self): ... + def shell_commands(self, requirement): ... + +class ToolShedDependency(GalaxyPackageDependency): + dependency_type = ... # type: str + +class BaseGalaxyPackageDependencyResolver(DependencyResolver, UsesToolDependencyDirMixin): + dict_collection_visible_keys = ... # type: Any + dependency_type = ... # type: Any + versionless = ... # type: Any + def __init__(self, dependency_manager, **kwds) -> None: ... + def resolve(self, name, version, type, **kwds): ... + +class GalaxyPackageDependencyResolver(BaseGalaxyPackageDependencyResolver, ListableDependencyResolver): + resolver_type = ... # type: str + def list_dependencies(self): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi new file mode 100644 index 000000000..fddfaec9a --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.tools.deps.resolvers.resolver_mixins (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from ..brew_exts import build_env_statements as build_env_statements, DEFAULT_HOMEBREW_ROOT as DEFAULT_HOMEBREW_ROOT, recipe_cellar_path as recipe_cellar_path +from ..resolvers import Dependency as Dependency, NullDependency as NullDependency + +class UsesHomebrewMixin: ... +class UsesToolDependencyDirMixin: ... +class UsesInstalledRepositoriesMixin: ... + +class HomebrewDependency(Dependency): + commands = ... # type: Any + def __init__(self, commands, exact: bool = ...) -> None: ... + @property + def exact(self): ... + def shell_commands(self, requirement): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi new file mode 100644 index 000000000..36681f606 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi @@ -0,0 +1,13 @@ +# Stubs for galaxy.tools.deps.resolvers.tool_shed_packages (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from .galaxy_packages import BaseGalaxyPackageDependencyResolver +from .resolver_mixins import UsesInstalledRepositoriesMixin + +class ToolShedPackageDependencyResolver(BaseGalaxyPackageDependencyResolver, UsesInstalledRepositoriesMixin): + resolver_type = ... # type: str + dependency_type = ... # type: Any + resolves_simple_dependencies = ... # type: bool + def __init__(self, dependency_manager, **kwds) -> None: ... diff --git a/typeshed/2.7/galaxy/util/__init__.pyi b/typeshed/2.7/galaxy/util/__init__.pyi new file mode 100644 index 000000000..d0c7e7a90 --- /dev/null +++ b/typeshed/2.7/galaxy/util/__init__.pyi @@ -0,0 +1,131 @@ +# Stubs for galaxy.util (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +import collections +from six.moves.urllib import parse as urlparse +from six.moves.urllib import request as urlrequest +from .inflection import English as English, Inflector as Inflector + +grp = ... # type: Any +docutils_core = ... # type: Any +docutils_html4css1 = ... # type: Any +inflector = ... # type: Any + +list_map = ... # type: Any +log = ... # type: Any +CHUNK_SIZE = ... # type: int +DATABASE_MAX_STRING_SIZE = ... # type: int +DATABASE_MAX_STRING_SIZE_PRETTY = ... # type: str +gzip_magic = ... # type: str +bz2_magic = ... # type: str +DEFAULT_ENCODING = ... # type: Any +NULL_CHAR = ... # type: str +BINARY_CHARS = ... # type: Any +FILENAME_VALID_CHARS = ... # type: str + +def remove_protocol_from_url(url): ... +def is_binary(value, binary_chars: Optional[Any] = ...): ... +def is_uuid(value): ... +def directory_hash_id(id): ... +def get_charset_from_http_headers(headers, default: Optional[Any] = ...): ... +def synchronized(func): ... +def file_iter(fname, sep: Optional[Any] = ...): ... +def file_reader(fp, chunk_size: Any = ...): ... +def unique_id(KEY_SIZE: int = ...): ... +def parse_xml(fname): ... +def parse_xml_string(xml_string): ... +def xml_to_string(elem, pretty: bool = ...): ... +def xml_element_compare(elem1, elem2): ... +def xml_element_list_compare(elem_list1, elem_list2): ... +def xml_element_to_dict(elem): ... +def pretty_print_xml(elem, level: int = ...): ... +def get_file_size(value, default: Optional[Any] = ...): ... +def shrink_stream_by_size(value, size, join_by: str = ..., left_larger: bool = ..., beginning_on_size_error: bool = ..., end_on_size_error: bool = ...): ... +def shrink_string_by_size(value, size, join_by: str = ..., left_larger: bool = ..., beginning_on_size_error: bool = ..., end_on_size_error: bool = ...): ... +def pretty_print_time_interval(time: bool = ..., precise: bool = ...): ... +def pretty_print_json(json_data, is_json_string: bool = ...): ... + +valid_chars = ... # type: Any +mapped_chars = ... # type: Any + +def restore_text(text, character_map: Any = ...): ... +def sanitize_text(text, valid_characters: Any = ..., character_map: Any = ..., invalid_character: str = ...): ... +def sanitize_lists_to_string(values, valid_characters: Any = ..., character_map: Any = ..., invalid_character: str = ...): ... +def sanitize_param(value, valid_characters: Any = ..., character_map: Any = ..., invalid_character: str = ...): ... + +valid_filename_chars = ... # type: Any +invalid_filenames = ... # type: Any + +def sanitize_for_filename(text, default: Optional[Any] = ...): ... +def mask_password_from_url(url): ... +def ready_name_for_url(raw_name): ... +def which(file): ... +def safe_makedirs(path): ... +def in_directory(file, directory, local_path_module: Any = ...): ... +def merge_sorted_iterables(operator, *iterables): ... + +class Params: + NEVER_SANITIZE = ... # type: Any + def __init__(self, params, sanitize: bool = ...) -> None: ... + def flatten(self): ... + def __getattr__(self, name): ... + def get(self, key, default): ... + def __len__(self): ... + def __iter__(self): ... + def update(self, values): ... + +def rst_to_html(s): ... +def xml_text(root, name: Optional[Any] = ...): ... + +truthy = ... # type: Any +falsy = ... # type: Any + +def asbool(obj): ... +def string_as_bool(string): ... +def string_as_bool_or_none(string): ... +def listify(item, do_strip: bool = ...): ... +def commaify(amount): ... +def roundify(amount, sfs: int = ...): ... +def unicodify(value, encoding: Any = ..., error: str = ..., default: Optional[Any] = ...): ... +def smart_str(s, encoding: Any = ..., strings_only: bool = ..., errors: str = ...): ... +def object_to_string(obj): ... +def string_to_object(s): ... + +class ParamsWithSpecs(collections.defaultdict): + specs = ... # type: Any + params = ... # type: Any + def __init__(self, specs: Optional[Any] = ..., params: Optional[Any] = ...) -> None: ... + def __missing__(self, name): ... + def __getattr__(self, name): ... + +def compare_urls(url1, url2, compare_scheme: bool = ..., compare_hostname: bool = ..., compare_path: bool = ...): ... +def read_dbnames(filename): ... +def read_build_sites(filename, check_builds: bool = ...): ... +def relativize_symlinks(path, start: Optional[Any] = ..., followlinks: bool = ...): ... +def stringify_dictionary_keys(in_dict): ... +def recursively_stringify_dictionary_keys(d): ... +def mkstemp_ln(src, prefix: str = ...): ... +def umask_fix_perms(path, umask, unmasked_perms, gid: Optional[Any] = ...): ... +def docstring_trim(docstring): ... +def nice_size(size): ... +def size_to_bytes(size): ... +def send_mail(frm, to, subject, body, config, html: Optional[Any] = ...): ... +def force_symlink(source, link_name): ... +def move_merge(source, target): ... +def safe_str_cmp(a, b): ... + +galaxy_root_path = ... # type: Any + +def galaxy_directory(): ... +def config_directories_from_setting(directories_setting, galaxy_root: Any = ...): ... +def parse_int(value, min_val: Optional[Any] = ..., max_val: Optional[Any] = ..., default: Optional[Any] = ..., allow_none: bool = ...): ... +def parse_non_hex_float(s): ... +def build_url(base_url, port: int = ..., scheme: str = ..., pathspec: Optional[Any] = ..., params: Optional[Any] = ..., doseq: bool = ...): ... +def url_get(base_url, password_mgr: Optional[Any] = ..., pathspec: Optional[Any] = ..., params: Optional[Any] = ...): ... +def safe_relpath(path): ... + +class ExecutionTimer: + begin = ... # type: Any + def __init__(self) -> None: ... diff --git a/typeshed/2.7/galaxy/util/dictifiable.pyi b/typeshed/2.7/galaxy/util/dictifiable.pyi new file mode 100644 index 000000000..eaec1d90d --- /dev/null +++ b/typeshed/2.7/galaxy/util/dictifiable.pyi @@ -0,0 +1,8 @@ +# Stubs for galaxy.util.dictifiable (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +class Dictifiable: + def to_dict(self, view: str = ..., value_mapper: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/util/inflection.pyi b/typeshed/2.7/galaxy/util/inflection.pyi new file mode 100644 index 000000000..065a7a347 --- /dev/null +++ b/typeshed/2.7/galaxy/util/inflection.pyi @@ -0,0 +1,46 @@ +# Stubs for galaxy.util.inflection (Python 3.5) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class Base: + def cond_plural(self, number_of_records, word): ... + def titleize(self, word, uppercase: str = ...): ... + def camelize(self, word): ... + def underscore(self, word): ... + def humanize(self, word, uppercase: str = ...): ... + def variablize(self, word): ... + def tableize(self, class_name): ... + def classify(self, table_name): ... + def ordinalize(self, number): ... + def unaccent(self, text): ... + def string_replace(self, word, find, replace): ... + def urlize(self, text): ... + def demodulize(self, module_name): ... + def modulize(self, module_description): ... + def foreignKey(self, class_name, separate_class_name_and_id_with_underscore: int = ...): ... + +class English(Base): + def pluralize(self, word): ... + def singularize(self, word): ... + +class Inflector: + Inflector = ... # type: Any + def __init__(self, Inflector: Any = ...) -> None: ... + def pluralize(self, word): ... + def singularize(self, word): ... + def cond_plural(self, number_of_records, word): ... + def titleize(self, word, uppercase: str = ...): ... + def camelize(self, word): ... + def underscore(self, word): ... + def humanize(self, word, uppercase: str = ...): ... + def variablize(self, word): ... + def tableize(self, class_name): ... + def classify(self, table_name): ... + def ordinalize(self, number): ... + def unaccent(self, text): ... + def urlize(self, text): ... + def demodulize(self, module_name): ... + def modulize(self, module_description): ... + def foreignKey(self, class_name, separate_class_name_and_id_with_underscore: int = ...): ... From ee833153f46be3bf0f3cd05edcb4409695bfb449 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 08:42:37 +0200 Subject: [PATCH 02/22] galaxy-lib as a setup.py plugin --- setup.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/setup.py b/setup.py index e5a260786..70c2106f2 100755 --- a/setup.py +++ b/setup.py @@ -61,6 +61,9 @@ 'typing >= 3.5.3', 'six >= 1.8.0', ], + extras_require = { + 'deps': ["galaxy-lib >= 17.09.1"] + }, setup_requires=[] + pytest_runner, test_suite='tests', tests_require=['pytest', 'mock >= 2.0.0',], From 1e6ea02dd8793b8b4564badcdb3bd0c31d544e35 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 11:26:53 +0200 Subject: [PATCH 03/22] Implement and add example for specifications in SoftwareRequirements. Run with the following command: ``` cwltool --beta-conda-dependencies tests/seqtk_seq_wrong_name.cwl tests/seqtk_seq_job.json ``` Required an update to galaxy-lib. --- cwltool/main.py | 3 +++ setup.py | 2 +- tests/seqtk_seq_wrong_name.cwl | 27 +++++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 1 deletion(-) create mode 100644 tests/seqtk_seq_wrong_name.cwl diff --git a/cwltool/main.py b/cwltool/main.py index ad3fded0c..52b7c2186 100755 --- a/cwltool/main.py +++ b/cwltool/main.py @@ -949,11 +949,14 @@ def _get_dependencies(builder): version = version[0] else: version = None + specs = [{"uri": s} for s in package.get("specs", [])] dependencies.append(ToolRequirement.from_dict(dict( name=package["package"].split("#")[-1], version=version, type="package", + specs=specs, ))) + return ToolRequirements.from_list(dependencies) diff --git a/setup.py b/setup.py index 70c2106f2..943a4fabd 100755 --- a/setup.py +++ b/setup.py @@ -62,7 +62,7 @@ 'six >= 1.8.0', ], extras_require = { - 'deps': ["galaxy-lib >= 17.09.1"] + 'deps': ["galaxy-lib >= 17.09.3"] }, setup_requires=[] + pytest_runner, test_suite='tests', diff --git a/tests/seqtk_seq_wrong_name.cwl b/tests/seqtk_seq_wrong_name.cwl new file mode 100644 index 000000000..4f2fc609f --- /dev/null +++ b/tests/seqtk_seq_wrong_name.cwl @@ -0,0 +1,27 @@ +cwlVersion: v1.0 +class: CommandLineTool +id: "seqtk_seq" +doc: "Convert to FASTA (seqtk)" +inputs: + - id: input1 + type: File + inputBinding: + position: 1 + prefix: "-a" +outputs: + - id: output1 + type: File + outputBinding: + glob: out +baseCommand: ["blast", "--help"] +arguments: [] +stdout: out +hints: + SoftwareRequirement: + packages: + - package: seqtk_seq + version: + - '1.2' + specs: + - https://anaconda.org/bioconda/seqtk + - https://packages.debian.org/sid/seqtk From ef3e6f2b7bd3790737b7dfeed6ec9f08f69cbf19 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 12:31:48 +0200 Subject: [PATCH 04/22] Fix --no-container, add example combining Conda + explicit docker options. Sorry about that @jxtx. This: ``` cwltool --no-container --beta-conda-dependencies tests/seqtk_seq_with_docker.cwl tests/seqtk_seq_job.json ``` is working for me now. --- cwltool/draft2tool.py | 2 +- tests/seqtk_seq_with_docker.cwl | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 tests/seqtk_seq_with_docker.cwl diff --git a/cwltool/draft2tool.py b/cwltool/draft2tool.py index 1a1e4d160..a8df7e135 100644 --- a/cwltool/draft2tool.py +++ b/cwltool/draft2tool.py @@ -293,7 +293,7 @@ def rm_pending_output_callback(output_callbacks, jobcachepending, reffiles = copy.deepcopy(builder.files) - j = self.makeJobRunner(kwargs.get("find_default_container")) + j = self.makeJobRunner(kwargs.get("use_container")) j.builder = builder j.joborder = builder.job j.stdin = None diff --git a/tests/seqtk_seq_with_docker.cwl b/tests/seqtk_seq_with_docker.cwl new file mode 100644 index 000000000..8c7834755 --- /dev/null +++ b/tests/seqtk_seq_with_docker.cwl @@ -0,0 +1,26 @@ +cwlVersion: v1.0 +class: CommandLineTool +id: "seqtk_seq" +doc: "Convert to FASTA (seqtk)" +inputs: + - id: input1 + type: File + inputBinding: + position: 1 + prefix: "-a" +outputs: + - id: output1 + type: File + outputBinding: + glob: out +baseCommand: ["seqtk", "seq"] +arguments: [] +stdout: out +hints: + SoftwareRequirement: + packages: + - package: seqtk + version: + - '1.2' + DockerRequirement: + dockerPull: quay.io/biocontainers/seqtk:1.2--0 From 432ad4cba78064648909eab9e1ac16abeb899fe2 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 12:34:40 +0200 Subject: [PATCH 05/22] Fix whitespace deleted in previous rebase. --- cwltool/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cwltool/main.py b/cwltool/main.py index 52b7c2186..6a8ece93b 100755 --- a/cwltool/main.py +++ b/cwltool/main.py @@ -741,7 +741,7 @@ def main(argsl=None, # type: List[str] make_tool_kwds["find_default_container"] = functools.partial(find_default_container, args) tool = make_tool(document_loader, avsc_names, metadata, uri, - makeTool, make_tool_kwds) + makeTool, make_tool_kwds) if args.validate: return 0 From e7059a28893e2674852ed395b445293153856770 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 12:38:21 +0200 Subject: [PATCH 06/22] Fixup seqtk seq example w/wrong name. --- tests/seqtk_seq_wrong_name.cwl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/seqtk_seq_wrong_name.cwl b/tests/seqtk_seq_wrong_name.cwl index 4f2fc609f..5e4665b1f 100644 --- a/tests/seqtk_seq_wrong_name.cwl +++ b/tests/seqtk_seq_wrong_name.cwl @@ -13,7 +13,7 @@ outputs: type: File outputBinding: glob: out -baseCommand: ["blast", "--help"] +baseCommand: ["seqtk", "seq"] arguments: [] stdout: out hints: From 913f61722935345d7c41fc32bcbc17ae66c12cf9 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 14:53:05 +0200 Subject: [PATCH 07/22] Example using environment-like modules. ``` cwltool --beta-dependency-resolvers-configuration tests/test_deps_env_resolvers_conf.xml tests/random_lines.cwl tests/random_lines_job.json ``` --- tests/random_lines.cwl | 29 +++++++ tests/random_lines_job.json | 8 ++ tests/test_deps_env/random-lines/1.0/env.sh | 8 ++ .../random-lines/1.0/scripts/random-lines | 78 +++++++++++++++++++ tests/test_deps_env_resolvers_conf.xml | 5 ++ 5 files changed, 128 insertions(+) create mode 100644 tests/random_lines.cwl create mode 100644 tests/random_lines_job.json create mode 100644 tests/test_deps_env/random-lines/1.0/env.sh create mode 100755 tests/test_deps_env/random-lines/1.0/scripts/random-lines create mode 100644 tests/test_deps_env_resolvers_conf.xml diff --git a/tests/random_lines.cwl b/tests/random_lines.cwl new file mode 100644 index 000000000..b352474a3 --- /dev/null +++ b/tests/random_lines.cwl @@ -0,0 +1,29 @@ +cwlVersion: v1.0 +class: CommandLineTool +id: "random_lines" +doc: "Select random lines from a file" +inputs: + - id: seed + type: int + inputBinding: + position: 1 + prefix: -s + - id: input1 + type: File + inputBinding: + position: 2 + - id: num_lines + type: int + inputBinding: + position: 3 +outputs: + output1: + type: stdout +baseCommand: ["random-lines"] +arguments: [] +hints: + SoftwareRequirement: + packages: + - package: 'random-lines' + version: + - '1.0' diff --git a/tests/random_lines_job.json b/tests/random_lines_job.json new file mode 100644 index 000000000..e1859c0e3 --- /dev/null +++ b/tests/random_lines_job.json @@ -0,0 +1,8 @@ +{ + "input1": { + "class": "File", + "location": "2.fastq" + }, + "seed": 5, + "num_lines": 2 +} diff --git a/tests/test_deps_env/random-lines/1.0/env.sh b/tests/test_deps_env/random-lines/1.0/env.sh new file mode 100644 index 000000000..453f53494 --- /dev/null +++ b/tests/test_deps_env/random-lines/1.0/env.sh @@ -0,0 +1,8 @@ + +#PACKAGE_DIRECTORY="/path/to/cwlroot/tests/test_deps_env/random-lines/1.0/" + +# This shouldn't need to use bash-isms - but we don't know the full path to this file, +# so for testing it is setup this way. For actual deployments just using full paths +# directly would be preferable. +PACKAGE_DIRECTORY="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +export PATH=$PATH:$PACKAGE_DIRECTORY/scripts diff --git a/tests/test_deps_env/random-lines/1.0/scripts/random-lines b/tests/test_deps_env/random-lines/1.0/scripts/random-lines new file mode 100755 index 000000000..bfde39bff --- /dev/null +++ b/tests/test_deps_env/random-lines/1.0/scripts/random-lines @@ -0,0 +1,78 @@ +#!/usr/bin/env python +# Dan Blankenberg +# Selects N random lines from a file and outputs to another file, maintaining original line order +# allows specifying a seed +# does two passes to determine line offsets/count, and then to output contents +from __future__ import print_function + +import optparse +import random +import sys + + +def get_random_by_subtraction( line_offsets, num_lines ): + while len( line_offsets ) > num_lines: + del line_offsets[ random.randint( 0, len( line_offsets ) - 1 ) ] + return line_offsets + + +def get_random_by_sample( line_offsets, num_lines ): + line_offsets = random.sample( line_offsets, num_lines ) + line_offsets.sort() + return line_offsets + + +def get_random( line_offsets, num_lines ): + if num_lines > ( len( line_offsets ) / 2 ): + return get_random_by_subtraction( line_offsets, num_lines ) + else: + return get_random_by_sample( line_offsets, num_lines ) + + +def __main__(): + open("/Users/john/moo", "w").write("cow") + parser = optparse.OptionParser() + parser.add_option( '-s', '--seed', dest='seed', action='store', type="string", default=None, help='Set the random seed.' ) + (options, args) = parser.parse_args() + + input = open( args[0], 'rb' ) + output = sys.stdout + num_lines = int( args[1] ) + assert num_lines > 0, "You must select at least one line." + + if options.seed is not None: + random.seed( options.seed ) + + # get line offsets + line_offsets = [] + teller = input.tell + readliner = input.readline + appender = line_offsets.append + while True: + offset = teller() + if readliner(): + appender( offset ) + else: + break + + total_lines = len( line_offsets ) + assert num_lines <= total_lines, "Error: asked to select more lines (%i) than there were in the file (%i)." % ( num_lines, total_lines ) + + # get random line offsets + line_offsets = get_random( line_offsets, num_lines ) + + # write out random lines + seeker = input.seek + writer = output.write + for line_offset in line_offsets: + seeker( line_offset ) + writer( readliner() ) + input.close() + output.close() + #print("Kept %i of %i total lines." % ( num_lines, total_lines )) + #if options.seed is not None: + # print('Used random seed of "%s".' % options.seed) + + +if __name__ == "__main__": + __main__() diff --git a/tests/test_deps_env_resolvers_conf.xml b/tests/test_deps_env_resolvers_conf.xml new file mode 100644 index 000000000..6d9349362 --- /dev/null +++ b/tests/test_deps_env_resolvers_conf.xml @@ -0,0 +1,5 @@ + + + + From 6b929f15a4b7dfb69d9690a7b067b1351d9296bd Mon Sep 17 00:00:00 2001 From: James Taylor Date: Tue, 27 Jun 2017 08:25:37 -0400 Subject: [PATCH 08/22] Fix Python 3 incompatibility in PYTHON_RUN_SCRIPT. `file` is no longer a built-in with the new `io` framework. Was causing jobs to fail depending on the python in the conda env. --- cwltool/job.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cwltool/job.py b/cwltool/job.py index 647803d2b..92c62d774 100644 --- a/cwltool/job.py +++ b/cwltool/job.py @@ -69,7 +69,7 @@ if sp.stdin: sp.stdin.close() rcode = sp.wait() - if isinstance(stdin, file): + if stdin is not subprocess.PIPE: stdin.close() if stdout is not sys.stderr: stdout.close() From 79bae7d16372c04637a67e0b5b873a0b2c035e3a Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 15:05:26 +0200 Subject: [PATCH 09/22] Fix stderr/stdout handling in external bash script processing of jobs. --- cwltool/job.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cwltool/job.py b/cwltool/job.py index 92c62d774..c5c847a5d 100644 --- a/cwltool/job.py +++ b/cwltool/job.py @@ -487,8 +487,8 @@ def _job_popen( ["bash", job_script.encode("utf-8")], shell=False, cwd=job_dir, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, + stdout=sys.stderr, # The nested script will output the paths to the correct files if they need + stderr=sys.stderr, # to be captured. Else just write everything to stderr (same as above). stdin=subprocess.PIPE, ) if sp.stdin: From b19f60d9b430852b4cb873ac020b72e67f78e2a9 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 15:28:14 +0200 Subject: [PATCH 10/22] Sorry for the XML - we didn't need to do that. --- tests/test_deps_env_resolvers_conf.xml | 5 ----- tests/test_deps_env_resolvers_conf.yml | 3 +++ 2 files changed, 3 insertions(+), 5 deletions(-) delete mode 100644 tests/test_deps_env_resolvers_conf.xml create mode 100644 tests/test_deps_env_resolvers_conf.yml diff --git a/tests/test_deps_env_resolvers_conf.xml b/tests/test_deps_env_resolvers_conf.xml deleted file mode 100644 index 6d9349362..000000000 --- a/tests/test_deps_env_resolvers_conf.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - diff --git a/tests/test_deps_env_resolvers_conf.yml b/tests/test_deps_env_resolvers_conf.yml new file mode 100644 index 000000000..e1e190c23 --- /dev/null +++ b/tests/test_deps_env_resolvers_conf.yml @@ -0,0 +1,3 @@ +- type: galaxy_packages + base_path: ./tests/test_deps_env + From 461b10fc0679316bbaa6216142c1409841d3ba28 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 16:21:02 +0200 Subject: [PATCH 11/22] Demonstrate mapping of SoftwareRequirements to local resources. --- tests/random_lines_mapping.cwl | 29 +++++++++++++++++++ .../test_deps_env_resolvers_conf_rewrite.yml | 3 ++ tests/test_deps_mapping.yml | 6 ++++ 3 files changed, 38 insertions(+) create mode 100644 tests/random_lines_mapping.cwl create mode 100644 tests/test_deps_env_resolvers_conf_rewrite.yml create mode 100644 tests/test_deps_mapping.yml diff --git a/tests/random_lines_mapping.cwl b/tests/random_lines_mapping.cwl new file mode 100644 index 000000000..b526b3c70 --- /dev/null +++ b/tests/random_lines_mapping.cwl @@ -0,0 +1,29 @@ +cwlVersion: v1.0 +class: CommandLineTool +id: "random_lines" +doc: "Select random lines from a file" +inputs: + - id: seed + type: int + inputBinding: + position: 1 + prefix: -s + - id: input1 + type: File + inputBinding: + position: 2 + - id: num_lines + type: int + inputBinding: + position: 3 +outputs: + output1: + type: stdout +baseCommand: ["random-lines"] +arguments: [] +hints: + SoftwareRequirement: + packages: + - package: randomLines + version: + - '1.0.0-rc1' diff --git a/tests/test_deps_env_resolvers_conf_rewrite.yml b/tests/test_deps_env_resolvers_conf_rewrite.yml new file mode 100644 index 000000000..a80b6d4bf --- /dev/null +++ b/tests/test_deps_env_resolvers_conf_rewrite.yml @@ -0,0 +1,3 @@ +- type: galaxy_packages + base_path: ./tests/test_deps_env + mapping_files: ./tests/test_deps_mapping.yml diff --git a/tests/test_deps_mapping.yml b/tests/test_deps_mapping.yml new file mode 100644 index 000000000..e09af5ad3 --- /dev/null +++ b/tests/test_deps_mapping.yml @@ -0,0 +1,6 @@ +- from: + name: randomLines + version: 1.0.0-rc1 + to: + name: random-lines + version: '1.0' From 368f8888f203ee58da636c4024e8b49211eb91a3 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Tue, 27 Jun 2017 17:27:09 +0200 Subject: [PATCH 12/22] Updated typeshed defs for Galaxy-lib? --- typeshed/2.7/galaxy/__init__.pyi | 2 +- typeshed/2.7/galaxy/exceptions/__init__.pyi | 143 ++++++++++++ .../2.7/galaxy/exceptions/error_codes.pyi | 17 ++ typeshed/2.7/galaxy/jobs/__init__.pyi | 4 + typeshed/2.7/galaxy/jobs/metrics/__init__.pyi | 38 ++++ .../galaxy/jobs/metrics/collectl/__init__.pyi | 4 + .../2.7/galaxy/jobs/metrics/collectl/cli.pyi | 12 + .../jobs/metrics/collectl/processes.pyi | 24 ++ .../galaxy/jobs/metrics/collectl/stats.pyi | 15 ++ .../jobs/metrics/collectl/subsystems.pyi | 35 +++ .../2.7/galaxy/jobs/metrics/formatting.pyi | 8 + .../jobs/metrics/instrumenters/__init__.pyi | 16 ++ .../jobs/metrics/instrumenters/collectl.pyi | 22 ++ .../jobs/metrics/instrumenters/core.pyi | 18 ++ .../jobs/metrics/instrumenters/cpuinfo.pyi | 18 ++ .../galaxy/jobs/metrics/instrumenters/env.pyi | 18 ++ .../jobs/metrics/instrumenters/meminfo.pyi | 18 ++ .../jobs/metrics/instrumenters/uname.pyi | 18 ++ typeshed/2.7/galaxy/objectstore/__init__.pyi | 80 +++++++ .../2.7/galaxy/objectstore/azure_blob.pyi | 29 +++ typeshed/2.7/galaxy/objectstore/pulsar.pyi | 24 ++ typeshed/2.7/galaxy/objectstore/rods.pyi | 32 +++ typeshed/2.7/galaxy/objectstore/s3.pyi | 34 +++ .../objectstore/s3_multipart_upload.pyi | 13 ++ typeshed/2.7/galaxy/tools/__init__.pyi | 2 +- typeshed/2.7/galaxy/tools/cwl/__init__.pyi | 8 + .../2.7/galaxy/tools/cwl/cwltool_deps.pyi | 22 ++ typeshed/2.7/galaxy/tools/cwl/parser.pyi | 94 ++++++++ .../2.7/galaxy/tools/cwl/representation.pyi | 12 + .../2.7/galaxy/tools/cwl/runtime_actions.pyi | 7 + typeshed/2.7/galaxy/tools/cwl/schema.pyi | 22 ++ typeshed/2.7/galaxy/tools/deps/__init__.pyi | 23 +- typeshed/2.7/galaxy/tools/deps/brew_exts.pyi | 2 +- typeshed/2.7/galaxy/tools/deps/brew_util.pyi | 18 ++ typeshed/2.7/galaxy/tools/deps/commands.pyi | 7 +- .../2.7/galaxy/tools/deps/conda_compat.pyi | 21 ++ typeshed/2.7/galaxy/tools/deps/conda_util.pyi | 20 +- .../deps/container_resolvers/__init__.pyi | 3 +- .../deps/container_resolvers/explicit.pyi | 7 +- .../tools/deps/container_resolvers/mulled.pyi | 41 +++- typeshed/2.7/galaxy/tools/deps/containers.pyi | 25 +- .../2.7/galaxy/tools/deps/dependencies.pyi | 13 ++ .../2.7/galaxy/tools/deps/docker_util.pyi | 4 +- .../2.7/galaxy/tools/deps/dockerfiles.pyi | 15 ++ .../2.7/galaxy/tools/deps/installable.pyi | 15 ++ .../2.7/galaxy/tools/deps/mulled/__init__.pyi | 4 + .../2.7/galaxy/tools/deps/mulled/_cli.pyi | 5 + .../galaxy/tools/deps/mulled/mulled_build.pyi | 24 ++ .../deps/mulled/mulled_build_channel.pyi | 7 + .../tools/deps/mulled/mulled_build_files.pyi | 10 + .../tools/deps/mulled/mulled_build_tool.pyi | 8 + .../tools/deps/mulled/mulled_search.pyi | 23 ++ .../2.7/galaxy/tools/deps/mulled/util.pyi | 23 ++ .../2.7/galaxy/tools/deps/requirements.pyi | 46 +++- .../galaxy/tools/deps/resolvers/__init__.pyi | 34 ++- .../resolvers/brewed_tool_shed_packages.pyi | 33 +++ .../2.7/galaxy/tools/deps/resolvers/conda.pyi | 44 +++- .../tools/deps/resolvers/galaxy_packages.pyi | 10 +- .../galaxy/tools/deps/resolvers/homebrew.pyi | 14 ++ .../galaxy/tools/deps/resolvers/modules.pyi | 42 ++++ .../tools/deps/resolvers/resolver_mixins.pyi | 2 +- .../deps/resolvers/tool_shed_packages.pyi | 2 +- .../resolvers/unlinked_tool_shed_packages.pyi | 25 ++ .../galaxy/tools/deps/singularity_util.pyi | 7 + typeshed/2.7/galaxy/tools/deps/views.pyi | 32 +++ typeshed/2.7/galaxy/tools/fetcher.pyi | 10 + typeshed/2.7/galaxy/tools/lint.pyi | 33 +++ typeshed/2.7/galaxy/tools/lint_util.pyi | 5 + .../2.7/galaxy/tools/linters/__init__.pyi | 4 + .../2.7/galaxy/tools/linters/citations.pyi | 5 + typeshed/2.7/galaxy/tools/linters/command.pyi | 6 + typeshed/2.7/galaxy/tools/linters/cwl.pyi | 12 + typeshed/2.7/galaxy/tools/linters/general.pyi | 19 ++ typeshed/2.7/galaxy/tools/linters/help.pyi | 6 + typeshed/2.7/galaxy/tools/linters/inputs.pyi | 8 + typeshed/2.7/galaxy/tools/linters/outputs.pyi | 5 + typeshed/2.7/galaxy/tools/linters/stdio.pyi | 7 + typeshed/2.7/galaxy/tools/linters/tests.pyi | 7 + .../2.7/galaxy/tools/linters/xml_order.pyi | 10 + typeshed/2.7/galaxy/tools/loader.pyi | 8 + .../2.7/galaxy/tools/loader_directory.pyi | 15 ++ .../2.7/galaxy/tools/locations/__init__.pyi | 7 + .../2.7/galaxy/tools/locations/dockstore.pyi | 19 ++ typeshed/2.7/galaxy/tools/locations/file.pyi | 9 + typeshed/2.7/galaxy/tools/locations/http.pyi | 13 ++ typeshed/2.7/galaxy/tools/parser/__init__.pyi | 7 + typeshed/2.7/galaxy/tools/parser/cwl.pyi | 41 ++++ typeshed/2.7/galaxy/tools/parser/factory.pyi | 9 + .../2.7/galaxy/tools/parser/interface.pyi | 96 ++++++++ .../galaxy/tools/parser/output_actions.pyi | 215 ++++++++++++++++++ .../tools/parser/output_collection_def.pyi | 29 +++ .../galaxy/tools/parser/output_objects.pyi | 68 ++++++ typeshed/2.7/galaxy/tools/parser/util.pyi | 9 + typeshed/2.7/galaxy/tools/parser/xml.pyi | 89 ++++++++ typeshed/2.7/galaxy/tools/parser/yaml.pyi | 56 +++++ typeshed/2.7/galaxy/tools/verify/__init__.pyi | 18 ++ .../galaxy/tools/verify/asserts/__init__.pyi | 14 ++ .../galaxy/tools/verify/asserts/tabular.pyi | 6 + .../2.7/galaxy/tools/verify/asserts/text.pyi | 9 + .../2.7/galaxy/tools/verify/asserts/xml.pyi | 15 ++ .../2.7/galaxy/tools/verify/test_data.pyi | 31 +++ typeshed/2.7/galaxy/util/__init__.pyi | 10 +- typeshed/2.7/galaxy/util/aliaspickler.pyi | 20 ++ typeshed/2.7/galaxy/util/bunch.pyi | 17 ++ typeshed/2.7/galaxy/util/checkers.pyi | 14 ++ .../2.7/galaxy/util/compression_utils.pyi | 7 + typeshed/2.7/galaxy/util/dictifiable.pyi | 2 +- typeshed/2.7/galaxy/util/expressions.pyi | 18 ++ typeshed/2.7/galaxy/util/filelock.pyi | 21 ++ typeshed/2.7/galaxy/util/hash_util.pyi | 14 ++ typeshed/2.7/galaxy/util/heartbeat.pyi | 30 +++ typeshed/2.7/galaxy/util/image_util.pyi | 13 ++ typeshed/2.7/galaxy/util/inflection.pyi | 2 +- typeshed/2.7/galaxy/util/json.pyi | 12 + typeshed/2.7/galaxy/util/lazy_process.pyi | 22 ++ typeshed/2.7/galaxy/util/object_wrapper.pyi | 95 ++++++++ typeshed/2.7/galaxy/util/odict.pyi | 27 +++ typeshed/2.7/galaxy/util/oset.pyi | 19 ++ typeshed/2.7/galaxy/util/plugin_config.pyi | 11 + typeshed/2.7/galaxy/util/properties.pyi | 19 ++ typeshed/2.7/galaxy/util/simplegraph.pyi | 26 +++ typeshed/2.7/galaxy/util/sleeper.pyi | 11 + typeshed/2.7/galaxy/util/sockets.pyi | 7 + typeshed/2.7/galaxy/util/specs.pyi | 9 + typeshed/2.7/galaxy/util/sqlite.pyi | 6 + typeshed/2.7/galaxy/util/submodules.pyi | 9 + typeshed/2.7/galaxy/util/topsort.pyi | 20 ++ typeshed/2.7/galaxy/util/xml_macros.pyi | 18 ++ 128 files changed, 2746 insertions(+), 71 deletions(-) create mode 100644 typeshed/2.7/galaxy/exceptions/__init__.pyi create mode 100644 typeshed/2.7/galaxy/exceptions/error_codes.pyi create mode 100644 typeshed/2.7/galaxy/jobs/__init__.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/__init__.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/collectl/__init__.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/collectl/cli.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/collectl/processes.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/collectl/stats.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/collectl/subsystems.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/formatting.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/instrumenters/__init__.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/instrumenters/collectl.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/instrumenters/core.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/instrumenters/cpuinfo.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/instrumenters/env.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/instrumenters/meminfo.pyi create mode 100644 typeshed/2.7/galaxy/jobs/metrics/instrumenters/uname.pyi create mode 100644 typeshed/2.7/galaxy/objectstore/__init__.pyi create mode 100644 typeshed/2.7/galaxy/objectstore/azure_blob.pyi create mode 100644 typeshed/2.7/galaxy/objectstore/pulsar.pyi create mode 100644 typeshed/2.7/galaxy/objectstore/rods.pyi create mode 100644 typeshed/2.7/galaxy/objectstore/s3.pyi create mode 100644 typeshed/2.7/galaxy/objectstore/s3_multipart_upload.pyi create mode 100644 typeshed/2.7/galaxy/tools/cwl/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/cwl/cwltool_deps.pyi create mode 100644 typeshed/2.7/galaxy/tools/cwl/parser.pyi create mode 100644 typeshed/2.7/galaxy/tools/cwl/representation.pyi create mode 100644 typeshed/2.7/galaxy/tools/cwl/runtime_actions.pyi create mode 100644 typeshed/2.7/galaxy/tools/cwl/schema.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/brew_util.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/conda_compat.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/dependencies.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/dockerfiles.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/installable.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/_cli.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/mulled_build.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_channel.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_files.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_tool.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/mulled_search.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/mulled/util.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/brewed_tool_shed_packages.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/homebrew.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/modules.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/resolvers/unlinked_tool_shed_packages.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/singularity_util.pyi create mode 100644 typeshed/2.7/galaxy/tools/deps/views.pyi create mode 100644 typeshed/2.7/galaxy/tools/fetcher.pyi create mode 100644 typeshed/2.7/galaxy/tools/lint.pyi create mode 100644 typeshed/2.7/galaxy/tools/lint_util.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/citations.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/command.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/cwl.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/general.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/help.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/inputs.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/outputs.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/stdio.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/tests.pyi create mode 100644 typeshed/2.7/galaxy/tools/linters/xml_order.pyi create mode 100644 typeshed/2.7/galaxy/tools/loader.pyi create mode 100644 typeshed/2.7/galaxy/tools/loader_directory.pyi create mode 100644 typeshed/2.7/galaxy/tools/locations/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/locations/dockstore.pyi create mode 100644 typeshed/2.7/galaxy/tools/locations/file.pyi create mode 100644 typeshed/2.7/galaxy/tools/locations/http.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/cwl.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/factory.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/interface.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/output_actions.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/output_collection_def.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/output_objects.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/util.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/xml.pyi create mode 100644 typeshed/2.7/galaxy/tools/parser/yaml.pyi create mode 100644 typeshed/2.7/galaxy/tools/verify/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/verify/asserts/__init__.pyi create mode 100644 typeshed/2.7/galaxy/tools/verify/asserts/tabular.pyi create mode 100644 typeshed/2.7/galaxy/tools/verify/asserts/text.pyi create mode 100644 typeshed/2.7/galaxy/tools/verify/asserts/xml.pyi create mode 100644 typeshed/2.7/galaxy/tools/verify/test_data.pyi create mode 100644 typeshed/2.7/galaxy/util/aliaspickler.pyi create mode 100644 typeshed/2.7/galaxy/util/bunch.pyi create mode 100644 typeshed/2.7/galaxy/util/checkers.pyi create mode 100644 typeshed/2.7/galaxy/util/compression_utils.pyi create mode 100644 typeshed/2.7/galaxy/util/expressions.pyi create mode 100644 typeshed/2.7/galaxy/util/filelock.pyi create mode 100644 typeshed/2.7/galaxy/util/hash_util.pyi create mode 100644 typeshed/2.7/galaxy/util/heartbeat.pyi create mode 100644 typeshed/2.7/galaxy/util/image_util.pyi create mode 100644 typeshed/2.7/galaxy/util/json.pyi create mode 100644 typeshed/2.7/galaxy/util/lazy_process.pyi create mode 100644 typeshed/2.7/galaxy/util/object_wrapper.pyi create mode 100644 typeshed/2.7/galaxy/util/odict.pyi create mode 100644 typeshed/2.7/galaxy/util/oset.pyi create mode 100644 typeshed/2.7/galaxy/util/plugin_config.pyi create mode 100644 typeshed/2.7/galaxy/util/properties.pyi create mode 100644 typeshed/2.7/galaxy/util/simplegraph.pyi create mode 100644 typeshed/2.7/galaxy/util/sleeper.pyi create mode 100644 typeshed/2.7/galaxy/util/sockets.pyi create mode 100644 typeshed/2.7/galaxy/util/specs.pyi create mode 100644 typeshed/2.7/galaxy/util/sqlite.pyi create mode 100644 typeshed/2.7/galaxy/util/submodules.pyi create mode 100644 typeshed/2.7/galaxy/util/topsort.pyi create mode 100644 typeshed/2.7/galaxy/util/xml_macros.pyi diff --git a/typeshed/2.7/galaxy/__init__.pyi b/typeshed/2.7/galaxy/__init__.pyi index 78ce81a70..aa2f6e6ef 100644 --- a/typeshed/2.7/galaxy/__init__.pyi +++ b/typeshed/2.7/galaxy/__init__.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy (Python 3.5) +# Stubs for galaxy (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. diff --git a/typeshed/2.7/galaxy/exceptions/__init__.pyi b/typeshed/2.7/galaxy/exceptions/__init__.pyi new file mode 100644 index 000000000..e59ac087d --- /dev/null +++ b/typeshed/2.7/galaxy/exceptions/__init__.pyi @@ -0,0 +1,143 @@ +# Stubs for galaxy.exceptions (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..exceptions import error_codes as error_codes + +class MessageException(Exception): + status_code = ... # type: int + err_code = ... # type: Any + err_msg = ... # type: Any + type = ... # type: Any + extra_error_info = ... # type: Any + def __init__(self, err_msg: Optional[Any] = ..., type: str = ..., **extra_error_info) -> None: ... + +class ItemDeletionException(MessageException): ... +class ObjectInvalid(Exception): ... + +class ActionInputError(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + def __init__(self, err_msg, type: str = ...) -> None: ... + +class DuplicatedSlugException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class DuplicatedIdentifierException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ObjectAttributeInvalidException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ObjectAttributeMissingException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class MalformedId(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class MalformedContents(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class UnknownContentsType(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class RequestParameterMissingException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ToolMetaParameterException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ToolMissingException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class RequestParameterInvalidException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class AuthenticationFailed(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class AuthenticationRequired(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ItemAccessibilityException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ItemOwnershipException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ConfigDoesNotAllowException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class InsufficientPermissionsException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class AdminRequiredException(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ObjectNotFound(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class DeprecatedMethod(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class Conflict(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ConfigurationError(Exception): + status_code = ... # type: int + err_code = ... # type: Any + +class InconsistentDatabase(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class InternalServerError(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class NotImplemented(MessageException): + status_code = ... # type: int + err_code = ... # type: Any + +class ContainerCLIError(Exception): + stdout = ... # type: Any + stderr = ... # type: Any + returncode = ... # type: Any + command = ... # type: Any + subprocess_command = ... # type: Any + def __init__(self, msg: Optional[Any] = ..., stdout: Optional[Any] = ..., stderr: Optional[Any] = ..., returncode: Optional[Any] = ..., command: Optional[Any] = ..., subprocess_command: Optional[Any] = ..., **kwargs) -> None: ... + +class ContainerNotFound(Exception): + container_id = ... # type: Any + def __init__(self, msg: Optional[Any] = ..., container_id: Optional[Any] = ..., **kwargs) -> None: ... + +class ContainerImageNotFound(Exception): + image = ... # type: Any + def __init__(self, msg: Optional[Any] = ..., image: Optional[Any] = ..., **kwargs) -> None: ... + +class ContainerRunError(Exception): + image = ... # type: Any + command = ... # type: Any + def __init__(self, msg: Optional[Any] = ..., image: Optional[Any] = ..., command: Optional[Any] = ..., **kwargs) -> None: ... diff --git a/typeshed/2.7/galaxy/exceptions/error_codes.pyi b/typeshed/2.7/galaxy/exceptions/error_codes.pyi new file mode 100644 index 000000000..b25ca3e99 --- /dev/null +++ b/typeshed/2.7/galaxy/exceptions/error_codes.pyi @@ -0,0 +1,17 @@ +# Stubs for galaxy.exceptions.error_codes (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +UNKNOWN_ERROR_MESSAGE = ... # type: str + +class ErrorCode: + code = ... # type: Any + default_error_message = ... # type: Any + def __init__(self, code, default_error_message) -> None: ... + def __int__(self): ... + +error_codes_json = ... # type: Any +name = ... # type: Any +error_code_obj = ... # type: Any diff --git a/typeshed/2.7/galaxy/jobs/__init__.pyi b/typeshed/2.7/galaxy/jobs/__init__.pyi new file mode 100644 index 000000000..c65cfb4ee --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/__init__.pyi @@ -0,0 +1,4 @@ +# Stubs for galaxy.jobs (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + diff --git a/typeshed/2.7/galaxy/jobs/metrics/__init__.pyi b/typeshed/2.7/galaxy/jobs/metrics/__init__.pyi new file mode 100644 index 000000000..758ddc677 --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/__init__.pyi @@ -0,0 +1,38 @@ +# Stubs for galaxy.jobs.metrics (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..metrics import formatting as formatting + +log = ... # type: Any +DEFAULT_FORMATTER = ... # type: Any + +class JobMetrics: + plugin_classes = ... # type: Any + default_job_instrumenter = ... # type: Any + job_instrumenters = ... # type: Any + def __init__(self, conf_file: Optional[Any] = ..., **kwargs) -> None: ... + def format(self, plugin, key, value): ... + def set_destination_conf_file(self, destination_id, conf_file): ... + def set_destination_conf_element(self, destination_id, element): ... + def set_destination_instrumenter(self, destination_id, job_instrumenter: Optional[Any] = ...): ... + def collect_properties(self, destination_id, job_id, job_directory): ... + +class NullJobInstrumenter: + def pre_execute_commands(self, job_directory): ... + def post_execute_commands(self, job_directory): ... + def collect_properties(self, job_id, job_directory): ... + +NULL_JOB_INSTRUMENTER = ... # type: Any + +class JobInstrumenter: + extra_kwargs = ... # type: Any + plugin_classes = ... # type: Any + plugins = ... # type: Any + def __init__(self, plugin_classes, plugins_source, **kwargs) -> None: ... + def pre_execute_commands(self, job_directory): ... + def post_execute_commands(self, job_directory): ... + def collect_properties(self, job_id, job_directory): ... + @staticmethod + def from_file(plugin_classes, conf_file, **kwargs): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/collectl/__init__.pyi b/typeshed/2.7/galaxy/jobs/metrics/collectl/__init__.pyi new file mode 100644 index 000000000..c58636ec4 --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/collectl/__init__.pyi @@ -0,0 +1,4 @@ +# Stubs for galaxy.jobs.metrics.collectl (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + diff --git a/typeshed/2.7/galaxy/jobs/metrics/collectl/cli.pyi b/typeshed/2.7/galaxy/jobs/metrics/collectl/cli.pyi new file mode 100644 index 000000000..88e3120dd --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/collectl/cli.pyi @@ -0,0 +1,12 @@ +# Stubs for galaxy.jobs.metrics.collectl.cli (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class CollectlCli: + mode = ... # type: Any + command_args = ... # type: Any + def __init__(self, **kwargs) -> None: ... + def build_command_line(self): ... + def run(self, stdout: Any = ..., stderr: Any = ...): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/collectl/processes.pyi b/typeshed/2.7/galaxy/jobs/metrics/collectl/processes.pyi new file mode 100644 index 000000000..ad6fbd88a --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/collectl/processes.pyi @@ -0,0 +1,24 @@ +# Stubs for galaxy.jobs.metrics.collectl.processes (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +def generate_process_statistics(collectl_playback_cli, pid, statistics: Any = ...): ... + +class CollectlProcessSummarizer: + pid = ... # type: Any + statistics = ... # type: Any + columns_of_interest = ... # type: Any + tree_statistics = ... # type: Any + process_accum_statistics = ... # type: Any + interval_count = ... # type: int + def __init__(self, pid, statistics) -> None: ... + def handle_interval(self, interval): ... + def get_statistics(self): ... + +class CollectlProcessInterval: + rows = ... # type: Any + def __init__(self) -> None: ... + def row_is_in(self, row): ... + def add_row(self, row): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/collectl/stats.pyi b/typeshed/2.7/galaxy/jobs/metrics/collectl/stats.pyi new file mode 100644 index 000000000..39f0c8d0d --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/collectl/stats.pyi @@ -0,0 +1,15 @@ +# Stubs for galaxy.jobs.metrics.collectl.stats (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class StatisticsTracker: + min = ... # type: Any + max = ... # type: Any + count = ... # type: int + sum = ... # type: int + def __init__(self) -> None: ... + def track(self, value): ... + @property + def avg(self): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/collectl/subsystems.pyi b/typeshed/2.7/galaxy/jobs/metrics/collectl/subsystems.pyi new file mode 100644 index 000000000..1da48e66d --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/collectl/subsystems.pyi @@ -0,0 +1,35 @@ +# Stubs for galaxy.jobs.metrics.collectl.subsystems (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +class CollectlSubsystem: + @property + def command_line_arg(self): ... + @property + def name(self, job_directory): ... + +class ProcessesSubsystem(CollectlSubsystem): + command_line_arg = ... # type: str + name = ... # type: str + +class CpuSubsystem(CollectlSubsystem): + command_line_arg = ... # type: str + name = ... # type: str + +class DiskSubsystem(CollectlSubsystem): + command_line_arg = ... # type: str + name = ... # type: str + +class NetworkSubsystem(CollectlSubsystem): + command_line_arg = ... # type: str + name = ... # type: str + +class EnvironmentSubsystem(CollectlSubsystem): + command_line_arg = ... # type: str + name = ... # type: str + +class MemorySubsystem(CollectlSubsystem): + command_line_arg = ... # type: str + name = ... # type: str + +def get_subsystem(name): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/formatting.pyi b/typeshed/2.7/galaxy/jobs/metrics/formatting.pyi new file mode 100644 index 000000000..ebc91b1ff --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/formatting.pyi @@ -0,0 +1,8 @@ +# Stubs for galaxy.jobs.metrics.formatting (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +class JobMetricFormatter: + def format(self, key, value): ... + +def seconds_to_str(value): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/instrumenters/__init__.pyi b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/__init__.pyi new file mode 100644 index 000000000..ede6717e0 --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/__init__.pyi @@ -0,0 +1,16 @@ +# Stubs for galaxy.jobs.metrics.instrumenters (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from ...metrics import formatting as formatting + +INSTRUMENT_FILE_PREFIX = ... # type: str + +class InstrumentPlugin: + formatter = ... # type: Any + @property + def plugin_type(self): ... + def pre_execute_instrument(self, job_directory): ... + def post_execute_instrument(self, job_directory): ... + def job_properties(self, job_id, job_directory): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/instrumenters/collectl.pyi b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/collectl.pyi new file mode 100644 index 000000000..080d2e5b4 --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/collectl.pyi @@ -0,0 +1,22 @@ +# Stubs for galaxy.jobs.metrics.instrumenters.collectl (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import formatting +from ..instrumenters import InstrumentPlugin + +class CollectlFormatter(formatting.JobMetricFormatter): + def format(self, key, value): ... + +class CollectlPlugin(InstrumentPlugin): + plugin_type = ... # type: str + formatter = ... # type: Any + saved_logs_path = ... # type: Any + summarize_process_data = ... # type: Any + log_collectl_program_output = ... # type: Any + process_statistics = ... # type: Any + def __init__(self, **kwargs) -> None: ... + def pre_execute_instrument(self, job_directory): ... + def post_execute_instrument(self, job_directory): ... + def job_properties(self, job_id, job_directory): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/instrumenters/core.pyi b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/core.pyi new file mode 100644 index 000000000..885df5a2b --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/core.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.jobs.metrics.instrumenters.core (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import formatting +from ..instrumenters import InstrumentPlugin + +class CorePluginFormatter(formatting.JobMetricFormatter): + def format(self, key, value): ... + +class CorePlugin(InstrumentPlugin): + plugin_type = ... # type: str + formatter = ... # type: Any + def __init__(self, **kwargs) -> None: ... + def pre_execute_instrument(self, job_directory): ... + def post_execute_instrument(self, job_directory): ... + def job_properties(self, job_id, job_directory): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/instrumenters/cpuinfo.pyi b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/cpuinfo.pyi new file mode 100644 index 000000000..00744ed68 --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/cpuinfo.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.jobs.metrics.instrumenters.cpuinfo (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import formatting +from ..instrumenters import InstrumentPlugin + +class CpuInfoFormatter(formatting.JobMetricFormatter): + def format(self, key, value): ... + +class CpuInfoPlugin(InstrumentPlugin): + plugin_type = ... # type: str + formatter = ... # type: Any + verbose = ... # type: Any + def __init__(self, **kwargs) -> None: ... + def pre_execute_instrument(self, job_directory): ... + def job_properties(self, job_id, job_directory): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/instrumenters/env.pyi b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/env.pyi new file mode 100644 index 000000000..411332205 --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/env.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.jobs.metrics.instrumenters.env (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import formatting +from ..instrumenters import InstrumentPlugin + +class EnvFormatter(formatting.JobMetricFormatter): ... + +class EnvPlugin(InstrumentPlugin): + plugin_type = ... # type: str + formatter = ... # type: Any + variables = ... # type: Any + def __init__(self, **kwargs) -> None: ... + def pre_execute_instrument(self, job_directory): ... + def post_execute_instrument(self, job_directory): ... + def job_properties(self, job_id, job_directory): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/instrumenters/meminfo.pyi b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/meminfo.pyi new file mode 100644 index 000000000..344cfa861 --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/meminfo.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.jobs.metrics.instrumenters.meminfo (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import formatting +from ..instrumenters import InstrumentPlugin + +class MemInfoFormatter(formatting.JobMetricFormatter): + def format(self, key, value): ... + +class MemInfoPlugin(InstrumentPlugin): + plugin_type = ... # type: str + formatter = ... # type: Any + verbose = ... # type: Any + def __init__(self, **kwargs) -> None: ... + def pre_execute_instrument(self, job_directory): ... + def job_properties(self, job_id, job_directory): ... diff --git a/typeshed/2.7/galaxy/jobs/metrics/instrumenters/uname.pyi b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/uname.pyi new file mode 100644 index 000000000..b541bd02e --- /dev/null +++ b/typeshed/2.7/galaxy/jobs/metrics/instrumenters/uname.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.jobs.metrics.instrumenters.uname (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import formatting +from ..instrumenters import InstrumentPlugin + +class UnameFormatter(formatting.JobMetricFormatter): + def format(self, key, value): ... + +class UnamePlugin(InstrumentPlugin): + plugin_type = ... # type: str + formatter = ... # type: Any + uname_args = ... # type: Any + def __init__(self, **kwargs) -> None: ... + def pre_execute_instrument(self, job_directory): ... + def job_properties(self, job_id, job_directory): ... diff --git a/typeshed/2.7/galaxy/objectstore/__init__.pyi b/typeshed/2.7/galaxy/objectstore/__init__.pyi new file mode 100644 index 000000000..82259bceb --- /dev/null +++ b/typeshed/2.7/galaxy/objectstore/__init__.pyi @@ -0,0 +1,80 @@ +# Stubs for galaxy.objectstore (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +object_session = ... # type: Any +NO_SESSION_ERROR_MESSAGE = ... # type: str +log = ... # type: Any + +class ObjectStore: + running = ... # type: bool + extra_dirs = ... # type: Any + config = ... # type: Any + check_old_style = ... # type: Any + def __init__(self, config, **kwargs) -> None: ... + def shutdown(self): ... + def exists(self, obj, base_dir: Optional[Any] = ..., dir_only: bool = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ...): ... + def file_ready(self, obj, base_dir: Optional[Any] = ..., dir_only: bool = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def create(self, obj, base_dir: Optional[Any] = ..., dir_only: bool = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def empty(self, obj, base_dir: Optional[Any] = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def size(self, obj, extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def delete(self, obj, entire_dir: bool = ..., base_dir: Optional[Any] = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def get_data(self, obj, start: int = ..., count: int = ..., base_dir: Optional[Any] = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def get_filename(self, obj, base_dir: Optional[Any] = ..., dir_only: bool = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def update_from_file(self, obj, base_dir: Optional[Any] = ..., extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ..., file_name: Optional[Any] = ..., create: bool = ...): ... + def get_object_url(self, obj, extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ..., obj_dir: bool = ...): ... + def get_store_usage_percent(self): ... + +class DiskObjectStore(ObjectStore): + file_path = ... # type: Any + def __init__(self, config, config_xml: Optional[Any] = ..., file_path: Optional[Any] = ..., extra_dirs: Optional[Any] = ...) -> None: ... + def exists(self, obj, **kwargs): ... + def create(self, obj, **kwargs): ... + def empty(self, obj, **kwargs): ... + def size(self, obj, **kwargs): ... + def delete(self, obj, entire_dir: bool = ..., **kwargs): ... + def get_data(self, obj, start: int = ..., count: int = ..., **kwargs): ... + def get_filename(self, obj, **kwargs): ... + def update_from_file(self, obj, file_name: Optional[Any] = ..., create: bool = ..., **kwargs): ... + def get_object_url(self, obj, **kwargs): ... + def get_store_usage_percent(self): ... + +class NestedObjectStore(ObjectStore): + backends = ... # type: Any + def __init__(self, config, config_xml: Optional[Any] = ...) -> None: ... + def shutdown(self): ... + def exists(self, obj, **kwargs): ... + def file_ready(self, obj, **kwargs): ... + def create(self, obj, **kwargs): ... + def empty(self, obj, **kwargs): ... + def size(self, obj, **kwargs): ... + def delete(self, obj, **kwargs): ... + def get_data(self, obj, **kwargs): ... + def get_filename(self, obj, **kwargs): ... + def update_from_file(self, obj, **kwargs): ... + def get_object_url(self, obj, **kwargs): ... + +class DistributedObjectStore(NestedObjectStore): + distributed_config = ... # type: Any + backends = ... # type: Any + weighted_backend_ids = ... # type: Any + original_weighted_backend_ids = ... # type: Any + max_percent_full = ... # type: Any + global_max_percent_full = ... # type: float + sleeper = ... # type: Any + filesystem_monitor_thread = ... # type: Any + def __init__(self, config, config_xml: Optional[Any] = ..., fsmon: bool = ...) -> None: ... + def shutdown(self): ... + def create(self, obj, **kwargs): ... + +class HierarchicalObjectStore(NestedObjectStore): + backends = ... # type: Any + def __init__(self, config, config_xml: Optional[Any] = ..., fsmon: bool = ...) -> None: ... + def exists(self, obj, **kwargs): ... + def create(self, obj, **kwargs): ... + +def build_object_store_from_config(config, fsmon: bool = ..., config_xml: Optional[Any] = ...): ... +def local_extra_dirs(func): ... +def convert_bytes(bytes): ... diff --git a/typeshed/2.7/galaxy/objectstore/azure_blob.pyi b/typeshed/2.7/galaxy/objectstore/azure_blob.pyi new file mode 100644 index 000000000..89b58c1ad --- /dev/null +++ b/typeshed/2.7/galaxy/objectstore/azure_blob.pyi @@ -0,0 +1,29 @@ +# Stubs for galaxy.objectstore.azure_blob (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..objectstore import convert_bytes as convert_bytes, ObjectStore as ObjectStore + +BlockBlobService = ... # type: Any +NO_BLOBSERVICE_ERROR_MESSAGE = ... # type: str +log = ... # type: Any + +class AzureBlobObjectStore(ObjectStore): + staging_path = ... # type: Any + transfer_progress = ... # type: int + cache_size = ... # type: Any + sleeper = ... # type: Any + cache_monitor_thread = ... # type: Any + def __init__(self, config, config_xml) -> None: ... + def exists(self, obj, **kwargs): ... + def file_ready(self, obj, **kwargs): ... + def create(self, obj, **kwargs): ... + def empty(self, obj, **kwargs): ... + def size(self, obj, **kwargs): ... + def delete(self, obj, entire_dir: bool = ..., **kwargs): ... + def get_data(self, obj, start: int = ..., count: int = ..., **kwargs): ... + def get_filename(self, obj, **kwargs): ... + def update_from_file(self, obj, file_name: Optional[Any] = ..., create: bool = ..., **kwargs): ... + def get_object_url(self, obj, **kwargs): ... + def get_store_usage_percent(self): ... diff --git a/typeshed/2.7/galaxy/objectstore/pulsar.pyi b/typeshed/2.7/galaxy/objectstore/pulsar.pyi new file mode 100644 index 000000000..bad104522 --- /dev/null +++ b/typeshed/2.7/galaxy/objectstore/pulsar.pyi @@ -0,0 +1,24 @@ +# Stubs for galaxy.objectstore.pulsar (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..objectstore import ObjectStore as ObjectStore + +ObjectStoreClientManager = ... # type: Any + +class PulsarObjectStore(ObjectStore): + pulsar_client = ... # type: Any + def __init__(self, config, config_xml) -> None: ... + def exists(self, obj, **kwds): ... + def file_ready(self, obj, **kwds): ... + def create(self, obj, **kwds): ... + def empty(self, obj, **kwds): ... + def size(self, obj, **kwds): ... + def delete(self, obj, **kwds): ... + def get_data(self, obj, **kwds): ... + def get_filename(self, obj, **kwds): ... + def update_from_file(self, obj, **kwds): ... + def get_store_usage_percent(self): ... + def get_object_url(self, obj, extra_dir: Optional[Any] = ..., extra_dir_at_root: bool = ..., alt_name: Optional[Any] = ...): ... + def shutdown(self): ... diff --git a/typeshed/2.7/galaxy/objectstore/rods.pyi b/typeshed/2.7/galaxy/objectstore/rods.pyi new file mode 100644 index 000000000..125444b20 --- /dev/null +++ b/typeshed/2.7/galaxy/objectstore/rods.pyi @@ -0,0 +1,32 @@ +# Stubs for galaxy.objectstore.rods (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from posixpath import basename as path_basename +from posixpath import dirname as path_dirname +from posixpath import join as path_join +from ..objectstore import DiskObjectStore as DiskObjectStore, local_extra_dirs as local_extra_dirs + +irods = ... # type: Any +IRODS_IMPORT_MESSAGE = ... # type: str +log = ... # type: Any + +class IRODSObjectStore(DiskObjectStore): + cache_path = ... # type: Any + default_resource = ... # type: Any + root_collection_path = ... # type: Any + root_collection = ... # type: Any + def __init__(self, config, file_path: Optional[Any] = ..., extra_dirs: Optional[Any] = ...) -> None: ... + def exists(self, obj, **kwargs): ... + def create(self, obj, **kwargs): ... + def empty(self, obj, **kwargs): ... + def size(self, obj, **kwargs): ... + def delete(self, obj, entire_dir: bool = ..., **kwargs): ... + def get_data(self, obj, start: int = ..., count: int = ..., **kwargs): ... + def get_filename(self, obj, **kwargs): ... + def update_from_file(self, obj, file_name: Optional[Any] = ..., create: bool = ..., **kwargs): ... + def get_object_url(self, obj, **kwargs): ... + def get_store_usage_percent(self): ... + +def rods_connect(): ... diff --git a/typeshed/2.7/galaxy/objectstore/s3.pyi b/typeshed/2.7/galaxy/objectstore/s3.pyi new file mode 100644 index 000000000..8ffbd92c5 --- /dev/null +++ b/typeshed/2.7/galaxy/objectstore/s3.pyi @@ -0,0 +1,34 @@ +# Stubs for galaxy.objectstore.s3 (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from .s3_multipart_upload import multipart_upload as multipart_upload +from ..objectstore import convert_bytes as convert_bytes, ObjectStore as ObjectStore + +boto = ... # type: Any +NO_BOTO_ERROR_MESSAGE = ... # type: str +log = ... # type: Any + +class S3ObjectStore(ObjectStore): + staging_path = ... # type: Any + transfer_progress = ... # type: int + bucket = ... # type: Any + cache_size = ... # type: Any + sleeper = ... # type: Any + cache_monitor_thread = ... # type: Any + use_axel = ... # type: bool + def __init__(self, config, config_xml) -> None: ... + def file_ready(self, obj, **kwargs): ... + def exists(self, obj, **kwargs): ... + def create(self, obj, **kwargs): ... + def empty(self, obj, **kwargs): ... + def size(self, obj, **kwargs): ... + def delete(self, obj, entire_dir: bool = ..., **kwargs): ... + def get_data(self, obj, start: int = ..., count: int = ..., **kwargs): ... + def get_filename(self, obj, **kwargs): ... + def update_from_file(self, obj, file_name: Optional[Any] = ..., create: bool = ..., **kwargs): ... + def get_object_url(self, obj, **kwargs): ... + def get_store_usage_percent(self): ... + +class SwiftObjectStore(S3ObjectStore): ... diff --git a/typeshed/2.7/galaxy/objectstore/s3_multipart_upload.pyi b/typeshed/2.7/galaxy/objectstore/s3_multipart_upload.pyi new file mode 100644 index 000000000..9bf1d8726 --- /dev/null +++ b/typeshed/2.7/galaxy/objectstore/s3_multipart_upload.pyi @@ -0,0 +1,13 @@ +# Stubs for galaxy.objectstore.s3_multipart_upload (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +boto = ... # type: Any + +def map_wrap(f): ... +def mp_from_ids(s3server, mp_id, mp_keyname, mp_bucketname): ... +def transfer_part(s3server, mp_id, mp_keyname, mp_bucketname, i, part): ... +def multipart_upload(s3server, bucket, s3_key_name, tarball, mb_size): ... +def multimap(cores: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/__init__.pyi b/typeshed/2.7/galaxy/tools/__init__.pyi index af6879982..ef883e955 100644 --- a/typeshed/2.7/galaxy/tools/__init__.pyi +++ b/typeshed/2.7/galaxy/tools/__init__.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools (Python 3.5) +# Stubs for galaxy.tools (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. diff --git a/typeshed/2.7/galaxy/tools/cwl/__init__.pyi b/typeshed/2.7/galaxy/tools/cwl/__init__.pyi new file mode 100644 index 000000000..72bc0494f --- /dev/null +++ b/typeshed/2.7/galaxy/tools/cwl/__init__.pyi @@ -0,0 +1,8 @@ +# Stubs for galaxy.tools.cwl (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from .cwltool_deps import needs_shell_quoting as needs_shell_quoting, shellescape as shellescape +from .parser import tool_proxy as tool_proxy, workflow_proxy as workflow_proxy +from .representation import to_cwl_job as to_cwl_job, to_galaxy_parameters as to_galaxy_parameters +from .runtime_actions import handle_outputs as handle_outputs diff --git a/typeshed/2.7/galaxy/tools/cwl/cwltool_deps.pyi b/typeshed/2.7/galaxy/tools/cwl/cwltool_deps.pyi new file mode 100644 index 000000000..351a4208d --- /dev/null +++ b/typeshed/2.7/galaxy/tools/cwl/cwltool_deps.pyi @@ -0,0 +1,22 @@ +# Stubs for galaxy.tools.cwl.cwltool_deps (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from cwltool import main as main, workflow as workflow, process as process, pathmapper as pathmapper +from cwltool import load_tool as load_tool +import shellescape as shellescape +import schema_salad as schema_salad +from schema_salad import ref_resolver as ref_resolver + +main = ... # type: Any +workflow = ... # type: Any +process = ... # type: Any +pathmapper = ... # type: Any +load_tool = ... # type: Any +shellescape = ... # type: Any +schema_salad = ... # type: Any +ref_resolver = ... # type: Any +needs_shell_quoting = ... # type: Any + +def ensure_cwltool_available(): ... diff --git a/typeshed/2.7/galaxy/tools/cwl/parser.pyi b/typeshed/2.7/galaxy/tools/cwl/parser.pyi new file mode 100644 index 000000000..369b14688 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/cwl/parser.pyi @@ -0,0 +1,94 @@ +# Stubs for galaxy.tools.cwl.parser (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def tool_proxy(tool_path, strict_cwl_validation: bool = ...): ... +def load_job_proxy(job_directory, strict_cwl_validation: bool = ...): ... + +class ToolProxy: + def __init__(self, tool, tool_path) -> None: ... + def job_proxy(self, input_dict, output_dict, job_directory: str = ...): ... + def input_instances(self): ... + def output_instances(self): ... + def docker_identifier(self): ... + def description(self): ... + def label(self): ... + +class CommandLineToolProxy(ToolProxy): + def description(self): ... + def label(self): ... + def input_instances(self): ... + def output_instances(self): ... + def docker_identifier(self): ... + +class ExpressionToolProxy(CommandLineToolProxy): ... + +class JobProxy: + def __init__(self, tool_proxy, input_dict, output_dict, job_directory) -> None: ... + def cwl_job(self): ... + @property + def is_command_line_job(self): ... + @property + def command_line(self): ... + @property + def stdin(self): ... + @property + def stdout(self): ... + @property + def environment(self): ... + @property + def generate_files(self): ... + def collect_outputs(self, tool_working_directory): ... + def save_job(self): ... + def output_id(self, output_name): ... + def output_path(self, output_name): ... + def output_secondary_files_dir(self, output_name, create: bool = ...): ... + def stage_files(self): ... + +class WorkflowProxy: + def __init__(self, workflow, workflow_path) -> None: ... + def step_proxies(self): ... + @property + def runnables(self): ... + def to_dict(self): ... + +class StepProxy: + def __init__(self, workflow_proxy, step) -> None: ... + def to_dict(self): ... + +class ConditionalInstance: + input_type = ... # type: Any + name = ... # type: Any + case = ... # type: Any + whens = ... # type: Any + def __init__(self, name, case, whens) -> None: ... + def to_dict(self): ... + +class SelectInputInstance: + input_type = ... # type: Any + name = ... # type: Any + label = ... # type: Any + description = ... # type: Any + options = ... # type: Any + def __init__(self, name, label, description, options) -> None: ... + def to_dict(self): ... + +class InputInstance: + input_type = ... # type: Any + name = ... # type: Any + label = ... # type: Any + description = ... # type: Any + required = ... # type: bool + array = ... # type: Any + area = ... # type: Any + def __init__(self, name, label, description, input_type, array: bool = ..., area: bool = ...) -> None: ... + def to_dict(self, itemwise: bool = ...): ... + +class OutputInstance: + name = ... # type: Any + output_data_type = ... # type: Any + output_type = ... # type: Any + path = ... # type: Any + def __init__(self, name, output_data_type, output_type, path: Optional[Any] = ...) -> None: ... diff --git a/typeshed/2.7/galaxy/tools/cwl/representation.pyi b/typeshed/2.7/galaxy/tools/cwl/representation.pyi new file mode 100644 index 000000000..ef2cc45fe --- /dev/null +++ b/typeshed/2.7/galaxy/tools/cwl/representation.pyi @@ -0,0 +1,12 @@ +# Stubs for galaxy.tools.cwl.representation (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +log = ... # type: Any +NOT_PRESENT = ... # type: Any +GALAXY_TO_CWL_TYPES = ... # type: Any + +def to_cwl_job(tool, param_dict, local_working_directory): ... +def to_galaxy_parameters(tool, as_dict): ... diff --git a/typeshed/2.7/galaxy/tools/cwl/runtime_actions.pyi b/typeshed/2.7/galaxy/tools/cwl/runtime_actions.pyi new file mode 100644 index 000000000..4f1558d99 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/cwl/runtime_actions.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.tools.cwl.runtime_actions (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def handle_outputs(job_directory: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/cwl/schema.pyi b/typeshed/2.7/galaxy/tools/cwl/schema.pyi new file mode 100644 index 000000000..1c51ead4e --- /dev/null +++ b/typeshed/2.7/galaxy/tools/cwl/schema.pyi @@ -0,0 +1,22 @@ +# Stubs for galaxy.tools.cwl.schema (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from .cwltool_deps import ensure_cwltool_available as ensure_cwltool_available, load_tool as load_tool, schema_salad as schema_salad, workflow as workflow +from collections import namedtuple + +RawProcessReference = namedtuple('RawProcessReference', ['process_object', 'uri']) + +ProcessDefinition = namedtuple('ProcessDefinition', ['process_object', 'metadata', 'document_loader', 'avsc_names', 'raw_process_reference']) + +class SchemaLoader: + def __init__(self, strict: bool = ...) -> None: ... + @property + def raw_document_loader(self): ... + def raw_process_reference(self, path): ... + def process_definition(self, raw_reference): ... + def tool(self, **kwds): ... + +schema_loader = ... # type: Any +non_strict_schema_loader = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/deps/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/__init__.pyi index b3bd75ccf..25c761e1b 100644 --- a/typeshed/2.7/galaxy/tools/deps/__init__.pyi +++ b/typeshed/2.7/galaxy/tools/deps/__init__.pyi @@ -1,32 +1,41 @@ -# Stubs for galaxy.tools.deps (Python 3.5) +# Stubs for galaxy.tools.deps (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. from typing import Any, Optional +from .requirements import ToolRequirement as ToolRequirement, ToolRequirements as ToolRequirements from .resolvers import NullDependency as NullDependency from .resolvers.conda import CondaDependencyResolver as CondaDependencyResolver from .resolvers.galaxy_packages import GalaxyPackageDependencyResolver as GalaxyPackageDependencyResolver from .resolvers.tool_shed_packages import ToolShedPackageDependencyResolver as ToolShedPackageDependencyResolver log = ... # type: Any -EXTRA_CONFIG_KWDS = ... # type: Any CONFIG_VAL_NOT_FOUND = ... # type: Any -def build_dependency_manager(config: Any): ... # type: DependencyManager +def build_dependency_manager(config): ... class NullDependencyManager: dependency_resolvers = ... # type: Any def uses_tool_shed_dependencies(self): ... - def dependency_shell_commands(self, requirements: Any, **kwds) -> List[str]: ... + def dependency_shell_commands(self, requirements, **kwds): ... def find_dep(self, name, version: Optional[Any] = ..., type: str = ..., **kwds): ... class DependencyManager: - extra_config = ... # type: Any default_base_path = ... # type: Any resolver_classes = ... # type: Any dependency_resolvers = ... # type: Any - def __init__(self, default_base_path, conf_file: Optional[Any] = ..., **extra_config) -> None: ... - def dependency_shell_commands(self, requirements: Any, **kwds) -> List[str]: ... + def __init__(self, default_base_path, conf_file: Optional[Any] = ..., app_config: Any = ...) -> None: ... + def get_resolver_option(self, resolver, key, explicit_resolver_options: Any = ...): ... + def get_app_option(self, key, default: Optional[Any] = ...): ... + def dependency_shell_commands(self, requirements, **kwds): ... def requirements_to_dependencies(self, requirements, **kwds): ... def uses_tool_shed_dependencies(self): ... def find_dep(self, name, version: Optional[Any] = ..., type: str = ..., **kwds): ... + +class CachedDependencyManager(DependencyManager): + tool_dependency_cache_dir = ... # type: Any + def __init__(self, default_base_path, conf_file: Optional[Any] = ..., app_config: Any = ..., tool_dependency_cache_dir: Optional[Any] = ...) -> None: ... + def build_cache(self, requirements, **kwds): ... + def dependency_shell_commands(self, requirements, **kwds): ... + def hash_dependencies(self, resolved_dependencies): ... + def get_hashed_dependencies_path(self, resolved_dependencies): ... diff --git a/typeshed/2.7/galaxy/tools/deps/brew_exts.pyi b/typeshed/2.7/galaxy/tools/deps/brew_exts.pyi index f2074d55d..c526375e6 100644 --- a/typeshed/2.7/galaxy/tools/deps/brew_exts.pyi +++ b/typeshed/2.7/galaxy/tools/deps/brew_exts.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.brew_exts (Python 3.5) +# Stubs for galaxy.tools.deps.brew_exts (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. diff --git a/typeshed/2.7/galaxy/tools/deps/brew_util.pyi b/typeshed/2.7/galaxy/tools/deps/brew_util.pyi new file mode 100644 index 000000000..c791f1832 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/brew_util.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.tools.deps.brew_util (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from ..deps import brew_exts as brew_exts + +DEFAULT_TAP = ... # type: str + +class HomebrewRecipe: + recipe = ... # type: Any + version = ... # type: Any + tap = ... # type: Any + def __init__(self, recipe, version, tap) -> None: ... + +def requirements_to_recipes(requirements): ... +def requirement_to_recipe(requirement): ... +def requirements_to_recipe_contexts(requirements, brew_context): ... diff --git a/typeshed/2.7/galaxy/tools/deps/commands.pyi b/typeshed/2.7/galaxy/tools/deps/commands.pyi index aad9cae42..a11b90dc8 100644 --- a/typeshed/2.7/galaxy/tools/deps/commands.pyi +++ b/typeshed/2.7/galaxy/tools/deps/commands.pyi @@ -1,10 +1,9 @@ -# Stubs for galaxy.tools.deps.commands (Python 3.5) +# Stubs for galaxy.tools.deps.commands (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. from typing import Any, Optional from galaxy.util import which as which -def shell_quote(args: Any = ...): ... def redirecting_io(sys: Any = ...): ... def redirect_aware_commmunicate(p, sys: Any = ...): ... @@ -18,6 +17,6 @@ class CommandLineException(Exception): command = ... # type: Any stdout = ... # type: Any stderr = ... # type: Any + returncode = ... # type: Any message = ... # type: Any - def __init__(self, command, stdout, stderr) -> None: ... - + def __init__(self, command, stdout, stderr, returncode) -> None: ... diff --git a/typeshed/2.7/galaxy/tools/deps/conda_compat.pyi b/typeshed/2.7/galaxy/tools/deps/conda_compat.pyi new file mode 100644 index 000000000..0cc638263 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/conda_compat.pyi @@ -0,0 +1,21 @@ +# Stubs for galaxy.tools.deps.conda_compat (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from conda_build.metadata import MetaData as MetaData + +MetaData = ... # type: Any + +class _Memoized: + func = ... # type: Any + cache = ... # type: Any + def __init__(self, func) -> None: ... + def __call__(self, *args): ... + +def raw_metadata(recipe_dir): ... + +class _MetaData: + meta = ... # type: Any + def __init__(self, input_dir) -> None: ... + def get_value(self, field, default: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/deps/conda_util.pyi b/typeshed/2.7/galaxy/tools/deps/conda_util.pyi index 55082a96a..1e418baab 100644 --- a/typeshed/2.7/galaxy/tools/deps/conda_util.pyi +++ b/typeshed/2.7/galaxy/tools/deps/conda_util.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.conda_util (Python 3.5) +# Stubs for galaxy.tools.deps.conda_util (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. @@ -12,11 +12,18 @@ class CondaContext(installable.InstallableContext): conda_exec = ... # type: Any debug = ... # type: Any shell_exec = ... # type: Any + copy_dependencies = ... # type: Any conda_prefix = ... # type: Any ensure_channels = ... # type: Any ensured_channels = ... # type: bool - def __init__(self, conda_prefix: Optional[Any] = ..., conda_exec: Optional[Any] = ..., shell_exec: Optional[Any] = ..., debug: bool = ..., ensure_channels: str = ..., condarc_override: Optional[Any] = ..., use_path_exec: Any = ...) -> None: ... + use_local = ... # type: Any + def __init__(self, conda_prefix: Optional[Any] = ..., conda_exec: Optional[Any] = ..., shell_exec: Optional[Any] = ..., debug: bool = ..., ensure_channels: str = ..., condarc_override: Optional[Any] = ..., use_path_exec: Any = ..., copy_dependencies: bool = ..., use_local: Any = ...) -> None: ... + @property + def conda_version(self): ... + @property + def conda_build_available(self): ... def ensure_channels_configured(self): ... + def ensure_conda_build_installed_if_needed(self): ... def conda_info(self): ... def is_conda_installed(self): ... def can_install_conda(self): ... @@ -26,9 +33,10 @@ class CondaContext(installable.InstallableContext): def condarc(self): ... def command(self, operation, args): ... def exec_command(self, operation, args): ... - def exec_create(self, args): ... + def exec_create(self, args, allow_local: bool = ...): ... def exec_remove(self, args): ... - def exec_install(self, args): ... + def exec_install(self, args, allow_local: bool = ...): ... + def exec_clean(self, args: Any = ..., quiet: bool = ...): ... def export_list(self, name, path): ... def env_path(self, env_name): ... @property @@ -56,6 +64,6 @@ class CondaTarget: def __eq__(self, other): ... def __ne__(self, other): ... -def install_conda(conda_context: Optional[Any] = ...): ... -def install_conda_target(conda_target, conda_context: Optional[Any] = ...): ... +def install_conda(conda_context: Optional[Any] = ..., force_conda_build: bool = ...): ... +def install_conda_target(conda_target, conda_context: Optional[Any] = ..., skip_environment: bool = ...): ... def requirements_to_conda_targets(requirements, conda_context: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi index 045ccdb39..4e8c8222c 100644 --- a/typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi +++ b/typeshed/2.7/galaxy/tools/deps/container_resolvers/__init__.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.container_resolvers (Python 3.5) +# Stubs for galaxy.tools.deps.container_resolvers (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. @@ -7,7 +7,6 @@ from galaxy.util.dictifiable import Dictifiable class ContainerResolver(Dictifiable): dict_collection_visible_keys = ... # type: Any - __metaclass__ = ... # type: Any app_info = ... # type: Any resolver_kwds = ... # type: Any def __init__(self, app_info: Optional[Any] = ..., **kwds) -> None: ... diff --git a/typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi b/typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi index 511140171..3f45f9529 100644 --- a/typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi +++ b/typeshed/2.7/galaxy/tools/deps/container_resolvers/explicit.pyi @@ -1,11 +1,8 @@ -# Stubs for galaxy.tools.deps.container_resolvers.explicit (Python 3.5) +# Stubs for galaxy.tools.deps.container_resolvers.explicit (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. -from typing import Any -from ..container_resolvers import ContainerResolver as ContainerResolver - -log = ... # type: Any +from ..container_resolvers import ContainerResolver class ExplicitContainerResolver(ContainerResolver): resolver_type = ... # type: str diff --git a/typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi b/typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi index 0c04bd898..5f7a6b4ee 100644 --- a/typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi +++ b/typeshed/2.7/galaxy/tools/deps/container_resolvers/mulled.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.container_resolvers.mulled (Python 3.5) +# Stubs for galaxy.tools.deps.container_resolvers.mulled (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. @@ -8,23 +8,48 @@ from collections import namedtuple CachedMulledImageSingleTarget = namedtuple('CachedMulledImageSingleTarget', ['package_name', 'version', 'build', 'image_identifier']) -CachedMulledImageMultiTarget = namedtuple('CachedMulledImageMultiTarget', ['hash', 'image_identifier']) +CachedV1MulledImageMultiTarget = namedtuple('CachedV1MulledImageMultiTarget', ['hash', 'build', 'image_identifier']) -class CachedMulledContainerResolver(ContainerResolver): +CachedV2MulledImageMultiTarget = namedtuple('CachedV2MulledImageMultiTarget', ['package_hash', 'version_hash', 'build', 'image_identifier']) + +class CachedMulledDockerContainerResolver(ContainerResolver): resolver_type = ... # type: str + container_type = ... # type: str namespace = ... # type: Any - def __init__(self, app_info: Optional[Any] = ..., namespace: Optional[Any] = ...) -> None: ... + hash_func = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., namespace: Optional[Any] = ..., hash_func: str = ...) -> None: ... + def resolve(self, enabled_container_types, tool_info): ... + +class CachedMulledSingularityContainerResolver(ContainerResolver): + resolver_type = ... # type: str + container_type = ... # type: str + cache_directory = ... # type: Any + hash_func = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., hash_func: str = ...) -> None: ... def resolve(self, enabled_container_types, tool_info): ... -class MulledContainerResolver(ContainerResolver): +class MulledDockerContainerResolver(ContainerResolver): resolver_type = ... # type: str + container_type = ... # type: str namespace = ... # type: Any - def __init__(self, app_info: Optional[Any] = ..., namespace: str = ...) -> None: ... + hash_func = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., namespace: str = ..., hash_func: str = ...) -> None: ... def resolve(self, enabled_container_types, tool_info): ... -class BuildMulledContainerResolver(ContainerResolver): +class BuildMulledDockerContainerResolver(ContainerResolver): resolver_type = ... # type: str + container_type = ... # type: str namespace = ... # type: Any + hash_func = ... # type: Any + auto_init = ... # type: Any + def __init__(self, app_info: Optional[Any] = ..., namespace: str = ..., hash_func: str = ..., **kwds) -> None: ... + def resolve(self, enabled_container_types, tool_info): ... + +class BuildMulledSingularityContainerResolver(ContainerResolver): + resolver_type = ... # type: str + container_type = ... # type: str + cache_directory = ... # type: Any + hash_func = ... # type: Any auto_init = ... # type: Any - def __init__(self, app_info: Optional[Any] = ..., namespace: str = ..., **kwds) -> None: ... + def __init__(self, app_info: Optional[Any] = ..., hash_func: str = ..., **kwds) -> None: ... def resolve(self, enabled_container_types, tool_info): ... diff --git a/typeshed/2.7/galaxy/tools/deps/containers.pyi b/typeshed/2.7/galaxy/tools/deps/containers.pyi index 4eecb5440..5a18a6321 100644 --- a/typeshed/2.7/galaxy/tools/deps/containers.pyi +++ b/typeshed/2.7/galaxy/tools/deps/containers.pyi @@ -1,16 +1,18 @@ -# Stubs for galaxy.tools.deps.containers (Python 3.5) +# Stubs for galaxy.tools.deps.containers (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. from typing import Any, Optional from .container_resolvers.explicit import ExplicitContainerResolver as ExplicitContainerResolver -from .container_resolvers.mulled import BuildMulledContainerResolver as BuildMulledContainerResolver, CachedMulledContainerResolver as CachedMulledContainerResolver, MulledContainerResolver as MulledContainerResolver +from .container_resolvers.mulled import BuildMulledDockerContainerResolver as BuildMulledDockerContainerResolver, BuildMulledSingularityContainerResolver as BuildMulledSingularityContainerResolver, CachedMulledDockerContainerResolver as CachedMulledDockerContainerResolver, CachedMulledSingularityContainerResolver as CachedMulledSingularityContainerResolver, MulledDockerContainerResolver as MulledDockerContainerResolver from .requirements import ContainerDescription as ContainerDescription from .requirements import DEFAULT_CONTAINER_RESOLVE_DEPENDENCIES as DEFAULT_CONTAINER_RESOLVE_DEPENDENCIES, DEFAULT_CONTAINER_SHELL as DEFAULT_CONTAINER_SHELL from ..deps import docker_util as docker_util +from ..deps import singularity_util as singularity_util log = ... # type: Any DOCKER_CONTAINER_TYPE = ... # type: str +SINGULARITY_CONTAINER_TYPE = ... # type: str DEFAULT_CONTAINER_TYPE = ... # type: Any ALL_CONTAINER_TYPES = ... # type: Any LOAD_CACHED_IMAGE_COMMAND_TEMPLATE = ... # type: str @@ -19,6 +21,7 @@ class ContainerFinder: app_info = ... # type: Any container_registry = ... # type: Any def __init__(self, app_info) -> None: ... + def find_best_container_description(self, enabled_container_types, tool_info): ... def find_container(self, tool_info, destination_info, job_info): ... class NullContainerFinder: @@ -47,8 +50,9 @@ class AppInfo: class ToolInfo: container_descriptions = ... # type: Any requirements = ... # type: Any + requires_galaxy_python_environment = ... # type: Any env_pass_through = ... # type: Any - def __init__(self, container_descriptions: Any = ..., requirements: Any = ...) -> None: ... + def __init__(self, container_descriptions: Any = ..., requirements: Any = ..., requires_galaxy_python_environment: bool = ...) -> None: ... class JobInfo: working_directory = ... # type: Any @@ -58,7 +62,6 @@ class JobInfo: def __init__(self, working_directory, tool_directory, job_directory, job_directory_type) -> None: ... class Container: - __metaclass__ = ... # type: Any container_id = ... # type: Any app_info = ... # type: Any tool_info = ... # type: Any @@ -72,15 +75,25 @@ class Container: def shell(self): ... def containerize_command(self, command): ... -class DockerContainer(Container): +def preprocess_volumes(volumes_raw_str, container_type): ... + +class HasDockerLikeVolumes: ... + +class DockerContainer(Container, HasDockerLikeVolumes): + container_type = ... # type: Any def containerize_command(self, command): ... def docker_cache_path(cache_directory, container_id): ... +class SingularityContainer(Container, HasDockerLikeVolumes): + container_type = ... # type: Any + def containerize_command(self, command): ... + CONTAINER_CLASSES = ... # type: Any class NullContainer: def __init__(self) -> None: ... - def __nonzero__(self): ... + def __bool__(self): ... + __nonzero__ = ... # type: Any NULL_CONTAINER = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/deps/dependencies.pyi b/typeshed/2.7/galaxy/tools/deps/dependencies.pyi new file mode 100644 index 000000000..04907feff --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/dependencies.pyi @@ -0,0 +1,13 @@ +# Stubs for galaxy.tools.deps.dependencies (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class DependenciesDescription: + requirements = ... # type: Any + installed_tool_dependencies = ... # type: Any + def __init__(self, requirements: Any = ..., installed_tool_dependencies: Any = ...) -> None: ... + def to_dict(self): ... + @staticmethod + def from_dict(as_dict): ... diff --git a/typeshed/2.7/galaxy/tools/deps/docker_util.pyi b/typeshed/2.7/galaxy/tools/deps/docker_util.pyi index c6ea13d51..e98b338cc 100644 --- a/typeshed/2.7/galaxy/tools/deps/docker_util.pyi +++ b/typeshed/2.7/galaxy/tools/deps/docker_util.pyi @@ -1,9 +1,9 @@ -# Stubs for galaxy.tools.deps.docker_util (Python 3.5) +# Stubs for galaxy.tools.deps.docker_util (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. from typing import Any, Optional -from .commands import argv_to_str as argv_to_str, shell_quote as shell_quote +from .commands import argv_to_str as argv_to_str DEFAULT_DOCKER_COMMAND = ... # type: str DEFAULT_SUDO = ... # type: bool diff --git a/typeshed/2.7/galaxy/tools/deps/dockerfiles.pyi b/typeshed/2.7/galaxy/tools/deps/dockerfiles.pyi new file mode 100644 index 000000000..3cf02d563 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/dockerfiles.pyi @@ -0,0 +1,15 @@ +# Stubs for galaxy.tools.deps.dockerfiles (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..deps import commands as commands +from ..deps import docker_util as docker_util +from ..deps.containers import docker_cache_path as docker_cache_path +from ..deps.requirements import parse_requirements_from_xml as parse_requirements_from_xml +from ...tools import loader_directory as loader_directory + +log = ... # type: Any + +def docker_host_args(**kwds): ... +def dockerfile_build(path, dockerfile: Optional[Any] = ..., error: Any = ..., **kwds): ... diff --git a/typeshed/2.7/galaxy/tools/deps/installable.pyi b/typeshed/2.7/galaxy/tools/deps/installable.pyi new file mode 100644 index 000000000..818eaa26e --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/installable.pyi @@ -0,0 +1,15 @@ +# Stubs for galaxy.tools.deps.installable (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +log = ... # type: Any + +class InstallableContext: + def is_installed(self): ... + def can_install(self): ... + def installable_description(self): ... + def parent_path(self): ... + +def ensure_installed(installable_context, install_func, auto_init): ... diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/__init__.pyi new file mode 100644 index 000000000..a3f6bf315 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/__init__.pyi @@ -0,0 +1,4 @@ +# Stubs for galaxy.tools.deps.mulled (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/_cli.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/_cli.pyi new file mode 100644 index 000000000..db35991cc --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/_cli.pyi @@ -0,0 +1,5 @@ +# Stubs for galaxy.tools.deps.mulled._cli (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def arg_parser(argv, globals): ... diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build.pyi new file mode 100644 index 000000000..25650508b --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build.pyi @@ -0,0 +1,24 @@ +# Stubs for galaxy.tools.deps.mulled.mulled_build (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +import installable +from sys import platform as _platform + +class BuildExistsException(Exception): ... + +class InvolucroContext(installable.InstallableContext): + installable_description = ... # type: str + involucro_bin = ... # type: str + shell_exec = ... # type: Any + verbose = ... # type: Any + def __init__(self, involucro_bin: Optional[Any] = ..., shell_exec: Optional[Any] = ..., verbose: str = ...) -> None: ... + def build_command(self, involucro_args): ... + def exec_command(self, involucro_args): ... + def is_installed(self): ... + def can_install(self): ... + @property + def parent_path(self): ... + +def main(argv: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_channel.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_channel.pyi new file mode 100644 index 000000000..c0039bf4d --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_channel.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.tools.deps.mulled.mulled_build_channel (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def main(argv: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_files.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_files.pyi new file mode 100644 index 000000000..0057d4d8b --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_files.pyi @@ -0,0 +1,10 @@ +# Stubs for galaxy.tools.deps.mulled.mulled_build_files (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from collections import namedtuple + +def main(argv: Optional[Any] = ...): ... + +_Line = namedtuple('_Line', ['targets', 'image_build', 'name_override']) diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_tool.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_tool.pyi new file mode 100644 index 000000000..e4af0daca --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_build_tool.pyi @@ -0,0 +1,8 @@ +# Stubs for galaxy.tools.deps.mulled.mulled_build_tool (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def main(argv: Optional[Any] = ...): ... +def requirements_to_mulled_targets(requirements): ... diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/mulled_search.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_search.pyi new file mode 100644 index 000000000..9c8e9c23e --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/mulled_search.pyi @@ -0,0 +1,23 @@ +# Stubs for galaxy.tools.deps.mulled.mulled_search (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +requests = ... # type: Any +Schema = ... # type: Any +TEXT = ... # type: Any +STORED = ... # type: Any +create_in = ... # type: Any +QueryParser = ... # type: Any +QUAY_API_URL = ... # type: str + +class QuaySearch: + index = ... # type: Any + organization = ... # type: Any + def __init__(self, organization) -> None: ... + def build_index(self): ... + def search_repository(self, search_string, non_strict): ... + def get_additional_repository_information(self, repository_string): ... + +def main(argv: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/deps/mulled/util.pyi b/typeshed/2.7/galaxy/tools/deps/mulled/util.pyi new file mode 100644 index 000000000..33063c28d --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/mulled/util.pyi @@ -0,0 +1,23 @@ +# Stubs for galaxy.tools.deps.mulled.util (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from collections import namedtuple + +def quay_versions(namespace, pkg_name): ... +def mulled_tags_for(namespace, image, tag_prefix: Optional[Any] = ...): ... +def split_tag(tag): ... +def version_sorted(elements): ... + +Target = namedtuple('Target', ['package_name', 'version', 'build']) + +def build_target(package_name, version: Optional[Any] = ..., build: Optional[Any] = ..., tag: Optional[Any] = ...): ... +def conda_build_target_str(target): ... +def v1_image_name(targets, image_build: Optional[Any] = ..., name_override: Optional[Any] = ...): ... +def v2_image_name(targets, image_build: Optional[Any] = ..., name_override: Optional[Any] = ...): ... + +image_name = ... # type: Any + +# Names in __all__ with no definition: +# Target diff --git a/typeshed/2.7/galaxy/tools/deps/requirements.pyi b/typeshed/2.7/galaxy/tools/deps/requirements.pyi index 2764d6430..d422ef012 100644 --- a/typeshed/2.7/galaxy/tools/deps/requirements.pyi +++ b/typeshed/2.7/galaxy/tools/deps/requirements.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.requirements (Python 3.5) +# Stubs for galaxy.tools.deps.requirements (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. @@ -11,10 +11,50 @@ class ToolRequirement: name = ... # type: Any type = ... # type: Any version = ... # type: Any - def __init__(self, name: Optional[Any] = ..., type: Optional[Any] = ..., version: Optional[Any] = ...) -> None: ... + specs = ... # type: Any + def __init__(self, name: Optional[Any] = ..., type: Optional[Any] = ..., version: Optional[Any] = ..., specs: Any = ...) -> None: ... def to_dict(self): ... + def copy(self): ... @staticmethod - def from_dict(dict: Dict[Any, Any]): ... # type: ToolRequirement + def from_dict(dict): ... + def __eq__(self, other): ... + def __ne__(self, other): ... + def __hash__(self): ... + +class RequirementSpecification: + uri = ... # type: Any + version = ... # type: Any + def __init__(self, uri, version: Optional[Any] = ...) -> None: ... + @property + def specifies_version(self): ... + @property + def short_name(self): ... + def to_dict(self): ... + @staticmethod + def from_dict(dict): ... + def __eq__(self, other): ... + def __ne__(self, other): ... + def __hash__(self): ... + +class ToolRequirements: + tool_requirements = ... # type: Any + def __init__(self, tool_requirements: Optional[Any] = ...) -> None: ... + @staticmethod + def from_list(requirements): ... + @property + def resolvable(self): ... + @property + def packages(self): ... + def to_list(self): ... + def append(self, requirement): ... + def __eq__(self, other): ... + def __ne__(self, other): ... + def __iter__(self): ... + def __getitem__(self, ii): ... + def __len__(self): ... + def __hash__(self): ... + +class ToolRequirementsException(Exception): ... DEFAULT_CONTAINER_TYPE = ... # type: str DEFAULT_CONTAINER_RESOLVE_DEPENDENCIES = ... # type: bool diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi index 057e51c0f..a1ffb20a5 100644 --- a/typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/__init__.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.resolvers (Python 3.5) +# Stubs for galaxy.tools.deps.resolvers (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. @@ -10,20 +10,40 @@ class DependencyResolver(Dictifiable): dict_collection_visible_keys = ... # type: Any disabled = ... # type: bool resolves_simple_dependencies = ... # type: bool - __metaclass__ = ... # type: Any - def resolve(self, name, version, type, **kwds): ... + can_uninstall_dependencies = ... # type: bool + config_options = ... # type: Any + def resolve(self, requirement, **kwds): ... + +class MultipleDependencyResolver: + def resolve_all(self, requirements, **kwds): ... class ListableDependencyResolver: - __metaclass__ = ... # type: Any def list_dependencies(self): ... +class MappableDependencyResolver: ... + +FROM_UNVERSIONED = ... # type: Any + +class RequirementMapping: + from_name = ... # type: Any + from_version = ... # type: Any + to_name = ... # type: Any + to_version = ... # type: Any + def __init__(self, from_name, from_version, to_name, to_version) -> None: ... + def matches_requirement(self, requirement): ... + def apply(self, requirement): ... + @staticmethod + def from_dict(raw_mapping): ... + +class SpecificationAwareDependencyResolver: ... +class SpecificationPatternDependencyResolver: ... + class InstallableDependencyResolver: - __metaclass__ = ... # type: Any def install_dependency(self, name, version, type, **kwds): ... class Dependency(Dictifiable): dict_collection_visible_keys = ... # type: Any - __metaclass__ = ... # type: Any + cacheable = ... # type: bool def shell_commands(self, requirement): ... def exact(self): ... @property @@ -38,3 +58,5 @@ class NullDependency(Dependency): @property def resolver_msg(self): ... def shell_commands(self, requirement): ... + +class DependencyException(Exception): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/brewed_tool_shed_packages.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/brewed_tool_shed_packages.pyi new file mode 100644 index 000000000..d581de350 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/brewed_tool_shed_packages.pyi @@ -0,0 +1,33 @@ +# Stubs for galaxy.tools.deps.resolvers.brewed_tool_shed_packages (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from xml.etree import ElementTree as ET +from .resolver_mixins import UsesHomebrewMixin, UsesInstalledRepositoriesMixin, UsesToolDependencyDirMixin +from ..resolvers import DependencyResolver + +class HomebrewToolShedDependencyResolver(DependencyResolver, UsesHomebrewMixin, UsesToolDependencyDirMixin, UsesInstalledRepositoriesMixin): + resolver_type = ... # type: str + def __init__(self, dependency_manager, **kwds) -> None: ... + def resolve(self, requirement, **kwds): ... + +class RawDependencies: + root = ... # type: Any + dependencies = ... # type: Any + def __init__(self, dependencies_file) -> None: ... + def find(self, package_name, package_version): ... + +class RawDependency: + dependencies = ... # type: Any + package_el = ... # type: Any + repository_el = ... # type: Any + def __init__(self, dependencies, package_el, repository_el) -> None: ... + @property + def repository_owner(self): ... + @property + def repository_name(self): ... + @property + def package_name(self): ... + @property + def package_version(self): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi index 8308d36f2..333954de7 100644 --- a/typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/conda.pyi @@ -1,13 +1,17 @@ -# Stubs for galaxy.tools.deps.resolvers.conda (Python 3.5) +# Stubs for galaxy.tools.deps.resolvers.conda (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. from typing import Any, Optional -from ..resolvers import Dependency, DependencyResolver, InstallableDependencyResolver, ListableDependencyResolver +from ..resolvers import Dependency, DependencyResolver, InstallableDependencyResolver, ListableDependencyResolver, MappableDependencyResolver, MultipleDependencyResolver, SpecificationPatternDependencyResolver -class CondaDependencyResolver(DependencyResolver, ListableDependencyResolver, InstallableDependencyResolver): +DEFAULT_ENSURE_CHANNELS = ... # type: str + +class CondaDependencyResolver(DependencyResolver, MultipleDependencyResolver, ListableDependencyResolver, InstallableDependencyResolver, SpecificationPatternDependencyResolver, MappableDependencyResolver): dict_collection_visible_keys = ... # type: Any resolver_type = ... # type: str + config_options = ... # type: Any + can_uninstall_dependencies = ... # type: bool versionless = ... # type: Any dependency_manager = ... # type: Any conda_prefix_parent = ... # type: Any @@ -17,24 +21,52 @@ class CondaDependencyResolver(DependencyResolver, ListableDependencyResolver, In disabled = ... # type: Any auto_install = ... # type: Any copy_dependencies = ... # type: Any - verbose_install_check = ... # type: Any def __init__(self, dependency_manager, **kwds) -> None: ... - def resolve(self, name, version, type, **kwds): ... + def clean(self, **kwds): ... + def uninstall(self, requirements): ... + def uninstall_environments(self, environments): ... + def install_all(self, conda_targets): ... + def resolve_all(self, requirements, **kwds): ... + def merged_environment_name(self, conda_targets): ... + def resolve(self, requirement, **kwds): ... + def unused_dependency_paths(self, toolbox_requirements_status): ... def list_dependencies(self): ... def install_dependency(self, name, version, type, **kwds): ... @property def prefix(self): ... +class MergedCondaDependency(Dependency): + dict_collection_visible_keys = ... # type: Any + dependency_type = ... # type: str + activate = ... # type: Any + conda_context = ... # type: Any + environment_path = ... # type: Any + cache_path = ... # type: Any + def __init__(self, conda_context, environment_path, exact, name: Optional[Any] = ..., version: Optional[Any] = ..., preserve_python_environment: bool = ...) -> None: ... + @property + def exact(self): ... + @property + def name(self): ... + @property + def version(self): ... + def shell_commands(self, requirement): ... + class CondaDependency(Dependency): dict_collection_visible_keys = ... # type: Any dependency_type = ... # type: str + cacheable = ... # type: bool activate = ... # type: Any + conda_context = ... # type: Any environment_path = ... # type: Any - def __init__(self, activate, environment_path, exact, name: Optional[Any] = ..., version: Optional[Any] = ...) -> None: ... + cache_path = ... # type: Any + def __init__(self, conda_context, environment_path, exact, name: Optional[Any] = ..., version: Optional[Any] = ..., preserve_python_environment: bool = ...) -> None: ... @property def exact(self): ... @property def name(self): ... @property def version(self): ... + def build_cache(self, cache_path): ... + def set_cache_path(self, cache_path): ... + def build_environment(self): ... def shell_commands(self, requirement): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi index c83a8a7b1..cb4551ffe 100644 --- a/typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/galaxy_packages.pyi @@ -1,10 +1,10 @@ -# Stubs for galaxy.tools.deps.resolvers.galaxy_packages (Python 3.5) +# Stubs for galaxy.tools.deps.resolvers.galaxy_packages (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. from typing import Any from .resolver_mixins import UsesToolDependencyDirMixin -from ..resolvers import Dependency, DependencyResolver, ListableDependencyResolver +from ..resolvers import Dependency, DependencyResolver, ListableDependencyResolver, MappableDependencyResolver class GalaxyPackageDependency(Dependency): dict_collection_visible_keys = ... # type: Any @@ -26,8 +26,10 @@ class BaseGalaxyPackageDependencyResolver(DependencyResolver, UsesToolDependency dependency_type = ... # type: Any versionless = ... # type: Any def __init__(self, dependency_manager, **kwds) -> None: ... - def resolve(self, name, version, type, **kwds): ... + def resolve(self, requirement, **kwds): ... -class GalaxyPackageDependencyResolver(BaseGalaxyPackageDependencyResolver, ListableDependencyResolver): +class GalaxyPackageDependencyResolver(BaseGalaxyPackageDependencyResolver, ListableDependencyResolver, MappableDependencyResolver): resolver_type = ... # type: str + def __init__(self, dependency_manager, **kwds) -> None: ... + def resolve(self, requirement, **kwds): ... def list_dependencies(self): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/homebrew.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/homebrew.pyi new file mode 100644 index 000000000..789617317 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/homebrew.pyi @@ -0,0 +1,14 @@ +# Stubs for galaxy.tools.deps.resolvers.homebrew (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from .resolver_mixins import UsesHomebrewMixin +from ..resolvers import DependencyResolver + +class HomebrewDependencyResolver(DependencyResolver, UsesHomebrewMixin): + resolver_type = ... # type: str + versionless = ... # type: Any + prefer_version = ... # type: Any + def __init__(self, dependency_manager, **kwds) -> None: ... + def resolve(self, requirement, **kwds): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/modules.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/modules.pyi new file mode 100644 index 000000000..5762d957d --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/modules.pyi @@ -0,0 +1,42 @@ +# Stubs for galaxy.tools.deps.resolvers.modules (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from ..resolvers import Dependency, DependencyResolver, MappableDependencyResolver + +class ModuleDependencyResolver(DependencyResolver, MappableDependencyResolver): + dict_collection_visible_keys = ... # type: Any + resolver_type = ... # type: str + versionless = ... # type: Any + modulecmd = ... # type: Any + modulepath = ... # type: Any + default_indicator = ... # type: Any + module_checker = ... # type: Any + def __init__(self, dependency_manager, **kwds) -> None: ... + def resolve(self, requirement, **kwds): ... + +class DirectoryModuleChecker: + module_dependency_resolver = ... # type: Any + directories = ... # type: Any + def __init__(self, module_dependency_resolver, modulepath, prefetch) -> None: ... + def has_module(self, module, version): ... + +class AvailModuleChecker: + module_dependency_resolver = ... # type: Any + modulepath = ... # type: Any + default_indicator = ... # type: Any + prefetched_modules = ... # type: Any + def __init__(self, module_dependency_resolver, modulepath, prefetch, default_indicator: Any = ...) -> None: ... + def has_module(self, module, version): ... + +class ModuleDependency(Dependency): + dict_collection_visible_keys = ... # type: Any + dependency_type = ... # type: str + module_dependency_resolver = ... # type: Any + module_name = ... # type: Any + module_version = ... # type: Any + def __init__(self, module_dependency_resolver, module_name, module_version: Optional[Any] = ..., exact: bool = ...) -> None: ... + @property + def exact(self): ... + def shell_commands(self, requirement): ... diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi index fddfaec9a..2b90a98d5 100644 --- a/typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/resolver_mixins.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.resolvers.resolver_mixins (Python 3.5) +# Stubs for galaxy.tools.deps.resolvers.resolver_mixins (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi index 36681f606..224d377c2 100644 --- a/typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/tool_shed_packages.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.tools.deps.resolvers.tool_shed_packages (Python 3.5) +# Stubs for galaxy.tools.deps.resolvers.tool_shed_packages (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. diff --git a/typeshed/2.7/galaxy/tools/deps/resolvers/unlinked_tool_shed_packages.pyi b/typeshed/2.7/galaxy/tools/deps/resolvers/unlinked_tool_shed_packages.pyi new file mode 100644 index 000000000..ddf2ee041 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/resolvers/unlinked_tool_shed_packages.pyi @@ -0,0 +1,25 @@ +# Stubs for galaxy.tools.deps.resolvers.unlinked_tool_shed_packages (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from .galaxy_packages import BaseGalaxyPackageDependencyResolver +from ..resolvers import Dependency + +class UnlinkedToolShedPackageDependencyResolver(BaseGalaxyPackageDependencyResolver): + dict_collection_visible_keys = ... # type: Any + resolver_type = ... # type: str + preferred_owners = ... # type: Any + select_by_owner = ... # type: Any + def __init__(self, dependency_manager, **kwds) -> None: ... + +class CandidateDependency(Dependency): + dict_collection_visible_keys = ... # type: Any + dependency_type = ... # type: str + @property + def exact(self): ... + dependency = ... # type: Any + path = ... # type: Any + owner = ... # type: Any + def __init__(self, dependency, path, owner: Any = ...) -> None: ... + def shell_commands(self, requirement): ... diff --git a/typeshed/2.7/galaxy/tools/deps/singularity_util.pyi b/typeshed/2.7/galaxy/tools/deps/singularity_util.pyi new file mode 100644 index 000000000..a701d1fd8 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/singularity_util.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.tools.deps.singularity_util (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +def build_singularity_run_command(container_command, image, volumes: Any = ..., env: Any = ..., working_directory: Any = ..., singularity_cmd: Any = ..., run_extra_arguments: Any = ..., sudo: Any = ..., sudo_cmd: Any = ...): ... diff --git a/typeshed/2.7/galaxy/tools/deps/views.pyi b/typeshed/2.7/galaxy/tools/deps/views.pyi new file mode 100644 index 000000000..2f5799018 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/deps/views.pyi @@ -0,0 +1,32 @@ +# Stubs for galaxy.tools.deps.views (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +class DependencyResolversView: + def __init__(self, app) -> None: ... + def index(self): ... + def show(self, index): ... + def reload(self): ... + def manager_requirements(self): ... + def resolver_requirements(self, index): ... + def manager_dependency(self, **kwds): ... + def resolver_dependency(self, index, **kwds): ... + def show_dependencies(self, tool_requirements_d, installed_tool_dependencies: Optional[Any] = ...): ... + def uninstall_dependencies(self, index: Optional[Any] = ..., **payload): ... + @property + def unused_dependency_paths(self): ... + def remove_unused_dependency_paths(self, envs): ... + def install_dependencies(self, requirements): ... + def install_dependency(self, index: Optional[Any] = ..., **payload): ... + @property + def installable_resolvers(self): ... + @property + def uninstallable_resolvers(self): ... + @property + def tool_ids_by_requirements(self): ... + @property + def toolbox_requirements_status(self): ... + def get_requirements_status(self, tool_requirements_d, installed_tool_dependencies: Optional[Any] = ...): ... + def clean(self, index: Optional[Any] = ..., **kwds): ... diff --git a/typeshed/2.7/galaxy/tools/fetcher.pyi b/typeshed/2.7/galaxy/tools/fetcher.pyi new file mode 100644 index 000000000..221bc58a8 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/fetcher.pyi @@ -0,0 +1,10 @@ +# Stubs for galaxy.tools.fetcher (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class ToolLocationFetcher: + resolver_classes = ... # type: Any + def __init__(self) -> None: ... + def to_tool_path(self, path_or_uri_like, **kwds): ... diff --git a/typeshed/2.7/galaxy/tools/lint.pyi b/typeshed/2.7/galaxy/tools/lint.pyi new file mode 100644 index 000000000..9cd7a2631 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/lint.pyi @@ -0,0 +1,33 @@ +# Stubs for galaxy.tools.lint (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from .parser import get_tool_source as get_tool_source + +LEVEL_ALL = ... # type: str +LEVEL_WARN = ... # type: str +LEVEL_ERROR = ... # type: str + +def lint_tool_source(tool_source, level: Any = ..., fail_level: Any = ..., extra_modules: Any = ..., skip_types: Any = ...): ... +def lint_xml(tool_xml, level: Any = ..., fail_level: Any = ..., extra_modules: Any = ..., skip_types: Any = ...): ... +def lint_tool_source_with(lint_context, tool_source, extra_modules: Any = ...): ... +def lint_xml_with(lint_context, tool_xml, extra_modules: Any = ...): ... + +class LintContext: + skip_types = ... # type: Any + level = ... # type: Any + found_errors = ... # type: bool + found_warns = ... # type: bool + def __init__(self, level, skip_types: Any = ...) -> None: ... + printed_linter_info = ... # type: bool + valid_messages = ... # type: Any + info_messages = ... # type: Any + warn_messages = ... # type: Any + error_messages = ... # type: Any + def lint(self, name, lint_func, lint_target): ... + def valid(self, message, *args): ... + def info(self, message, *args): ... + def error(self, message, *args): ... + def warn(self, message, *args): ... + def failed(self, fail_level): ... diff --git a/typeshed/2.7/galaxy/tools/lint_util.pyi b/typeshed/2.7/galaxy/tools/lint_util.pyi new file mode 100644 index 000000000..e1e1ac02b --- /dev/null +++ b/typeshed/2.7/galaxy/tools/lint_util.pyi @@ -0,0 +1,5 @@ +# Stubs for galaxy.tools.lint_util (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def is_datasource(tool_xml): ... diff --git a/typeshed/2.7/galaxy/tools/linters/__init__.pyi b/typeshed/2.7/galaxy/tools/linters/__init__.pyi new file mode 100644 index 000000000..7f56312cb --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/__init__.pyi @@ -0,0 +1,4 @@ +# Stubs for galaxy.tools.linters (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + diff --git a/typeshed/2.7/galaxy/tools/linters/citations.pyi b/typeshed/2.7/galaxy/tools/linters/citations.pyi new file mode 100644 index 000000000..5378b8ded --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/citations.pyi @@ -0,0 +1,5 @@ +# Stubs for galaxy.tools.linters.citations (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def lint_citations(tool_xml, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/linters/command.pyi b/typeshed/2.7/galaxy/tools/linters/command.pyi new file mode 100644 index 000000000..2e6863537 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/command.pyi @@ -0,0 +1,6 @@ +# Stubs for galaxy.tools.linters.command (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def lint_command(tool_xml, lint_ctx): ... +def get_command(tool_xml): ... diff --git a/typeshed/2.7/galaxy/tools/linters/cwl.pyi b/typeshed/2.7/galaxy/tools/linters/cwl.pyi new file mode 100644 index 000000000..a94292e46 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/cwl.pyi @@ -0,0 +1,12 @@ +# Stubs for galaxy.tools.linters.cwl (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +lint_tool_types = ... # type: Any + +def lint_cwl_validation(tool_source, lint_ctx): ... +def lint_new_draft(tool_source, lint_ctx): ... +def lint_docker_image(tool_source, lint_ctx): ... +def lint_description(tool_source, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/linters/general.pyi b/typeshed/2.7/galaxy/tools/linters/general.pyi new file mode 100644 index 000000000..812e6e9f7 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/general.pyi @@ -0,0 +1,19 @@ +# Stubs for galaxy.tools.linters.general (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +ERROR_VERSION_MSG = ... # type: str +VALID_VERSION_MSG = ... # type: str +ERROR_NAME_MSG = ... # type: str +VALID_NAME_MSG = ... # type: str +ERROR_ID_MSG = ... # type: str +VALID_ID_MSG = ... # type: str +PROFILE_PATTERN = ... # type: Any +PROFILE_INFO_DEFAULT_MSG = ... # type: str +PROFILE_INFO_SPECIFIED_MSG = ... # type: str +PROFILE_INVALID_MSG = ... # type: str +lint_tool_types = ... # type: Any + +def lint_general(tool_source, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/linters/help.pyi b/typeshed/2.7/galaxy/tools/linters/help.pyi new file mode 100644 index 000000000..306fa3bda --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/help.pyi @@ -0,0 +1,6 @@ +# Stubs for galaxy.tools.linters.help (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def lint_help(tool_xml, lint_ctx): ... +def rst_invalid(text): ... diff --git a/typeshed/2.7/galaxy/tools/linters/inputs.pyi b/typeshed/2.7/galaxy/tools/linters/inputs.pyi new file mode 100644 index 000000000..c06f46c9e --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/inputs.pyi @@ -0,0 +1,8 @@ +# Stubs for galaxy.tools.linters.inputs (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from ..lint_util import is_datasource as is_datasource + +def lint_inputs(tool_xml, lint_ctx): ... +def lint_repeats(tool_xml, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/linters/outputs.pyi b/typeshed/2.7/galaxy/tools/linters/outputs.pyi new file mode 100644 index 000000000..ecb922098 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/outputs.pyi @@ -0,0 +1,5 @@ +# Stubs for galaxy.tools.linters.outputs (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def lint_output(tool_xml, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/linters/stdio.pyi b/typeshed/2.7/galaxy/tools/linters/stdio.pyi new file mode 100644 index 000000000..d7c2927c9 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/stdio.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.tools.linters.stdio (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from .command import get_command as get_command + +def lint_stdio(tool_source, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/linters/tests.pyi b/typeshed/2.7/galaxy/tools/linters/tests.pyi new file mode 100644 index 000000000..d9c7cf0f9 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/tests.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.tools.linters.tests (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from ..lint_util import is_datasource as is_datasource + +def lint_tsts(tool_xml, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/linters/xml_order.pyi b/typeshed/2.7/galaxy/tools/linters/xml_order.pyi new file mode 100644 index 000000000..a3904f6c2 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/linters/xml_order.pyi @@ -0,0 +1,10 @@ +# Stubs for galaxy.tools.linters.xml_order (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +TAG_ORDER = ... # type: Any +DATASOURCE_TAG_ORDER = ... # type: Any + +def lint_xml_order(tool_xml, lint_ctx): ... diff --git a/typeshed/2.7/galaxy/tools/loader.pyi b/typeshed/2.7/galaxy/tools/loader.pyi new file mode 100644 index 000000000..4e7cd7fdd --- /dev/null +++ b/typeshed/2.7/galaxy/tools/loader.pyi @@ -0,0 +1,8 @@ +# Stubs for galaxy.tools.loader (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from galaxy.util.xml_macros import imported_macro_paths as imported_macro_paths, raw_tool_xml_tree as raw_tool_xml_tree, template_macro_params as template_macro_params + +load_tool = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/loader_directory.pyi b/typeshed/2.7/galaxy/tools/loader_directory.pyi new file mode 100644 index 000000000..628289f72 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/loader_directory.pyi @@ -0,0 +1,15 @@ +# Stubs for galaxy.tools.loader_directory (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def find_possible_tools_from_path(path, recursive: bool = ..., enable_beta_formats: bool = ...): ... +def load_tool_sources_from_path(path, load_exception_handler: Any = ..., recursive: bool = ..., register_load_errors: bool = ...): ... +def load_tool_elements_from_path(path, load_exception_handler: Any = ..., recursive: bool = ..., register_load_errors: bool = ...): ... +def is_tool_load_error(obj): ... +def looks_like_a_tool_xml(path): ... +def is_a_yaml_with_class(path, classes): ... +def looks_like_a_tool_yaml(path): ... +def looks_like_a_cwl_artifact(path, classes: Optional[Any] = ...): ... +def looks_like_a_tool_cwl(path): ... diff --git a/typeshed/2.7/galaxy/tools/locations/__init__.pyi b/typeshed/2.7/galaxy/tools/locations/__init__.pyi new file mode 100644 index 000000000..af99abd8b --- /dev/null +++ b/typeshed/2.7/galaxy/tools/locations/__init__.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.tools.locations (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +class ToolLocationResolver: + def scheme(self): ... + def get_tool_source_path(self, uri_like): ... diff --git a/typeshed/2.7/galaxy/tools/locations/dockstore.pyi b/typeshed/2.7/galaxy/tools/locations/dockstore.pyi new file mode 100644 index 000000000..3420e1eba --- /dev/null +++ b/typeshed/2.7/galaxy/tools/locations/dockstore.pyi @@ -0,0 +1,19 @@ +# Stubs for galaxy.tools.locations.dockstore (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from ..locations import ToolLocationResolver + +class DockStoreResolver(ToolLocationResolver): + scheme = ... # type: str + def get_tool_source_path(self, uri_like): ... + +class _Ga4ghToolClient: + base_url = ... # type: Any + def __init__(self, base_url: str = ...) -> None: ... + def get_tools(self): ... + def get_tool(self, tool_id): ... + def get_tool_version(self, tool_id, version: str = ...): ... + def get_tool_descriptor(self, tool_id, version: str = ..., tool_type: str = ...): ... + def get_tool_cwl(self, tool_id, version: str = ..., as_string: bool = ...): ... diff --git a/typeshed/2.7/galaxy/tools/locations/file.pyi b/typeshed/2.7/galaxy/tools/locations/file.pyi new file mode 100644 index 000000000..8268d650f --- /dev/null +++ b/typeshed/2.7/galaxy/tools/locations/file.pyi @@ -0,0 +1,9 @@ +# Stubs for galaxy.tools.locations.file (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from ..locations import ToolLocationResolver as ToolLocationResolver + +class HttpToolResolver(ToolLocationResolver): + scheme = ... # type: str + def get_tool_source_path(self, uri_like): ... diff --git a/typeshed/2.7/galaxy/tools/locations/http.pyi b/typeshed/2.7/galaxy/tools/locations/http.pyi new file mode 100644 index 000000000..2687afcef --- /dev/null +++ b/typeshed/2.7/galaxy/tools/locations/http.pyi @@ -0,0 +1,13 @@ +# Stubs for galaxy.tools.locations.http (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from ..locations import ToolLocationResolver + +class HttpToolResolver(ToolLocationResolver): + scheme = ... # type: str + def __init__(self, **kwds) -> None: ... + def get_tool_source_path(self, uri_like): ... + +class HttpsToolResolver(HttpToolResolver): + scheme = ... # type: str diff --git a/typeshed/2.7/galaxy/tools/parser/__init__.pyi b/typeshed/2.7/galaxy/tools/parser/__init__.pyi new file mode 100644 index 000000000..0b1bc8142 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/__init__.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.tools.parser (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from .factory import get_input_source as get_input_source, get_tool_source as get_tool_source +from .interface import ToolSource as ToolSource +from .output_objects import ToolOutputCollectionPart as ToolOutputCollectionPart diff --git a/typeshed/2.7/galaxy/tools/parser/cwl.pyi b/typeshed/2.7/galaxy/tools/parser/cwl.pyi new file mode 100644 index 000000000..1b754a91b --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/cwl.pyi @@ -0,0 +1,41 @@ +# Stubs for galaxy.tools.parser.cwl (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from .interface import PageSource as PageSource +from .interface import PagesSource as PagesSource +from .interface import ToolSource as ToolSource +from .interface import ToolStdioExitCode as ToolStdioExitCode +from .output_actions import ToolOutputActionGroup as ToolOutputActionGroup +from .output_objects import ToolOutput as ToolOutput +from .yaml import YamlInputSource as YamlInputSource + +log = ... # type: Any + +class CwlToolSource(ToolSource): + def __init__(self, tool_file, strict_cwl_validation: bool = ...) -> None: ... + @property + def tool_proxy(self): ... + def parse_tool_type(self): ... + def parse_id(self): ... + def parse_name(self): ... + def parse_command(self): ... + def parse_environment_variables(self): ... + def parse_edam_operations(self): ... + def parse_edam_topics(self): ... + def parse_help(self): ... + def parse_sanitize(self): ... + def parse_strict_shell(self): ... + def parse_stdio(self): ... + def parse_interpreter(self): ... + def parse_version(self): ... + def parse_description(self): ... + def parse_input_pages(self): ... + def parse_outputs(self, tool): ... + def parse_requirements_and_containers(self): ... + def parse_profile(self): ... + +class CwlPageSource(PageSource): + def __init__(self, tool_proxy) -> None: ... + def parse_input_sources(self): ... diff --git a/typeshed/2.7/galaxy/tools/parser/factory.pyi b/typeshed/2.7/galaxy/tools/parser/factory.pyi new file mode 100644 index 000000000..c6b63d099 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/factory.pyi @@ -0,0 +1,9 @@ +# Stubs for galaxy.tools.parser.factory (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from galaxy.tools.loader import load_tool as load_tool_xml + +def get_tool_source(config_file: Optional[Any] = ..., xml_tree: Optional[Any] = ..., enable_beta_formats: bool = ..., tool_location_fetcher: Optional[Any] = ...): ... +def get_input_source(content): ... diff --git a/typeshed/2.7/galaxy/tools/parser/interface.pyi b/typeshed/2.7/galaxy/tools/parser/interface.pyi new file mode 100644 index 000000000..e219c74a3 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/interface.pyi @@ -0,0 +1,96 @@ +# Stubs for galaxy.tools.parser.interface (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +NOT_IMPLEMENTED_MESSAGE = ... # type: str + +class ToolSource: + default_is_multi_byte = ... # type: bool + def parse_id(self): ... + def parse_version(self): ... + def parse_tool_module(self): ... + def parse_action_module(self): ... + def parse_tool_type(self): ... + def parse_name(self): ... + def parse_description(self): ... + def parse_is_multi_byte(self): ... + def parse_display_interface(self, default): ... + def parse_require_login(self, default): ... + def parse_request_param_translation_elem(self): ... + def parse_command(self): ... + def parse_environment_variables(self): ... + def parse_interpreter(self): ... + def parse_redirect_url_params_elem(self): ... + def parse_version_command(self): ... + def parse_version_command_interpreter(self): ... + def parse_parallelism(self): ... + def parse_hidden(self): ... + def parse_sanitize(self): ... + def parse_refresh(self): ... + def parse_requirements_and_containers(self): ... + def parse_input_pages(self): ... + def parse_outputs(self, tool): ... + def parse_strict_shell(self): ... + def parse_stdio(self): ... + def parse_help(self): ... + def parse_profile(self): ... + def parse_tests_to_dict(self): ... + +class PagesSource: + page_sources = ... # type: Any + def __init__(self, page_sources) -> None: ... + @property + def inputs_defined(self): ... + +class PageSource: + def parse_display(self): ... + def parse_input_sources(self): ... + +class InputSource: + default_optional = ... # type: bool + def elem(self): ... + def get(self, key, value: Optional[Any] = ...): ... + def get_bool(self, key, default): ... + def parse_label(self): ... + def parse_help(self): ... + def parse_sanitizer_elem(self): ... + def parse_validator_elems(self): ... + def parse_optional(self, default: Optional[Any] = ...): ... + def parse_dynamic_options_elem(self): ... + def parse_static_options(self): ... + def parse_conversion_tuples(self): ... + def parse_nested_inputs_source(self): ... + def parse_test_input_source(self): ... + def parse_when_input_sources(self): ... + +class ToolStdioRegex: + match = ... # type: str + stdout_match = ... # type: bool + stderr_match = ... # type: bool + error_level = ... # type: str + desc = ... # type: str + def __init__(self) -> None: ... + +class ToolStdioExitCode: + range_start = ... # type: Any + range_end = ... # type: Any + error_level = ... # type: str + desc = ... # type: str + def __init__(self) -> None: ... + +class TestCollectionDef: + elements = ... # type: Any + collection_type = ... # type: Any + name = ... # type: Any + def __init__(self, elem, parse_param_elem) -> None: ... + def collect_inputs(self): ... + +class TestCollectionOutputDef: + name = ... # type: Any + collection_type = ... # type: Any + count = ... # type: Any + attrib = ... # type: Any + element_tests = ... # type: Any + def __init__(self, name, attrib, element_tests) -> None: ... diff --git a/typeshed/2.7/galaxy/tools/parser/output_actions.pyi b/typeshed/2.7/galaxy/tools/parser/output_actions.pyi new file mode 100644 index 000000000..5945acdcd --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/output_actions.pyi @@ -0,0 +1,215 @@ +# Stubs for galaxy.tools.parser.output_actions (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +log = ... # type: Any +COLLECTION_ATTRIBUTES = ... # type: Any + +class ToolOutputActionGroup: + tag = ... # type: str + parent = ... # type: Any + actions = ... # type: Any + def __init__(self, parent, config_elem) -> None: ... + def apply_action(self, output_dataset, other_values): ... + @property + def tool(self): ... + def __len__(self): ... + +class ToolOutputActionConditionalWhen(ToolOutputActionGroup): + tag = ... # type: str + @classmethod + def from_elem(cls, parent, when_elem): ... + value = ... # type: Any + def __init__(self, parent, config_elem, value) -> None: ... + def is_case(self, output_dataset, other_values): ... + def get_ref(self, output_dataset, other_values): ... + def apply_action(self, output_dataset, other_values): ... + +class ValueToolOutputActionConditionalWhen(ToolOutputActionConditionalWhen): + tag = ... # type: str + def is_case(self, output_dataset, other_values): ... + +class DatatypeIsInstanceToolOutputActionConditionalWhen(ToolOutputActionConditionalWhen): + tag = ... # type: str + value = ... # type: Any + def __init__(self, parent, config_elem, value) -> None: ... + def is_case(self, output_dataset, other_values): ... + +class ToolOutputActionConditional: + tag = ... # type: str + parent = ... # type: Any + name = ... # type: Any + cases = ... # type: Any + def __init__(self, parent, config_elem) -> None: ... + def apply_action(self, output_dataset, other_values): ... + @property + def tool(self): ... + +class ToolOutputAction: + tag = ... # type: str + @classmethod + def from_elem(cls, parent, elem): ... + parent = ... # type: Any + default = ... # type: Any + option = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def apply_action(self, output_dataset, other_values): ... + @property + def tool(self): ... + +class ToolOutputActionOption: + tag = ... # type: str + @classmethod + def from_elem(cls, parent, elem): ... + parent = ... # type: Any + filters = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def get_value(self, other_values): ... + @property + def tool(self): ... + +class NullToolOutputActionOption(ToolOutputActionOption): + tag = ... # type: str + def get_value(self, other_values): ... + +class FromFileToolOutputActionOption(ToolOutputActionOption): + tag = ... # type: str + name = ... # type: Any + column = ... # type: Any + offset = ... # type: Any + separator = ... # type: Any + options = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def get_value(self, other_values): ... + +class FromParamToolOutputActionOption(ToolOutputActionOption): + tag = ... # type: str + name = ... # type: Any + column = ... # type: Any + offset = ... # type: Any + param_attribute = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def get_value(self, other_values): ... + +class FromDataTableOutputActionOption(ToolOutputActionOption): + tag = ... # type: str + name = ... # type: Any + missing_tool_data_table_name = ... # type: Any + options = ... # type: Any + column = ... # type: Any + offset = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def get_value(self, other_values): ... + +class MetadataToolOutputAction(ToolOutputAction): + tag = ... # type: str + name = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def apply_action(self, output_dataset, other_values): ... + +class FormatToolOutputAction(ToolOutputAction): + tag = ... # type: str + default = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def apply_action(self, output_dataset, other_values): ... + +class ToolOutputActionOptionFilter: + tag = ... # type: str + @classmethod + def from_elem(cls, parent, elem): ... + parent = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + @property + def tool(self): ... + +class ParamValueToolOutputActionOptionFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + ref = ... # type: Any + value = ... # type: Any + column = ... # type: Any + keep = ... # type: Any + compare = ... # type: Any + cast = ... # type: Any + param_attribute = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +class InsertColumnToolOutputActionOptionFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + ref = ... # type: Any + value = ... # type: Any + column = ... # type: Any + iterate = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +class MultipleSplitterFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + column = ... # type: Any + separator = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +class ColumnStripFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + column = ... # type: Any + strip = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +class ColumnReplaceFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + old_column = ... # type: Any + old_value = ... # type: Any + new_value = ... # type: Any + new_column = ... # type: Any + column = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +class MetadataValueFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + ref = ... # type: Any + name = ... # type: Any + column = ... # type: Any + keep = ... # type: Any + compare = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +class BooleanFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + column = ... # type: Any + keep = ... # type: Any + cast = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +class StringFunctionFilter(ToolOutputActionOptionFilter): + tag = ... # type: str + column = ... # type: Any + function = ... # type: Any + def __init__(self, parent, elem) -> None: ... + def filter_options(self, options, other_values): ... + +action_types = ... # type: Any +option_types = ... # type: Any +filter_types = ... # type: Any + +def parse_cast_attribute(cast): ... +def parse_compare_type(compare): ... +def compare_eq(value1, value2): ... +def compare_neq(value1, value2): ... +def compare_gt(value1, value2): ... +def compare_gte(value1, value2): ... +def compare_lt(value1, value2): ... +def compare_lte(value1, value2): ... +def compare_in(value1, value2): ... +def compare_startswith(value1, value2): ... +def compare_endswith(value1, value2): ... +def compare_re_search(value1, value2): ... + +compare_types = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/parser/output_collection_def.pyi b/typeshed/2.7/galaxy/tools/parser/output_collection_def.pyi new file mode 100644 index 000000000..2cc4b1bd7 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/output_collection_def.pyi @@ -0,0 +1,29 @@ +# Stubs for galaxy.tools.parser.output_collection_def (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +DEFAULT_EXTRA_FILENAME_PATTERN = ... # type: str +DEFAULT_SORT_BY = ... # type: str +DEFAULT_SORT_COMP = ... # type: str +NAMED_PATTERNS = ... # type: Any +INPUT_DBKEY_TOKEN = ... # type: str +LEGACY_DEFAULT_DBKEY = ... # type: Any + +def dataset_collector_descriptions_from_elem(elem, legacy: bool = ...): ... +def dataset_collector_descriptions_from_list(discover_datasets_dicts): ... + +class DatasetCollectionDescription: + pattern = ... # type: Any + default_dbkey = ... # type: Any + default_ext = ... # type: Any + default_visible = ... # type: Any + directory = ... # type: Any + assign_primary_output = ... # type: Any + sort_reverse = ... # type: bool + sort_key = ... # type: Any + sort_comp = ... # type: Any + def __init__(self, **kwargs) -> None: ... + +DEFAULT_DATASET_COLLECTOR_DESCRIPTION = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/parser/output_objects.pyi b/typeshed/2.7/galaxy/tools/parser/output_objects.pyi new file mode 100644 index 000000000..68f824eaf --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/output_objects.pyi @@ -0,0 +1,68 @@ +# Stubs for galaxy.tools.parser.output_objects (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from galaxy.util.dictifiable import Dictifiable + +class ToolOutputBase(Dictifiable): + name = ... # type: Any + label = ... # type: Any + filters = ... # type: Any + hidden = ... # type: Any + collection = ... # type: bool + def __init__(self, name, label: Optional[Any] = ..., filters: Optional[Any] = ..., hidden: bool = ...) -> None: ... + +class ToolOutput(ToolOutputBase): + dict_collection_visible_keys = ... # type: Any + format = ... # type: Any + format_source = ... # type: Any + metadata_source = ... # type: Any + parent = ... # type: Any + actions = ... # type: Any + change_format = ... # type: Any + implicit = ... # type: Any + from_work_dir = ... # type: Any + def __init__(self, name, format: Optional[Any] = ..., format_source: Optional[Any] = ..., metadata_source: Optional[Any] = ..., parent: Optional[Any] = ..., label: Optional[Any] = ..., filters: Optional[Any] = ..., actions: Optional[Any] = ..., hidden: bool = ..., implicit: bool = ...) -> None: ... + def __len__(self): ... + def __getitem__(self, index): ... + def __iter__(self): ... + def to_dict(self, view: str = ..., value_mapper: Optional[Any] = ..., app: Optional[Any] = ...): ... + +class ToolOutputCollection(ToolOutputBase): + collection = ... # type: bool + default_format = ... # type: Any + structure = ... # type: Any + outputs = ... # type: Any + inherit_format = ... # type: Any + inherit_metadata = ... # type: Any + metadata_source = ... # type: Any + format_source = ... # type: Any + change_format = ... # type: Any + def __init__(self, name, structure, label: Optional[Any] = ..., filters: Optional[Any] = ..., hidden: bool = ..., default_format: str = ..., default_format_source: Optional[Any] = ..., default_metadata_source: Optional[Any] = ..., inherit_format: bool = ..., inherit_metadata: bool = ...) -> None: ... + def known_outputs(self, inputs, type_registry): ... + @property + def dynamic_structure(self): ... + @property + def dataset_collector_descriptions(self): ... + +class ToolOutputCollectionStructure: + collection_type = ... # type: Any + collection_type_source = ... # type: Any + structured_like = ... # type: Any + dataset_collector_descriptions = ... # type: Any + dynamic = ... # type: Any + def __init__(self, collection_type, collection_type_source, structured_like, dataset_collector_descriptions) -> None: ... + +class ToolOutputCollectionPart: + output_collection_def = ... # type: Any + element_identifier = ... # type: Any + output_def = ... # type: Any + parent_ids = ... # type: Any + def __init__(self, output_collection_def, element_identifier, output_def, parent_ids: Any = ...) -> None: ... + @property + def effective_output_name(self): ... + @staticmethod + def is_named_collection_part_name(name): ... + @staticmethod + def split_output_name(name): ... diff --git a/typeshed/2.7/galaxy/tools/parser/util.pyi b/typeshed/2.7/galaxy/tools/parser/util.pyi new file mode 100644 index 000000000..97b33962c --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/util.pyi @@ -0,0 +1,9 @@ +# Stubs for galaxy.tools.parser.util (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from .interface import ToolStdioExitCode as ToolStdioExitCode +from .interface import ToolStdioRegex as ToolStdioRegex + +def error_on_exit_code(): ... +def aggressive_error_checks(): ... diff --git a/typeshed/2.7/galaxy/tools/parser/xml.pyi b/typeshed/2.7/galaxy/tools/parser/xml.pyi new file mode 100644 index 000000000..ffdd7e92f --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/xml.pyi @@ -0,0 +1,89 @@ +# Stubs for galaxy.tools.parser.xml (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from .interface import InputSource as InputSource, PageSource as PageSource, PagesSource as PagesSource, TestCollectionDef as TestCollectionDef, TestCollectionOutputDef as TestCollectionOutputDef, ToolSource as ToolSource, ToolStdioExitCode as ToolStdioExitCode, ToolStdioRegex as ToolStdioRegex +from .output_actions import ToolOutputActionGroup as ToolOutputActionGroup +from .output_collection_def import dataset_collector_descriptions_from_elem as dataset_collector_descriptions_from_elem +from .output_objects import ToolOutput as ToolOutput, ToolOutputCollection as ToolOutputCollection, ToolOutputCollectionStructure as ToolOutputCollectionStructure +from .util import aggressive_error_checks as aggressive_error_checks, error_on_exit_code as error_on_exit_code + +log = ... # type: Any + +class XmlToolSource(ToolSource): + xml_tree = ... # type: Any + root = ... # type: Any + legacy_defaults = ... # type: Any + def __init__(self, xml_tree, source_path: Optional[Any] = ...) -> None: ... + def parse_version(self): ... + def parse_id(self): ... + def parse_tool_module(self): ... + def parse_action_module(self): ... + def parse_tool_type(self): ... + def parse_name(self): ... + def parse_edam_operations(self): ... + def parse_edam_topics(self): ... + def parse_description(self): ... + def parse_is_multi_byte(self): ... + def parse_display_interface(self, default): ... + def parse_require_login(self, default): ... + def parse_request_param_translation_elem(self): ... + def parse_command(self): ... + def parse_environment_variables(self): ... + def parse_interpreter(self): ... + def parse_version_command(self): ... + def parse_version_command_interpreter(self): ... + def parse_parallelism(self): ... + def parse_hidden(self): ... + def parse_redirect_url_params_elem(self): ... + def parse_sanitize(self): ... + def parse_refresh(self): ... + def parse_requirements_and_containers(self): ... + def parse_input_pages(self): ... + def parse_outputs(self, tool): ... + def parse_stdio(self): ... + def parse_strict_shell(self): ... + def parse_help(self): ... + def parse_tests_to_dict(self): ... + def parse_profile(self): ... + +class StdioParser: + stdio_exit_codes = ... # type: Any + stdio_regexes = ... # type: Any + def __init__(self, root) -> None: ... + def parse_stdio_exit_codes(self, stdio_elem): ... + def parse_stdio_regexes(self, stdio_elem): ... + def parse_error_level(self, err_level): ... + +class XmlPagesSource(PagesSource): + input_elem = ... # type: Any + def __init__(self, root) -> None: ... + @property + def inputs_defined(self): ... + +class XmlPageSource(PageSource): + parent_elem = ... # type: Any + def __init__(self, parent_elem) -> None: ... + def parse_display(self): ... + def parse_input_sources(self): ... + +class XmlInputSource(InputSource): + input_elem = ... # type: Any + input_type = ... # type: Any + def __init__(self, input_elem) -> None: ... + def parse_input_type(self): ... + def elem(self): ... + def get(self, key, value: Optional[Any] = ...): ... + def get_bool(self, key, default): ... + def parse_label(self): ... + def parse_help(self): ... + def parse_sanitizer_elem(self): ... + def parse_validator_elems(self): ... + def parse_dynamic_options_elem(self): ... + def parse_static_options(self): ... + def parse_optional(self, default: Optional[Any] = ...): ... + def parse_conversion_tuples(self): ... + def parse_nested_inputs_source(self): ... + def parse_test_input_source(self): ... + def parse_when_input_sources(self): ... diff --git a/typeshed/2.7/galaxy/tools/parser/yaml.pyi b/typeshed/2.7/galaxy/tools/parser/yaml.pyi new file mode 100644 index 000000000..ccf2afdd7 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/parser/yaml.pyi @@ -0,0 +1,56 @@ +# Stubs for galaxy.tools.parser.yaml (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from .interface import InputSource as InputSource +from .interface import PageSource as PageSource +from .interface import PagesSource as PagesSource +from .interface import ToolSource as ToolSource +from .output_actions import ToolOutputActionGroup as ToolOutputActionGroup +from .output_collection_def import dataset_collector_descriptions_from_list as dataset_collector_descriptions_from_list +from .output_objects import ToolOutput as ToolOutput, ToolOutputCollection as ToolOutputCollection, ToolOutputCollectionStructure as ToolOutputCollectionStructure +from .util import error_on_exit_code as error_on_exit_code + +class YamlToolSource(ToolSource): + root_dict = ... # type: Any + def __init__(self, root_dict, source_path: Optional[Any] = ...) -> None: ... + def parse_id(self): ... + def parse_version(self): ... + def parse_name(self): ... + def parse_description(self): ... + def parse_edam_operations(self): ... + def parse_edam_topics(self): ... + def parse_is_multi_byte(self): ... + def parse_sanitize(self): ... + def parse_display_interface(self, default): ... + def parse_require_login(self, default): ... + def parse_command(self): ... + def parse_environment_variables(self): ... + def parse_interpreter(self): ... + def parse_version_command(self): ... + def parse_version_command_interpreter(self): ... + def parse_requirements_and_containers(self): ... + def parse_input_pages(self): ... + def parse_strict_shell(self): ... + def parse_stdio(self): ... + def parse_help(self): ... + def parse_outputs(self, tool): ... + def parse_tests_to_dict(self): ... + def parse_profile(self): ... + +class YamlPageSource(PageSource): + inputs_list = ... # type: Any + def __init__(self, inputs_list) -> None: ... + def parse_input_sources(self): ... + +class YamlInputSource(InputSource): + input_dict = ... # type: Any + def __init__(self, input_dict) -> None: ... + def get(self, key, default: Optional[Any] = ...): ... + def get_bool(self, key, default): ... + def parse_input_type(self): ... + def parse_nested_inputs_source(self): ... + def parse_test_input_source(self): ... + def parse_when_input_sources(self): ... + def parse_static_options(self): ... diff --git a/typeshed/2.7/galaxy/tools/verify/__init__.pyi b/typeshed/2.7/galaxy/tools/verify/__init__.pyi new file mode 100644 index 000000000..624255387 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/verify/__init__.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.tools.verify (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from .asserts import verify_assertions as verify_assertions +from .test_data import TestDataResolver as TestDataResolver + +log = ... # type: Any +DEFAULT_TEST_DATA_RESOLVER = ... # type: Any + +def verify(item_label, output_content, attributes, filename: Optional[Any] = ..., get_filename: Optional[Any] = ..., keep_outputs_dir: Optional[Any] = ..., verify_extra_files: Optional[Any] = ...): ... +def make_temp_fname(fname: Optional[Any] = ...): ... +def check_command(command, description): ... +def files_diff(file1, file2, attributes: Optional[Any] = ...): ... +def files_re_match(file1, file2, attributes: Optional[Any] = ...): ... +def files_re_match_multiline(file1, file2, attributes: Optional[Any] = ...): ... +def files_contains(file1, file2, attributes: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/tools/verify/asserts/__init__.pyi b/typeshed/2.7/galaxy/tools/verify/asserts/__init__.pyi new file mode 100644 index 000000000..3ba17d3da --- /dev/null +++ b/typeshed/2.7/galaxy/tools/verify/asserts/__init__.pyi @@ -0,0 +1,14 @@ +# Stubs for galaxy.tools.verify.asserts (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +log = ... # type: Any +assertion_module_names = ... # type: Any +assertion_modules = ... # type: Any +full_assertion_module_name = ... # type: Any +assertion_module = ... # type: Any + +def verify_assertions(data, assertion_description_list): ... +def verify_assertion(data, assertion_description): ... diff --git a/typeshed/2.7/galaxy/tools/verify/asserts/tabular.pyi b/typeshed/2.7/galaxy/tools/verify/asserts/tabular.pyi new file mode 100644 index 000000000..4515eab5f --- /dev/null +++ b/typeshed/2.7/galaxy/tools/verify/asserts/tabular.pyi @@ -0,0 +1,6 @@ +# Stubs for galaxy.tools.verify.asserts.tabular (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def get_first_line(output): ... +def assert_has_n_columns(output, n, sep: str = ...): ... diff --git a/typeshed/2.7/galaxy/tools/verify/asserts/text.pyi b/typeshed/2.7/galaxy/tools/verify/asserts/text.pyi new file mode 100644 index 000000000..549d53481 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/verify/asserts/text.pyi @@ -0,0 +1,9 @@ +# Stubs for galaxy.tools.verify.asserts.text (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def assert_has_text(output, text): ... +def assert_not_has_text(output, text): ... +def assert_has_line(output, line): ... +def assert_has_text_matching(output, expression): ... +def assert_has_line_matching(output, expression): ... diff --git a/typeshed/2.7/galaxy/tools/verify/asserts/xml.pyi b/typeshed/2.7/galaxy/tools/verify/asserts/xml.pyi new file mode 100644 index 000000000..57af9c728 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/verify/asserts/xml.pyi @@ -0,0 +1,15 @@ +# Stubs for galaxy.tools.verify.asserts.xml (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def to_xml(output): ... +def xml_find_text(output, path): ... +def xml_find(output, path): ... +def assert_is_valid_xml(output): ... +def assert_has_element_with_path(output, path): ... +def assert_has_n_elements_with_path(output, path, n): ... +def assert_element_text_matches(output, path, expression): ... +def assert_element_text_is(output, path, text): ... +def assert_attribute_matches(output, path, attribute, expression): ... +def assert_attribute_is(output, path, attribute, text): ... +def assert_element_text(output, path, verify_assertions_function, children): ... diff --git a/typeshed/2.7/galaxy/tools/verify/test_data.pyi b/typeshed/2.7/galaxy/tools/verify/test_data.pyi new file mode 100644 index 000000000..1fe6f0d38 --- /dev/null +++ b/typeshed/2.7/galaxy/tools/verify/test_data.pyi @@ -0,0 +1,31 @@ +# Stubs for galaxy.tools.verify.test_data (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +UPDATE_TEMPLATE = ... # type: Any +UPDATE_FAILED_TEMPLATE = ... # type: Any +LIST_SEP = ... # type: Any + +class TestDataResolver: + resolvers = ... # type: Any + def __init__(self, env_var: str = ..., environ: Any = ...) -> None: ... + def get_filename(self, name): ... + +def build_resolver(uri, environ): ... + +class FileDataResolver: + file_dir = ... # type: Any + def __init__(self, file_dir) -> None: ... + def exists(self, filename): ... + def path(self, filename): ... + +class GitDataResolver(FileDataResolver): + repository = ... # type: Any + updated = ... # type: bool + fetch_data = ... # type: Any + def __init__(self, repository, environ) -> None: ... + def exists(self, filename): ... + def update_repository(self): ... + def execute(self, cmd): ... diff --git a/typeshed/2.7/galaxy/util/__init__.pyi b/typeshed/2.7/galaxy/util/__init__.pyi index d0c7e7a90..d2c9a1ee5 100644 --- a/typeshed/2.7/galaxy/util/__init__.pyi +++ b/typeshed/2.7/galaxy/util/__init__.pyi @@ -1,19 +1,16 @@ -# Stubs for galaxy.util (Python 3.5) +# Stubs for galaxy.util (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. from typing import Any, Optional import collections -from six.moves.urllib import parse as urlparse -from six.moves.urllib import request as urlrequest +from six.moves.urllib import parse as urlparse, request as urlrequest from .inflection import English as English, Inflector as Inflector grp = ... # type: Any docutils_core = ... # type: Any docutils_html4css1 = ... # type: Any inflector = ... # type: Any - -list_map = ... # type: Any log = ... # type: Any CHUNK_SIZE = ... # type: int DATABASE_MAX_STRING_SIZE = ... # type: int @@ -124,8 +121,11 @@ def parse_int(value, min_val: Optional[Any] = ..., max_val: Optional[Any] = ..., def parse_non_hex_float(s): ... def build_url(base_url, port: int = ..., scheme: str = ..., pathspec: Optional[Any] = ..., params: Optional[Any] = ..., doseq: bool = ...): ... def url_get(base_url, password_mgr: Optional[Any] = ..., pathspec: Optional[Any] = ..., params: Optional[Any] = ...): ... +def download_to_file(url, dest_file_path, timeout: int = ..., chunk_size: Any = ...): ... def safe_relpath(path): ... class ExecutionTimer: begin = ... # type: Any def __init__(self) -> None: ... + @property + def elapsed(self): ... diff --git a/typeshed/2.7/galaxy/util/aliaspickler.pyi b/typeshed/2.7/galaxy/util/aliaspickler.pyi new file mode 100644 index 000000000..95c60a9b1 --- /dev/null +++ b/typeshed/2.7/galaxy/util/aliaspickler.pyi @@ -0,0 +1,20 @@ +# Stubs for galaxy.util.aliaspickler (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import pickle +from six.moves import cStringIO as StringIO + +class AliasUnpickler(pickle.Unpickler): + aliases = ... # type: Any + def __init__(self, aliases, *args, **kw) -> None: ... + def find_class(self, module, name): ... + +class AliasPickleModule: + aliases = ... # type: Any + def __init__(self, aliases) -> None: ... + def dump(self, obj, fileobj, protocol: int = ...): ... + def dumps(self, obj, protocol: int = ...): ... + def load(self, fileobj): ... + def loads(self, string): ... diff --git a/typeshed/2.7/galaxy/util/bunch.pyi b/typeshed/2.7/galaxy/util/bunch.pyi new file mode 100644 index 000000000..87dc09bd0 --- /dev/null +++ b/typeshed/2.7/galaxy/util/bunch.pyi @@ -0,0 +1,17 @@ +# Stubs for galaxy.util.bunch (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +class Bunch: + def __init__(self, **kwds) -> None: ... + def dict(self): ... + def get(self, key, default: Optional[Any] = ...): ... + def __iter__(self): ... + def items(self): ... + def keys(self): ... + def values(self): ... + def __nonzero__(self): ... + def __setitem__(self, k, v): ... + def __contains__(self, item): ... diff --git a/typeshed/2.7/galaxy/util/checkers.pyi b/typeshed/2.7/galaxy/util/checkers.pyi new file mode 100644 index 000000000..cdcb3efb6 --- /dev/null +++ b/typeshed/2.7/galaxy/util/checkers.pyi @@ -0,0 +1,14 @@ +# Stubs for galaxy.util.checkers (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def check_html(file_path, chunk: Optional[Any] = ...): ... +def check_binary(name, file_path: bool = ...): ... +def check_gzip(file_path): ... +def check_bz2(file_path): ... +def check_zip(file_path): ... +def is_bz2(file_path): ... +def is_gzip(file_path): ... +def check_image(file_path): ... diff --git a/typeshed/2.7/galaxy/util/compression_utils.pyi b/typeshed/2.7/galaxy/util/compression_utils.pyi new file mode 100644 index 000000000..0c6ca3a79 --- /dev/null +++ b/typeshed/2.7/galaxy/util/compression_utils.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.util.compression_utils (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from .checkers import is_bz2 as is_bz2, is_gzip as is_gzip + +def get_fileobj(filename, mode: str = ..., gzip_only: bool = ..., bz2_only: bool = ..., zip_only: bool = ...): ... diff --git a/typeshed/2.7/galaxy/util/dictifiable.pyi b/typeshed/2.7/galaxy/util/dictifiable.pyi index eaec1d90d..babdeb4d3 100644 --- a/typeshed/2.7/galaxy/util/dictifiable.pyi +++ b/typeshed/2.7/galaxy/util/dictifiable.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.util.dictifiable (Python 3.5) +# Stubs for galaxy.util.dictifiable (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. diff --git a/typeshed/2.7/galaxy/util/expressions.pyi b/typeshed/2.7/galaxy/util/expressions.pyi new file mode 100644 index 000000000..f1eca5ee8 --- /dev/null +++ b/typeshed/2.7/galaxy/util/expressions.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.util.expressions (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from collections import MutableMapping + +class ExpressionContext(MutableMapping): + dict = ... # type: Any + parent = ... # type: Any + def __init__(self, dict, parent: Optional[Any] = ...) -> None: ... + def __delitem__(self, key): ... + def __iter__(self): ... + def __len__(self): ... + def __getitem__(self, key): ... + def __setitem__(self, key, value): ... + def __contains__(self, key): ... + def __nonzero__(self): ... diff --git a/typeshed/2.7/galaxy/util/filelock.pyi b/typeshed/2.7/galaxy/util/filelock.pyi new file mode 100644 index 000000000..f74dd9a4c --- /dev/null +++ b/typeshed/2.7/galaxy/util/filelock.pyi @@ -0,0 +1,21 @@ +# Stubs for galaxy.util.filelock (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class FileLockException(Exception): ... + +class FileLock: + is_locked = ... # type: bool + lockfile = ... # type: Any + file_name = ... # type: Any + timeout = ... # type: Any + delay = ... # type: Any + def __init__(self, file_name, timeout: int = ..., delay: float = ...) -> None: ... + fd = ... # type: Any + def acquire(self): ... + def release(self): ... + def __enter__(self): ... + def __exit__(self, type, value, traceback): ... + def __del__(self): ... diff --git a/typeshed/2.7/galaxy/util/hash_util.pyi b/typeshed/2.7/galaxy/util/hash_util.pyi new file mode 100644 index 000000000..1a81a16aa --- /dev/null +++ b/typeshed/2.7/galaxy/util/hash_util.pyi @@ -0,0 +1,14 @@ +# Stubs for galaxy.util.hash_util (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +import hashlib as hashlib + +sha1 = ... # type: Any +sha = ... # type: Any +md5 = ... # type: Any + +def new_secure_hash(text_type: Optional[Any] = ...): ... +def hmac_new(key, value): ... +def is_hashable(value): ... diff --git a/typeshed/2.7/galaxy/util/heartbeat.pyi b/typeshed/2.7/galaxy/util/heartbeat.pyi new file mode 100644 index 000000000..e7152ae9a --- /dev/null +++ b/typeshed/2.7/galaxy/util/heartbeat.pyi @@ -0,0 +1,30 @@ +# Stubs for galaxy.util.heartbeat (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +import threading + +def get_current_thread_object_dict(): ... + +class Heartbeat(threading.Thread): + config = ... # type: Any + should_stop = ... # type: bool + period = ... # type: Any + fname = ... # type: Any + file = ... # type: Any + fname_nonsleeping = ... # type: Any + file_nonsleeping = ... # type: Any + pid = ... # type: Any + nonsleeping_heartbeats = ... # type: Any + wait_event = ... # type: Any + def __init__(self, config, name: str = ..., period: int = ..., fname: str = ...) -> None: ... + def run(self): ... + def open_logs(self): ... + def close_logs(self): ... + def dump(self): ... + def shutdown(self): ... + def thread_is_sleeping(self, last_stack_frame): ... + def get_interesting_stack_frame(self, stack_frames): ... + def print_nonsleeping(self, threads_object_dict): ... + def dump_signal_handler(self, signum, frame): ... diff --git a/typeshed/2.7/galaxy/util/image_util.pyi b/typeshed/2.7/galaxy/util/image_util.pyi new file mode 100644 index 000000000..ce04019bf --- /dev/null +++ b/typeshed/2.7/galaxy/util/image_util.pyi @@ -0,0 +1,13 @@ +# Stubs for galaxy.util.image_util (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from PIL import Image as PIL + +PIL = ... # type: Any +log = ... # type: Any + +def image_type(filename): ... +def check_image_type(filename, types): ... +def get_image_ext(file_path): ... diff --git a/typeshed/2.7/galaxy/util/inflection.pyi b/typeshed/2.7/galaxy/util/inflection.pyi index 065a7a347..92f01cb91 100644 --- a/typeshed/2.7/galaxy/util/inflection.pyi +++ b/typeshed/2.7/galaxy/util/inflection.pyi @@ -1,4 +1,4 @@ -# Stubs for galaxy.util.inflection (Python 3.5) +# Stubs for galaxy.util.inflection (Python 3.4) # # NOTE: This dynamically typed stub was automatically generated by stubgen. diff --git a/typeshed/2.7/galaxy/util/json.pyi b/typeshed/2.7/galaxy/util/json.pyi new file mode 100644 index 000000000..d124bd660 --- /dev/null +++ b/typeshed/2.7/galaxy/util/json.pyi @@ -0,0 +1,12 @@ +# Stubs for galaxy.util.json (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def json_fix(val): ... +def safe_dumps(*args, **kwargs): ... +def validate_jsonrpc_request(request, regular_methods, notification_methods): ... +def validate_jsonrpc_response(response, id: Optional[Any] = ...): ... +def jsonrpc_request(method, params: Optional[Any] = ..., id: Optional[Any] = ..., jsonrpc: str = ...): ... +def jsonrpc_response(request: Optional[Any] = ..., id: Optional[Any] = ..., result: Optional[Any] = ..., error: Optional[Any] = ..., jsonrpc: str = ...): ... diff --git a/typeshed/2.7/galaxy/util/lazy_process.pyi b/typeshed/2.7/galaxy/util/lazy_process.pyi new file mode 100644 index 000000000..045e42f44 --- /dev/null +++ b/typeshed/2.7/galaxy/util/lazy_process.pyi @@ -0,0 +1,22 @@ +# Stubs for galaxy.util.lazy_process (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class LazyProcess: + command_and_args = ... # type: Any + thread_lock = ... # type: Any + allow_process_request = ... # type: bool + process = ... # type: Any + def __init__(self, command_and_args) -> None: ... + def start_process(self): ... + def shutdown(self): ... + @property + def running(self): ... + +class NoOpLazyProcess: + def start_process(self): ... + def shutdown(self): ... + @property + def running(self): ... diff --git a/typeshed/2.7/galaxy/util/object_wrapper.pyi b/typeshed/2.7/galaxy/util/object_wrapper.pyi new file mode 100644 index 000000000..8a5cda6e6 --- /dev/null +++ b/typeshed/2.7/galaxy/util/object_wrapper.pyi @@ -0,0 +1,95 @@ +# Stubs for galaxy.util.object_wrapper (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from six.moves import copyreg as copy_reg +from galaxy.util import sanitize_lists_to_string as _sanitize_lists_to_string + +NoneType = ... # type: Any +NotImplementedType = ... # type: Any +EllipsisType = ... # type: Any +XRangeType = ... # type: Any +SliceType = ... # type: Any +BufferType = ... # type: Any +DictProxyType = ... # type: Any +log = ... # type: Any +__CALLABLE_TYPES__ = ... # type: Any +__WRAP_NO_SUBCLASS__ = ... # type: Any +__DONT_SANITIZE_TYPES__ = ... # type: Any +__DONT_WRAP_TYPES__ = ... # type: Any +__WRAP_SEQUENCES__ = ... # type: Any +__WRAP_SETS__ = ... # type: Any +__WRAP_MAPPINGS__ = ... # type: Any +VALID_CHARACTERS = ... # type: Any +CHARACTER_MAP = ... # type: Any +INVALID_CHARACTER = ... # type: str + +def coerce(x, y): ... +def cmp(x, y): ... +def sanitize_lists_to_string(values, valid_characters: Any = ..., character_map: Any = ..., invalid_character: Any = ...): ... +def wrap_with_safe_string(value, no_wrap_classes: Optional[Any] = ...): ... + +class SafeStringWrapper: + __UNSANITIZED_ATTRIBUTE_NAME__ = ... # type: str + __NO_WRAP_NAMES__ = ... # type: Any + def __new__(cls, *arg, **kwd): ... + unsanitized = ... # type: Any + __safe_string_wrapper_function__ = ... # type: Any + def __init__(self, value, safe_string_wrapper_function: Any = ...) -> None: ... + def __lt__(self, other): ... + def __le__(self, other): ... + def __eq__(self, other): ... + def __ne__(self, other): ... + def __gt__(self, other): ... + def __ge__(self, other): ... + def __cmp__(self, other): ... + def __hash__(self): ... + def __bool__(self): ... + __nonzero__ = ... # type: Any + def __getattr__(self, name): ... + def __setattr__(self, name, value): ... + def __delattr__(self, name): ... + def __getattribute__(self, name): ... + def __len__(self): ... + def __getitem__(self, key): ... + def __setitem__(self, key, value): ... + def __delitem__(self, key): ... + def __iter__(self): ... + def __contains__(self, item): ... + def __getslice__(self, i, j): ... + def __setslice__(self, i, j, value): ... + def __delslice__(self, i, j): ... + def __add__(self, other): ... + def __sub__(self, other): ... + def __mul__(self, other): ... + def __floordiv__(self, other): ... + def __mod__(self, other): ... + def __divmod__(self, other): ... + def __pow__(self, *other): ... + def __lshift__(self, other): ... + def __rshift__(self, other): ... + def __and__(self, other): ... + def __xor__(self, other): ... + def __or__(self, other): ... + def __div__(self, other): ... + def __truediv__(self, other): ... + def __rpow__(self, other): ... + def __neg__(self): ... + def __pos__(self): ... + def __abs__(self): ... + def __invert__(self): ... + def __complex__(self): ... + def __int__(self): ... + def __float__(self): ... + def __oct__(self): ... + def __hex__(self): ... + def __index__(self): ... + def __coerce__(self, other): ... + def __enter__(self): ... + def __exit__(self, *args): ... + +class CallableSafeStringWrapper(SafeStringWrapper): + def __call__(self, *args, **kwds): ... + +def pickle_SafeStringWrapper(safe_object): ... diff --git a/typeshed/2.7/galaxy/util/odict.pyi b/typeshed/2.7/galaxy/util/odict.pyi new file mode 100644 index 000000000..459e30699 --- /dev/null +++ b/typeshed/2.7/galaxy/util/odict.pyi @@ -0,0 +1,27 @@ +# Stubs for galaxy.util.odict (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from six.moves import UserDict + +dict_alias = ... # type: Any + +class odict(UserDict): + def __init__(self, dict: Optional[Any] = ...) -> None: ... + def __delitem__(self, key): ... + def __setitem__(self, key, item): ... + def clear(self): ... + def copy(self): ... + def items(self): ... + def keys(self): ... + def popitem(self): ... + def setdefault(self, key, failobj: Optional[Any] = ...): ... + def update(self, dict): ... + def values(self): ... + def iterkeys(self): ... + def itervalues(self): ... + def iteritems(self): ... + def __iter__(self): ... + def reverse(self): ... + def insert(self, index, key, item): ... diff --git a/typeshed/2.7/galaxy/util/oset.pyi b/typeshed/2.7/galaxy/util/oset.pyi new file mode 100644 index 000000000..1d1f7bd80 --- /dev/null +++ b/typeshed/2.7/galaxy/util/oset.pyi @@ -0,0 +1,19 @@ +# Stubs for galaxy.util.oset (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +import collections + +class OrderedSet(collections.MutableSet): + end = ... # type: Any + map = ... # type: Any + def __init__(self, iterable: Optional[Any] = ...) -> None: ... + def __len__(self): ... + def __contains__(self, key): ... + def add(self, key): ... + def discard(self, key): ... + def __iter__(self): ... + def __reversed__(self): ... + def pop(self, last: bool = ...): ... + def __eq__(self, other): ... diff --git a/typeshed/2.7/galaxy/util/plugin_config.pyi b/typeshed/2.7/galaxy/util/plugin_config.pyi new file mode 100644 index 000000000..887a82b7b --- /dev/null +++ b/typeshed/2.7/galaxy/util/plugin_config.pyi @@ -0,0 +1,11 @@ +# Stubs for galaxy.util.plugin_config (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +yaml = ... # type: Any + +def plugins_dict(module, plugin_type_identifier): ... +def load_plugins(plugins_dict, plugin_source, extra_kwds: Any = ...): ... +def plugin_source_from_path(path): ... diff --git a/typeshed/2.7/galaxy/util/properties.pyi b/typeshed/2.7/galaxy/util/properties.pyi new file mode 100644 index 000000000..b59fdd0c8 --- /dev/null +++ b/typeshed/2.7/galaxy/util/properties.pyi @@ -0,0 +1,19 @@ +# Stubs for galaxy.util.properties (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional +from six.moves.configparser import ConfigParser + +def find_config_file(default, old_default, explicit, cwd: Optional[Any] = ...): ... +def load_app_properties(kwds: Any = ..., ini_file: Optional[Any] = ..., ini_section: str = ..., config_prefix: str = ...): ... + +class NicerConfigParser(ConfigParser): + filename = ... # type: Any + def __init__(self, filename, *args, **kw) -> None: ... + read_file = ... # type: Any + def defaults(self): ... + class InterpolateWrapper: + def __init__(self, original) -> None: ... + def __getattr__(self, name): ... + def before_get(self, parser, section, option, value, defaults): ... diff --git a/typeshed/2.7/galaxy/util/simplegraph.pyi b/typeshed/2.7/galaxy/util/simplegraph.pyi new file mode 100644 index 000000000..b1aba1a6e --- /dev/null +++ b/typeshed/2.7/galaxy/util/simplegraph.pyi @@ -0,0 +1,26 @@ +# Stubs for galaxy.util.simplegraph (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +class SimpleGraphNode: + index = ... # type: Any + data = ... # type: Any + def __init__(self, index, **data) -> None: ... + +class SimpleGraphEdge: + source_index = ... # type: Any + target_index = ... # type: Any + data = ... # type: Any + def __init__(self, source_index, target_index, **data) -> None: ... + +class SimpleGraph: + nodes = ... # type: Any + edges = ... # type: Any + def __init__(self, nodes: Optional[Any] = ..., edges: Optional[Any] = ...) -> None: ... + def add_node(self, node_id, **data): ... + def add_edge(self, source_id, target_id, **data): ... + def gen_node_dicts(self): ... + def gen_edge_dicts(self): ... + def as_dict(self): ... diff --git a/typeshed/2.7/galaxy/util/sleeper.pyi b/typeshed/2.7/galaxy/util/sleeper.pyi new file mode 100644 index 000000000..600adf989 --- /dev/null +++ b/typeshed/2.7/galaxy/util/sleeper.pyi @@ -0,0 +1,11 @@ +# Stubs for galaxy.util.sleeper (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +class Sleeper: + condition = ... # type: Any + def __init__(self) -> None: ... + def sleep(self, seconds): ... + def wake(self): ... diff --git a/typeshed/2.7/galaxy/util/sockets.pyi b/typeshed/2.7/galaxy/util/sockets.pyi new file mode 100644 index 000000000..100aba9a9 --- /dev/null +++ b/typeshed/2.7/galaxy/util/sockets.pyi @@ -0,0 +1,7 @@ +# Stubs for galaxy.util.sockets (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any, Optional + +def unused_port(range: Optional[Any] = ...): ... diff --git a/typeshed/2.7/galaxy/util/specs.pyi b/typeshed/2.7/galaxy/util/specs.pyi new file mode 100644 index 000000000..e3a95b9de --- /dev/null +++ b/typeshed/2.7/galaxy/util/specs.pyi @@ -0,0 +1,9 @@ +# Stubs for galaxy.util.specs (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def to_str_or_none(value): ... +def to_bool_or_none(value): ... +def to_bool(value): ... +def to_float_or_none(value): ... +def is_in(*args): ... diff --git a/typeshed/2.7/galaxy/util/sqlite.pyi b/typeshed/2.7/galaxy/util/sqlite.pyi new file mode 100644 index 000000000..53c7eaeaf --- /dev/null +++ b/typeshed/2.7/galaxy/util/sqlite.pyi @@ -0,0 +1,6 @@ +# Stubs for galaxy.util.sqlite (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +def is_read_only_query(query): ... +def connect(path): ... diff --git a/typeshed/2.7/galaxy/util/submodules.pyi b/typeshed/2.7/galaxy/util/submodules.pyi new file mode 100644 index 000000000..dbae0501f --- /dev/null +++ b/typeshed/2.7/galaxy/util/submodules.pyi @@ -0,0 +1,9 @@ +# Stubs for galaxy.util.submodules (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +log = ... # type: Any + +def submodules(module): ... diff --git a/typeshed/2.7/galaxy/util/topsort.pyi b/typeshed/2.7/galaxy/util/topsort.pyi new file mode 100644 index 000000000..79396d98a --- /dev/null +++ b/typeshed/2.7/galaxy/util/topsort.pyi @@ -0,0 +1,20 @@ +# Stubs for galaxy.util.topsort (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any +from galaxy.util.odict import odict as OrderedDict + +class CycleError(Exception): + preds = ... # type: Any + def __init__(self, sofar, numpreds, succs) -> None: ... + def get_partial(self): ... + def get_pred_counts(self): ... + def get_succs(self): ... + def get_elements(self): ... + def get_pairlist(self): ... + def get_preds(self): ... + def pick_a_cycle(self): ... + +def topsort(pairlist): ... +def topsort_levels(pairlist): ... diff --git a/typeshed/2.7/galaxy/util/xml_macros.pyi b/typeshed/2.7/galaxy/util/xml_macros.pyi new file mode 100644 index 000000000..27607dfd1 --- /dev/null +++ b/typeshed/2.7/galaxy/util/xml_macros.pyi @@ -0,0 +1,18 @@ +# Stubs for galaxy.util.xml_macros (Python 3.4) +# +# NOTE: This dynamically typed stub was automatically generated by stubgen. + +from typing import Any + +REQUIRED_PARAMETER = ... # type: Any + +def load(path): ... +def template_macro_params(root): ... +def raw_tool_xml_tree(path): ... +def imported_macro_paths(root): ... + +class XmlMacroDef: + elements = ... # type: Any + parameters = ... # type: Any + def __init__(self, el) -> None: ... + def macro_tokens(self, expand_el): ... From 02ecac61eae1700ce8e2be7d4a3800ab0cb52253 Mon Sep 17 00:00:00 2001 From: "Michael R. Crusoe" Date: Thu, 29 Jun 2017 10:01:08 +0200 Subject: [PATCH 13/22] E251, E241 --- setup.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/setup.py b/setup.py index 943a4fabd..a2bf00f9a 100755 --- a/setup.py +++ b/setup.py @@ -61,8 +61,8 @@ 'typing >= 3.5.3', 'six >= 1.8.0', ], - extras_require = { - 'deps': ["galaxy-lib >= 17.09.3"] + extras_require={ + 'deps': ["galaxy-lib >= 17.09.3"] }, setup_requires=[] + pytest_runner, test_suite='tests', From a55aee13f51bea26d9a37a45fc41968ed32df749 Mon Sep 17 00:00:00 2001 From: "Michael R. Crusoe" Date: Thu, 29 Jun 2017 10:01:38 +0200 Subject: [PATCH 14/22] E261 --- cwltool/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cwltool/main.py b/cwltool/main.py index 6a8ece93b..f4d723322 100755 --- a/cwltool/main.py +++ b/cwltool/main.py @@ -40,7 +40,7 @@ from galaxy.tools import deps except ImportError: ToolRequirement = None # type: ignore - ToolRequirements = None # type: ignore + ToolRequirements = None # type: ignore deps = None _logger = logging.getLogger("cwltool") From 0823ce82884e16989727872c70c3d18741ad1672 Mon Sep 17 00:00:00 2001 From: "Michael R. Crusoe" Date: Thu, 29 Jun 2017 10:05:21 +0200 Subject: [PATCH 15/22] add default value for find_default_container --- cwltool/main.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cwltool/main.py b/cwltool/main.py index f4d723322..16b86d148 100755 --- a/cwltool/main.py +++ b/cwltool/main.py @@ -660,7 +660,8 @@ def main(argsl=None, # type: List[str] 'relax_path_checks': False, 'validate': False, 'enable_ga4gh_tool_registry': False, - 'ga4gh_tool_registries': [] + 'ga4gh_tool_registries': [], + 'find_default_container': None }.iteritems(): if not hasattr(args, k): setattr(args, k, v) From 3ed134dfefd563316a5c11a71a3b9cfafdcade2a Mon Sep 17 00:00:00 2001 From: "Michael R. Crusoe" Date: Thu, 29 Jun 2017 10:24:24 +0200 Subject: [PATCH 16/22] Update job.py --- cwltool/job.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cwltool/job.py b/cwltool/job.py index c5c847a5d..6aea779c0 100644 --- a/cwltool/job.py +++ b/cwltool/job.py @@ -334,7 +334,7 @@ def run(self, pull_image=True, rm_container=True, default_container = find_default_container and find_default_container() if default_container: img_id = default_container - env = os.environ + env = cast(MutableMapping[Text, Text], os.environ) if docker_req and img_id is None and kwargs.get("use_container"): raise Exception("Docker image not available") From 15a6ab4acb7c48166724e8e935f1a9a66193e95e Mon Sep 17 00:00:00 2001 From: John Chilton Date: Mon, 3 Jul 2017 07:13:30 -0400 Subject: [PATCH 17/22] Fix null handling in redone find_default_container usage. Causing tests to fail. --- cwltool/draft2tool.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cwltool/draft2tool.py b/cwltool/draft2tool.py index a8df7e135..ad4fe10ce 100644 --- a/cwltool/draft2tool.py +++ b/cwltool/draft2tool.py @@ -174,7 +174,7 @@ class CommandLineTool(Process): def __init__(self, toolpath_object, **kwargs): # type: (Dict[Text, Any], **Any) -> None super(CommandLineTool, self).__init__(toolpath_object, **kwargs) - self.find_default_container = kwargs["find_default_container"] + self.find_default_container = kwargs.get("find_default_container", None) def makeJobRunner(self, use_container=True): # type: (Optional[bool]) -> JobBase dockerReq, _ = self.get_requirement("DockerRequirement") From dac661e6fd5197cb5350cf0658a5eaf346905b74 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Mon, 3 Jul 2017 07:14:08 -0400 Subject: [PATCH 18/22] Refine type signatures a bit for galaxy-lib. --- typeshed/2.7/galaxy/tools/deps/__init__.pyi | 6 +++--- typeshed/2.7/galaxy/tools/deps/requirements.pyi | 12 ++++++------ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/typeshed/2.7/galaxy/tools/deps/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/__init__.pyi index 25c761e1b..abac465e9 100644 --- a/typeshed/2.7/galaxy/tools/deps/__init__.pyi +++ b/typeshed/2.7/galaxy/tools/deps/__init__.pyi @@ -17,7 +17,7 @@ def build_dependency_manager(config): ... class NullDependencyManager: dependency_resolvers = ... # type: Any def uses_tool_shed_dependencies(self): ... - def dependency_shell_commands(self, requirements, **kwds): ... + def dependency_shell_commands(self, requirements: ToolRequirements, **kwds) -> List[str]: ... def find_dep(self, name, version: Optional[Any] = ..., type: str = ..., **kwds): ... class DependencyManager: @@ -27,7 +27,7 @@ class DependencyManager: def __init__(self, default_base_path, conf_file: Optional[Any] = ..., app_config: Any = ...) -> None: ... def get_resolver_option(self, resolver, key, explicit_resolver_options: Any = ...): ... def get_app_option(self, key, default: Optional[Any] = ...): ... - def dependency_shell_commands(self, requirements, **kwds): ... + def dependency_shell_commands(self, requirements: ToolRequirements, **kwds) -> List[str]: ... def requirements_to_dependencies(self, requirements, **kwds): ... def uses_tool_shed_dependencies(self): ... def find_dep(self, name, version: Optional[Any] = ..., type: str = ..., **kwds): ... @@ -36,6 +36,6 @@ class CachedDependencyManager(DependencyManager): tool_dependency_cache_dir = ... # type: Any def __init__(self, default_base_path, conf_file: Optional[Any] = ..., app_config: Any = ..., tool_dependency_cache_dir: Optional[Any] = ...) -> None: ... def build_cache(self, requirements, **kwds): ... - def dependency_shell_commands(self, requirements, **kwds): ... + def dependency_shell_commands(self, requirements: ToolRequirements, **kwds) -> List[str]: ... def hash_dependencies(self, resolved_dependencies): ... def get_hashed_dependencies_path(self, resolved_dependencies): ... diff --git a/typeshed/2.7/galaxy/tools/deps/requirements.pyi b/typeshed/2.7/galaxy/tools/deps/requirements.pyi index d422ef012..0e62c2a54 100644 --- a/typeshed/2.7/galaxy/tools/deps/requirements.pyi +++ b/typeshed/2.7/galaxy/tools/deps/requirements.pyi @@ -2,7 +2,7 @@ # # NOTE: This dynamically typed stub was automatically generated by stubgen. -from typing import Any, Optional +from typing import Any, Dict, List, Optional DEFAULT_REQUIREMENT_TYPE = ... # type: str DEFAULT_REQUIREMENT_VERSION = ... # type: Any @@ -13,10 +13,10 @@ class ToolRequirement: version = ... # type: Any specs = ... # type: Any def __init__(self, name: Optional[Any] = ..., type: Optional[Any] = ..., version: Optional[Any] = ..., specs: Any = ...) -> None: ... - def to_dict(self): ... + def to_dict(self) -> Dict[Any]: ... def copy(self): ... @staticmethod - def from_dict(dict): ... + def from_dict(dict: Dict[Any]) -> ToolRequirement: ... def __eq__(self, other): ... def __ne__(self, other): ... def __hash__(self): ... @@ -31,7 +31,7 @@ class RequirementSpecification: def short_name(self): ... def to_dict(self): ... @staticmethod - def from_dict(dict): ... + def from_dict(dict: Dict[Any]) -> ToolRequirements: ... def __eq__(self, other): ... def __ne__(self, other): ... def __hash__(self): ... @@ -40,12 +40,12 @@ class ToolRequirements: tool_requirements = ... # type: Any def __init__(self, tool_requirements: Optional[Any] = ...) -> None: ... @staticmethod - def from_list(requirements): ... + def from_list(requirements: List[ToolRequirement]) -> ToolRequirements: ... @property def resolvable(self): ... @property def packages(self): ... - def to_list(self): ... + def to_list(self) -> List[ToolRequirement]: ... def append(self, requirement): ... def __eq__(self, other): ... def __ne__(self, other): ... From 0b6f27e83064e80b6a2ce21f9a3d2be3f21cba9f Mon Sep 17 00:00:00 2001 From: John Chilton Date: Mon, 3 Jul 2017 07:28:25 -0400 Subject: [PATCH 19/22] Refactor utilities for dealing with SoftwareRequirements into own module. Per great suggestion from Michael on #217. --- cwltool/main.py | 108 ++-------------------------- cwltool/software_requirements.py | 117 +++++++++++++++++++++++++++++++ 2 files changed, 124 insertions(+), 101 deletions(-) create mode 100644 cwltool/software_requirements.py diff --git a/cwltool/main.py b/cwltool/main.py index 16b86d148..3150f2534 100755 --- a/cwltool/main.py +++ b/cwltool/main.py @@ -32,16 +32,10 @@ relocateOutputs, scandeps, shortname, use_custom_schema, use_standard_schema) from .resolver import ga4gh_tool_registries, tool_resolver +from .software_requirements import DependenciesConfiguration from .stdfsaccess import StdFsAccess from .update import ALLUPDATES, UPDATES -from .utils import get_feature -try: - from galaxy.tools.deps.requirements import ToolRequirement, ToolRequirements - from galaxy.tools import deps -except ImportError: - ToolRequirement = None # type: ignore - ToolRequirements = None # type: ignore - deps = None + _logger = logging.getLogger("cwltool") @@ -863,102 +857,14 @@ def locToPath(p): _logger.addHandler(defaultStreamHandler) -COMMAND_WITH_DEPENDENCIES_TEMPLATE = string.Template("""#!/bin/bash -$handle_dependencies -python "run_job.py" "job.json" -""") - - def find_default_container(args, builder): + default_container = None if args.default_container: - return args.default_container + default_container = args.default_container elif args.beta_use_biocontainers: - try: - from galaxy.tools.deps.containers import ContainerRegistry, AppInfo, ToolInfo, DOCKER_CONTAINER_TYPE - except ImportError: - raise Exception("galaxy-lib not found") - - app_info = AppInfo( - involucro_auto_init=True, - enable_beta_mulled_containers=True, - container_image_cache_path=".", - ) # type: AppInfo - container_registry = ContainerRegistry(app_info) # type: ContainerRegistry - requirements = _get_dependencies(builder) - tool_info = ToolInfo(requirements=requirements) # type: ToolInfo - container_description = container_registry.find_best_container_description([DOCKER_CONTAINER_TYPE], tool_info) - if container_description: - return container_description.identifier - - return None - - -class DependenciesConfiguration(object): - - def __init__(self, args): - # type: (argparse.Namespace) -> None - conf_file = getattr(args, "beta_dependency_resolvers_configuration", None) - tool_dependency_dir = getattr(args, "beta_dependencies_directory", None) - conda_dependencies = getattr(args, "beta_conda_dependencies", None) - if conf_file is not None and os.path.exists(conf_file): - self.use_tool_dependencies = True - if not tool_dependency_dir: - tool_dependency_dir = os.path.abspath(os.path.dirname(conf_file)) - self.tool_dependency_dir = tool_dependency_dir - self.dependency_resolvers_config_file = conf_file - elif conda_dependencies: - if not tool_dependency_dir: - tool_dependency_dir = os.path.abspath("./cwltool_deps") - self.tool_dependency_dir = tool_dependency_dir - self.use_tool_dependencies = True - self.dependency_resolvers_config_file = None - else: - self.use_tool_dependencies = False - - @property - def config_dict(self): - return { - 'conda_auto_install': True, - 'conda_auto_init': True, - } - - def build_job_script(self, builder, command): - # type: (Any, List[str]) -> Text - if deps is None: - raise Exception("galaxy-lib not found") - tool_dependency_manager = deps.build_dependency_manager(self) # type: deps.DependencyManager - dependencies = _get_dependencies(builder) - handle_dependencies = "" # str - if dependencies: - handle_dependencies = "\n".join(tool_dependency_manager.dependency_shell_commands(dependencies, job_directory=builder.tmpdir)) - - template_kwds = dict(handle_dependencies=handle_dependencies) # type: Dict[str, str] - job_script = COMMAND_WITH_DEPENDENCIES_TEMPLATE.substitute(template_kwds) - return job_script - - -def _get_dependencies(builder): - # type: (Any) -> List[ToolRequirement] - (software_requirement, _) = get_feature(builder, "SoftwareRequirement") - dependencies = [] # type: List[ToolRequirement] - if software_requirement and software_requirement.get("packages"): - packages = software_requirement.get("packages") - for package in packages: - version = package.get("version", None) - if isinstance(version, list): - if version: - version = version[0] - else: - version = None - specs = [{"uri": s} for s in package.get("specs", [])] - dependencies.append(ToolRequirement.from_dict(dict( - name=package["package"].split("#")[-1], - version=version, - type="package", - specs=specs, - ))) - - return ToolRequirements.from_list(dependencies) + default_container = get_container_from_software_requirements(args, builder) + + return default_container if __name__ == "__main__": diff --git a/cwltool/software_requirements.py b/cwltool/software_requirements.py new file mode 100644 index 000000000..0a2630cb7 --- /dev/null +++ b/cwltool/software_requirements.py @@ -0,0 +1,117 @@ +"""This module handles resolution of SoftwareRequirement hints. + +This is accomplished mainly by adapting cwltool internals to galaxy-lib's +concept of "dependencies". Despite the name, galaxy-lib is a light weight +library that can be used to map SoftwareRequirements in all sorts of ways - +Homebrew, Conda, custom scripts, environment modules. We'd be happy to find +ways to adapt new packages managers and such as well. +""" + +import os +import string + +try: + from galaxy.tools.deps.requirements import ToolRequirement, ToolRequirements + from galaxy.tools import deps +except ImportError: + ToolRequirement = None # type: ignore + ToolRequirements = None # type: ignore + deps = None + +from .utils import get_feature + + +COMMAND_WITH_DEPENDENCIES_TEMPLATE = string.Template("""#!/bin/bash +$handle_dependencies +python "run_job.py" "job.json" +""") + + +class DependenciesConfiguration(object): + + def __init__(self, args): + # type: (argparse.Namespace) -> None + conf_file = getattr(args, "beta_dependency_resolvers_configuration", None) + tool_dependency_dir = getattr(args, "beta_dependencies_directory", None) + conda_dependencies = getattr(args, "beta_conda_dependencies", None) + if conf_file is not None and os.path.exists(conf_file): + self.use_tool_dependencies = True + if not tool_dependency_dir: + tool_dependency_dir = os.path.abspath(os.path.dirname(conf_file)) + self.tool_dependency_dir = tool_dependency_dir + self.dependency_resolvers_config_file = conf_file + elif conda_dependencies: + if not tool_dependency_dir: + tool_dependency_dir = os.path.abspath("./cwltool_deps") + self.tool_dependency_dir = tool_dependency_dir + self.use_tool_dependencies = True + self.dependency_resolvers_config_file = None + else: + self.use_tool_dependencies = False + + @property + def config_dict(self): + return { + 'conda_auto_install': True, + 'conda_auto_init': True, + } + + def build_job_script(self, builder, command): + # type: (Any, List[str]) -> Text + if deps is None: + raise Exception("galaxy-lib not found") + tool_dependency_manager = deps.build_dependency_manager(self) # type: deps.DependencyManager + dependencies = get_dependencies(builder) + handle_dependencies = "" # str + if dependencies: + handle_dependencies = "\n".join(tool_dependency_manager.dependency_shell_commands(dependencies, job_directory=builder.tmpdir)) + + template_kwds = dict(handle_dependencies=handle_dependencies) # type: Dict[str, str] + job_script = COMMAND_WITH_DEPENDENCIES_TEMPLATE.substitute(template_kwds) + return job_script + + +def get_dependencies(builder): + # type: (Any) -> List[ToolRequirement] + (software_requirement, _) = get_feature(builder, "SoftwareRequirement") + dependencies = [] # type: List[ToolRequirement] + if software_requirement and software_requirement.get("packages"): + packages = software_requirement.get("packages") + for package in packages: + version = package.get("version", None) + if isinstance(version, list): + if version: + version = version[0] + else: + version = None + specs = [{"uri": s} for s in package.get("specs", [])] + dependencies.append(ToolRequirement.from_dict(dict( + name=package["package"].split("#")[-1], + version=version, + type="package", + specs=specs, + ))) + + return ToolRequirements.from_list(dependencies) + + +def get_container_from_software_requirements(args, builder): + if args.beta_use_biocontainers: + try: + from galaxy.tools.deps.containers import ContainerRegistry, AppInfo, ToolInfo, DOCKER_CONTAINER_TYPE + except ImportError: + raise Exception("Optional requirement galaxy-lib not found, it is required for this configuration.") + + app_info = AppInfo( + involucro_auto_init=True, + enable_beta_mulled_containers=True, + container_image_cache_path=".", + ) # type: AppInfo + container_registry = ContainerRegistry(app_info) # type: ContainerRegistry + requirements = get_dependencies(builder) + tool_info = ToolInfo(requirements=requirements) # type: ToolInfo + container_description = container_registry.find_best_container_description([DOCKER_CONTAINER_TYPE], tool_info) + if container_description: + return container_description.identifier + + return None From d473e3a1e6e337568d367a4f359f3d0a88272c41 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Mon, 3 Jul 2017 07:36:32 -0400 Subject: [PATCH 20/22] Furter refine type descriptions for galaxy-lib. --- cwltool/software_requirements.py | 4 +++- typeshed/2.7/galaxy/tools/deps/__init__.pyi | 2 +- typeshed/2.7/galaxy/tools/deps/requirements.pyi | 6 +++--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/cwltool/software_requirements.py b/cwltool/software_requirements.py index 0a2630cb7..fe8d28f5a 100644 --- a/cwltool/software_requirements.py +++ b/cwltool/software_requirements.py @@ -7,8 +7,10 @@ ways to adapt new packages managers and such as well. """ +import argparse import os import string +from typing import (Any, Dict, List, Text) try: from galaxy.tools.deps.requirements import ToolRequirement, ToolRequirements @@ -72,7 +74,7 @@ def build_job_script(self, builder, command): def get_dependencies(builder): - # type: (Any) -> List[ToolRequirement] + # type: (Any) -> ToolRequirements (software_requirement, _) = get_feature(builder, "SoftwareRequirement") dependencies = [] # type: List[ToolRequirement] if software_requirement and software_requirement.get("packages"): diff --git a/typeshed/2.7/galaxy/tools/deps/__init__.pyi b/typeshed/2.7/galaxy/tools/deps/__init__.pyi index abac465e9..2bf533cd6 100644 --- a/typeshed/2.7/galaxy/tools/deps/__init__.pyi +++ b/typeshed/2.7/galaxy/tools/deps/__init__.pyi @@ -12,7 +12,7 @@ from .resolvers.tool_shed_packages import ToolShedPackageDependencyResolver as T log = ... # type: Any CONFIG_VAL_NOT_FOUND = ... # type: Any -def build_dependency_manager(config): ... +def build_dependency_manager(config: Any) -> DependencyManager: ... class NullDependencyManager: dependency_resolvers = ... # type: Any diff --git a/typeshed/2.7/galaxy/tools/deps/requirements.pyi b/typeshed/2.7/galaxy/tools/deps/requirements.pyi index 0e62c2a54..9cb78c3ae 100644 --- a/typeshed/2.7/galaxy/tools/deps/requirements.pyi +++ b/typeshed/2.7/galaxy/tools/deps/requirements.pyi @@ -13,10 +13,10 @@ class ToolRequirement: version = ... # type: Any specs = ... # type: Any def __init__(self, name: Optional[Any] = ..., type: Optional[Any] = ..., version: Optional[Any] = ..., specs: Any = ...) -> None: ... - def to_dict(self) -> Dict[Any]: ... + def to_dict(self) -> Dict[str, Any]: ... def copy(self): ... @staticmethod - def from_dict(dict: Dict[Any]) -> ToolRequirement: ... + def from_dict(dict: Dict[str, Any]) -> ToolRequirement: ... def __eq__(self, other): ... def __ne__(self, other): ... def __hash__(self): ... @@ -31,7 +31,7 @@ class RequirementSpecification: def short_name(self): ... def to_dict(self): ... @staticmethod - def from_dict(dict: Dict[Any]) -> ToolRequirements: ... + def from_dict(dict: Dict[str, Any]) -> ToolRequirements: ... def __eq__(self, other): ... def __ne__(self, other): ... def __hash__(self): ... From 75c8d8153d8909ea79f35b0c65a7cc438c1118f6 Mon Sep 17 00:00:00 2001 From: John Chilton Date: Thu, 6 Jul 2017 21:17:39 -0400 Subject: [PATCH 21/22] I suppose this is needed after 0b6f27e83064e80b6a2ce21f9a3d2be3f21cba9f. --- cwltool/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cwltool/main.py b/cwltool/main.py index 3150f2534..20bbc8c58 100755 --- a/cwltool/main.py +++ b/cwltool/main.py @@ -32,7 +32,7 @@ relocateOutputs, scandeps, shortname, use_custom_schema, use_standard_schema) from .resolver import ga4gh_tool_registries, tool_resolver -from .software_requirements import DependenciesConfiguration +from .software_requirements import DependenciesConfiguration, get_container_from_software_requirements from .stdfsaccess import StdFsAccess from .update import ALLUPDATES, UPDATES From 5522e882d8feeb666230c5e19fbb3b5b2ccc634a Mon Sep 17 00:00:00 2001 From: John Chilton Date: Fri, 7 Jul 2017 14:22:45 -0400 Subject: [PATCH 22/22] Docs for resolving SoftwareRequirements. --- README.rst | 206 ++++++++++++++++++++ tests/test_deps_env/random-lines/1.0/env.sh | 4 +- 2 files changed, 208 insertions(+), 2 deletions(-) diff --git a/README.rst b/README.rst index 28c0dd051..4fffee191 100644 --- a/README.rst +++ b/README.rst @@ -139,6 +139,212 @@ The easiest way to use cwltool to run a tool or workflow from Python is to use a # result["out"] == "foo" +Leveraging SoftwareRequirements (Beta) +-------------------------------------- + +CWL tools may be decoarated with ``SoftwareRequirement`` hints that cwltool +may in turn use to resolve to packages in various package managers or +dependency management systems such as `Environment Modules +`__. + +Utilizing ``SoftwareRequirement`` hints using cwltool requires an optional +dependency, for this reason be sure to use specify the ``deps`` modifier when +installing cwltool. For instance:: + + $ pip install 'cwltool[deps]' + +Installing cwltool in this fashion enables several new command line options. +The most general of these options is ``--beta-dependency-resolvers-configuration``. +This option allows one to specify a dependency resolvers configuration file. +This file may be specified as either XML or YAML and very simply describes various +plugins to enable to "resolve" ``SoftwareRequirement`` dependencies. + +To discuss some of these plugins and how to configure them, first consider the +following ``hint`` definition for an example CWL tool. + +.. code:: yaml + + SoftwareRequirement: + packages: + - package: seqtk + version: + - r93 + +Now imagine deploying cwltool on a cluster with Software Modules installed +and that a ``seqtk`` module is avaialble at version ``r93``. This means cluster +users likely won't have the ``seqtk`` the binary on their ``PATH`` by default but after +sourcing this module with the command ``modulecmd sh load seqtk/r93`` ``seqtk`` is +available on the ``PATH``. A simple dependency resolvers configuration file, called +``dependency-resolvers-conf.yml`` for instance, that would enable cwltool to source +the correct module environment before executing the above tool would simply be: + +.. code:: yaml + + - type: module + +The outer list indicates that one plugin is being enabled, the plugin parameters are +defined as a dictionary for this one list item. There is only one required parameter +for the plugin above, this is ``type`` and defines the plugin type. This parameter +is required for all plugins. The available plugins and the parameters +available for each are documented (incompletely) `here +`__. +Unfortunately, this documentation is in the context of Galaxy tool ``requirement`` s instead of CWL ``SoftwareRequirement`` s, but the concepts map fairly directly. + +cwltool is distributed with an example of such seqtk tool and sample corresponding +job. It could executed from the cwltool root using a dependency resolvers +configuration file such as the above one using the command:: + + cwltool --beta-dependency-resolvers-configuration /path/to/dependency-resolvers-conf.yml \ + tests/seqtk_seq.cwl \ + tests/seqtk_seq_job.json + +This example demonstrates both that cwltool can leverage +existing software installations and also handle workflows with dependencies +on different versions of the same software and libraries. However the above +example does require an existing module setup so it is impossible to test this example +"out of the box" with cwltool. For a more isolated test that demonstrates all +the same concepts - the resolver plugin type ``galaxy_packages`` can be used. + +"Galaxy packages" are a lighter weight alternative to Environment Modules that are +really just defined by a way to lay out directories into packages and versions +to find little scripts that are sourced to modify the environment. They have +been used for years in Galaxy community to adapt Galaxy tools to cluster +environments but require neither knowledge of Galaxy nor any special tools to +setup. These should work just fine for CWL tools. + +The cwltool source code repository's test directory is setup with a very simple +directory that defines a set of "Galaxy packages" (but really just defines one +package named ``random-lines``). The directory layout is simply:: + + tests/test_deps_env/ + random-lines/ + 1.0/ + env.sh + +If the ``galaxy_packages`` plugin is enabled and pointed at the +``tests/test_deps_env`` directory in cwltool's root and a ``SoftwareRequirement`` +such as the following is encountered. + +.. code:: yaml + + hints: + SoftwareRequirement: + packages: + - package: 'random-lines' + version: + - '1.0' + +Then cwltool will simply find that ``env.sh`` file and source it before executing +the corresponding tool. That ``env.sh`` script is only responsible for modifying +the job's ``PATH`` to add the required binaries. + +This is a full example that works since resolving "Galaxy packages" has no +external requirements. Try it out by executing the following command from cwltool's +root directory:: + + cwltool --beta-dependency-resolvers-configuration tests/test_deps_env_resolvers_conf.yml \ + tests/random_lines.cwl \ + tests/random_lines_job.json + +The resolvers configuration file in the above example was simply: + +.. code:: yaml + + - type: galaxy_packages + base_path: ./tests/test_deps_env + +It is possible that the ``SoftwareRequirement`` s in a given CWL tool will not +match the module names for a given cluster. Such requirements can be re-mapped +to specific deployed packages and/or versions using another file specified using +the resolver plugin parameter `mapping_files`. We will +demonstrate this using `galaxy_packages` but the concepts apply equally well +to Environment Modules or Conda packages (described below) for instance. + +So consider the resolvers configuration file +(`tests/test_deps_env_resolvers_conf_rewrite.yml`): + +.. code:: yaml + + - type: galaxy_packages + base_path: ./tests/test_deps_env + mapping_files: ./tests/test_deps_mapping.yml + +And the corresponding mapping configuraiton file (`tests/test_deps_mapping.yml`): + +.. code:: yaml + + - from: + name: randomLines + version: 1.0.0-rc1 + to: + name: random-lines + version: '1.0' + +This is saying if cwltool encounters a requirement of ``randomLines`` at version +``1.0.0-rc1`` in a tool, to rewrite to our specific plugin as ``random-lines`` at +version ``1.0``. cwltool has such a test tool called ``random_lines_mapping.cwl`` +that contains such a source ``SoftwareRequirement``. To try out this example with +mapping, execute the following command from the cwltool root directory:: + + cwltool --beta-dependency-resolvers-configuration tests/test_deps_env_resolvers_conf_rewrite.yml \ + tests/random_lines_mapping.cwl \ + tests/random_lines_job.json + +The previous examples demonstrated leveraging existing infrastructure to +provide requirements for CWL tools. If instead a real package manager is used +cwltool has the oppertunity to install requirements as needed. While initial +support for Homebrew/Linuxbrew plugins is available, the most developed such +plugin is for the `Conda `__ package manager. Conda has the nice properties +of allowing multiple versions of a package to be installed simultaneously, +not requiring evalated permissions to install Conda itself or packages using +Conda, and being cross platform. For these reasons, cwltool may run as a normal +user, install its own Conda environment and manage multiple versions of Conda packages +on both Linux and Mac OS X. + +The Conda plugin can be endlessly configured, but a sensible set of defaults +that has proven a powerful stack for dependency management within the Galaxy tool +development ecosystem can be enabled by simply passing cwltool the +``--beta-conda-dependencies`` flag. + +With this we can use the seqtk example above without Docker and without +any externally managed services - cwltool should install everything it needs +and create an environment for the tool. Try it out with the follwing command:: + + cwltool --beta-conda-dependencies tests/seqtk_seq.cwl tests/seqtk_seq_job.json + +The CWL specification allows URIs to be attached to ``SoftwareRequirement`` s +that allow disambiguation of package names. If the mapping files described above +allow deployers to adapt tools to their infrastructure, this mechanism allows +tools to adapt their requirements to multiple package managers. To demonstrate +this within the context of the seqtk, we can simply break the package name we +use and then specify a specific Conda package as follows: + +.. code:: yaml + + hints: + SoftwareRequirement: + packages: + - package: seqtk_seq + version: + - '1.2' + specs: + - https://anaconda.org/bioconda/seqtk + - https://packages.debian.org/sid/seqtk + +The example can be executed using the command:: + + cwltool --beta-conda-dependencies tests/seqtk_seq_wrong_name.cwl tests/seqtk_seq_job.json + +The plugin framework for managing resolution of these software requirements +as maintained as part of `galaxy-lib `__ - a small, portable subset of the Galaxy +project. More information on configuration and implementation can be found +at the following links: + +- `Dependency Resolvers in Galaxy `__ +- `Conda for [Galaxy] Tool Dependencies `__ +- `Mapping Files - Implementation `__ +- `Specifications - Implementation `__ +- `Initial cwltool Integration Pull Request `__ Cwltool control flow -------------------- diff --git a/tests/test_deps_env/random-lines/1.0/env.sh b/tests/test_deps_env/random-lines/1.0/env.sh index 453f53494..48e1611b6 100644 --- a/tests/test_deps_env/random-lines/1.0/env.sh +++ b/tests/test_deps_env/random-lines/1.0/env.sh @@ -1,8 +1,8 @@ -#PACKAGE_DIRECTORY="/path/to/cwlroot/tests/test_deps_env/random-lines/1.0/" +#PACKAGE_DIRECTORY="/path/to/cwlroot" # This shouldn't need to use bash-isms - but we don't know the full path to this file, # so for testing it is setup this way. For actual deployments just using full paths # directly would be preferable. -PACKAGE_DIRECTORY="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PACKAGE_DIRECTORY="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/tests/test_deps_env/random-lines/1.0/" export PATH=$PATH:$PACKAGE_DIRECTORY/scripts