Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* 85 release (#28) * Update cromwell version from 83 to 84 * BW-1255 Implement POST /runs endpoint (broadinstitute#6779) * Adding route * Fixing HTTP method error * All formFields made optional * A compliling state * Saving * Saving * All three endpoints functioning as expected; updated RESTAPI.md * Updated response for submission from 200 to 201 to pass tests * Test submission response * Moved updated submission response to askSubmit * test * updating RESTAPI.md * saving * Adding utility file for submitRequest * cleanup * Update awssdkv from 2.17.152 to 2.17.194 (broadinstitute#6814) * BW-1305 Swagger Update (broadinstitute#6818) * Properly documenting metadataArchiveStatus in WorkflowQueryResult model * Update docs * BT-710 Add configs for BlobPathBuilderFactory (broadinstitute#6817) BT-710 Add configs for BlobPathBuilderFactory * BW-1305 Make "name" optional in workflow query response (broadinstitute#6821) * BT-724 Fix BlobPathBuilder failing on retrieving existing filesystem (broadinstitute#6816) Modify blobPathBuilder to fallback to creating a filesystem if one is not found * Logging updates: (broadinstitute#6813) * [BT-698] first pass on BlobTokenGenerator with E2E test (broadinstitute#6824) * first pass on BlobTokenGenerator with E2E test * update BlobPathBuilder constructor args in test * account -> container level client * [BT-687] specify correct types (broadinstitute#6829) * specify correct types * fix test with new type * remove type declarations in function call * remove unnecessary sas-token config * BW-1206 - Combine all Wes Endpoints & add Tests (broadinstitute#6833) * Add tests, getting frid of WesRunRoutes.scala * wesWorkflowId fix, ec implicits errors gone * Refactoring path for GET /runs * Indentation fix * Commit to rollback * Revert "Indentation fix" This reverts commit 63fc484. * PR trigger * Optimize imports * Missed import * BW-1354 - Porting CBAS preliminary step (broadinstitute#6837) * Getting rid of shared utility file; Adding/Updating WES version of submit. * Edit spec file * Adding Wes-like error * BW-1378 Addl CromIAM user enablement checks (broadinstitute#6826) * Update cromwell version from 84 to 85 * BW-1393 Release doc updates (broadinstitute#6839) * BT-732 Checksum validation for blobs read by engine (broadinstitute#6838) * Draft support for optional FileHash * Draft getMd5 for BlobPath * Resolve non-parallel IO to fix tests * Checksum validation for BlobPath * Nicer error message * Test for missing Blob hash * Break attr acquisition into separate method * Cleanup, comments * In-progress tests of blob hash command * Remove test * Remove unused import * BT-711 Refresh SAS token for filesystem on expiry (broadinstitute#6831) * BT-711 Refresh SAS token for filesystem on expiry * Rough cut of token refresh using exceptions * Ignore tests, and minor cleanup * Remove stray line * Draft of manager class for handling expiring file systems * Style fixes * Refactor of blobfilesystemManager and tests covering its functionality * Refined tests to validate close filesystem as separate unit * Ignore connected tests * Clean up of some things * Refactor BlobFileSystemManager to separate file, and some other cleanup * Some additional scala-ifying * Small cleanup * Correcting imports * trigger tests * trigger tests * Batch 1 of scala steward updates (broadinstitute#6903) * Batch 1 of scala steward updates * Rollback snakeYAML * Attempt 3, with only the passing dependancies * Revert google API and Big Query udpates * Winding back other google deps * rollback remaining google updates * trigger tests * trigger tests * [BW-1398] Migrate PKs to BIGINT (broadinstitute#6907) * BT-745 Batch 2 of scala steward updates (broadinstitute#6906) * Update SBT to 2.0.0 * Fix sbt-git import * Update mouse to 1.0.11 * Update rhino 1.7.14 * SUP-692 Retry with more memory after RC 137 (broadinstitute#6912) * Reorder execution result checks so 137 can retry with more memory * Test for memory retry after 137 RC * Fix test expectations * Make memory retry checks consistent * Revert changes to existing test * Rename retryWithMoreMemory to outOfMemoryDetected * Scala steward updates batch 3 (broadinstitute#6913) * Scala steward updates batch 3 * WX-745 Batch 4 scala steward updates (broadinstitute#6916) * WX-746 Localize all DRS inputs in a single Action (broadinstitute#6914) Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> * WX-755 Build all images instead of just Cromwell (broadinstitute#6919) * WX-755 Add `isRelease` option for Docker builds (broadinstitute#6923) * WX-755 Cromwell/CromIAM automatically board train (broadinstitute#6924) * WX-755 Fix environment variable syntax (broadinstitute#6926) * WX-743 Enable TES task creation with BlobPaths (broadinstitute#6921) * Give blob SAS tokens write permission * Case class wrapper for subscription id * Resolve duplicate container name in absolute BlobPath * Ignored test demonstrating correct absolute path generation * Update filesystems/blob/src/test/scala/cromwell/filesystems/blob/BlobPathBuilderSpec.scala Co-authored-by: Brian Reilly <breilly@broadinstitute.org> * PR feedback Co-authored-by: Brian Reilly <breilly@broadinstitute.org> * [WX-765] Update snakeyaml to 1.33 (broadinstitute#6927) * update snakeyaml to 1.33 * Don't use deprecated no-arg Constructor constructor Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> * WM-1414 Refactoring WesRunLog to omit Cromwell's "workflowLog" object (broadinstitute#6925) * Upgrade Postgres to 42.4.1 (broadinstitute#6932) * WX-735 Fix incorrect and/or nondeterministic filesystem ordering (broadinstitute#6930) * WX-772 Update Scala to 2.13.9 (broadinstitute#6928) * Update Scala to 2.13.9 * Try updating sbt-scoverage * Does this version exist anywhere we can see? * This version actually exists * Update library version to remove conflict * Codegen version * Fix fun new 2.13.9 compiler errors * Resolve warnings * Newest Scala? * I guess not * Does this please Travis? * force ci * Back out changes to generated code Co-authored-by: Adam Nichols <aednichols@gmail.com> * WX-781 Bump jackson-databind in /CromwellRefdiskManifestCreator (broadinstitute#6935) Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.13.2.2 to 2.13.4.1. - [Release notes](https://github.com/FasterXML/jackson/releases) - [Commits](https://github.com/FasterXML/jackson/commits) --- updated-dependencies: - dependency-name: com.fasterxml.jackson.core:jackson-databind dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * WX-808 Host allowlist for HTTP imports (broadinstitute#6938) * `hostAllowlist` that allows everything * Refactor * Stick allow list in HttpResolver * Better default config * Allow list tests * Make it build Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> * Update commons text to 1.10.0 (broadinstitute#6937) * WX-751 Token refresh signal for monitoring (broadinstitute#6939) * Log messages * `DEBUG` -> `INFO` * WX-744 Optionally rewrite blob paths to appear as local paths (broadinstitute#6941) * Modify blob paths for TES * Make blob transformation configurable * Update supportedBackends/tes/src/main/scala/cromwell/backend/impl/tes/TesTask.scala Co-authored-by: Adam Nichols <anichols@broadinstitute.org> * Apply PR feedback in second place Co-authored-by: Adam Nichols <anichols@broadinstitute.org> * Update changelog for wdl http allow list (broadinstitute#6944) * WM-1491 Fixing Cromwell-client (broadinstitute#6943) * More updated client for use in cbas * Removing excess code * Fix client build script (broadinstitute#6945) * WX-837: Remove CWL references from documentation (broadinstitute#6949) * wx-837 removed cwl references in markdown doc files * wx-837 removed cwlParsingOverview.md, updated mkdocs.yml * wx-837 updated cromwell.yaml, generated new RESTAPI file * WX-728 Add configurable WSM client to Cromwell (broadinstitute#6948) * Dependencies * Compiles but no tests * Formatting * Moar exclusions * Update to latest WSM * Add additional dependency * We need some UUID here to make the request * Formatting * Clarify what is fake * Formatting * Use our own version of Jersey and Jackson stuff * Port-in Khalid's changes (thank you!) Co-authored-by: Khalid Shakir <kshakir@broadinstitute.org> * Test longevity Don't break the test if someone decides to add a cert to `ws.org` * Cleanup * Cleanup * Cleanup * Adjust TES config file for CI Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> Co-authored-by: Khalid Shakir <kshakir@broadinstitute.org> * CROM-6554: Removed PAPIv1 references from doc (broadinstitute#6950) * crom-6554 removed references to PAPI v1 from doc * crom-6554 pr feedback, reworded doc to use example conf as a starting point * WX-833 Real Azure DRS Credentials (broadinstitute#6952) * Remove B2C reference from name * Get token for current user rather than getting from KeyVault * Remove KeyVault config for engine * Remove KeyVault config for DRSLocalizer * Remove KeyVault dependency * Remove KeyVault support from localizer repo template * Cleaned up and working Azure token acquisition for engine * Collapse localizer's AccessTokenStrategy into DrsCredentials * Cleanup * WX-853 Remove most CWL (broadinstitute#6955) * WX-696 Enable getting SAS token from WSM (broadinstitute#6954) * WX-696 Enable getting SAS token from WSM * Wire container resource id from config * Move resource-container-id config path * First pass at config for WSM * Remove unused singleton config * Tests for new config * Fix config parsing * Modified b2c token to be provided each time * Remove singletonConfig arg from factory * Restore types to factory configs * Clean up comments and empty token default * Default to config b2c before searching environment * Fix token default on api client * Fix test * Refactor error handling for when there is no token * Remove token constructor arg for clientProvider * Move configs to global singleton config * Update filesystems/blob/src/main/scala/cromwell/filesystems/blob/BlobFileSystemManager.scala * default -> override * Add override token to test * Update filesystems/blob/src/main/scala/cromwell/filesystems/blob/BlobFileSystemManager.scala Co-authored-by: Adam Nichols <anichols@broadinstitute.org> * Parentheses * Reduce token timeout * Move AzureCredentials to separate file * Make AzureCredentials an object * WSM token cleanup * Config refactor (broadinstitute#6960) Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> * Initial blob token documentation * Refine language in BlobSasTokenGenerator * Update comment and formatting Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> Co-authored-by: Adam Nichols <anichols@broadinstitute.org> * WX-853 Remove CWL language factory, Centaur runner (broadinstitute#6961) * WX-842 Add Pact Dependency for Cromwell (broadinstitute#6962) * WX-842 Add Pact Dependency for Cromwell * Remove incomplete test spec * Initial Pact Test * Fix pact so it compiles * Add breadcrumb comment and clean up * ID-125 Add support for drshub, rename all the things (broadinstitute#6959) * Add support for drshub, rename all the things * fallback to martha if resolver is not in config * WX-867 Translate crc32c hashes to b64 for getm (broadinstitute#6970) * Translate crc32c hashes to b64 for getm * Update tests * Remove obsolete b64 handling for md5, centralize hex validation * Restore old test, fix other test * WX-843 Workflow failure reason should accurately indicate issues opening blob filesystem (broadinstitute#6965) * WX-859 Accept workflow execution identity in config (broadinstitute#6967) * WX-892 Trim down `ValueStore` logging to prevent OOMs (broadinstitute#6981) * Add Nirvana 3.18.1 reference image test, minor cleanup [VS-705] (broadinstitute#6975) * WX-863 Turn off Azure NIO logging (broadinstitute#6982) * Turn off Azure NIO logging * Poke Travis * WM-1616: Allow repeating attempts at initialization (take 2) (broadinstitute#6985) * WX-878 Single shared BlobFileSystemManager (broadinstitute#6986) * Make BlobFileSystemManager shared across all BlobPathBuilders * Update TES conf file to reflect new singleton config * Shell escape reference image files [VS-796] [WX-910] (broadinstitute#6989) * WX-769 `disks` compatibility for TES backend (broadinstitute#6991) * Update FiveMinuteIntro.md (broadinstitute#6994) * WX-906 Sbt Unit Tests as Github Actions (broadinstitute#6992) * WX-926 Support falling back to OCI Manifest Format (broadinstitute#7003) * WX-926 Support falling back to OCI Manifest Forma * Only mount reference disks if requested [WX-925] (broadinstitute#7001) * [WM-1646] Add missing fields for `WorkflowDescription` for WomTool /describe endpoint to Swagger (broadinstitute#7004) * WX-876 Surface TES System Logs to Cromwell when TES backend returns task error status (broadinstitute#6980) * WX-876 Surface TES System Logs to Cromwell when TES backend returns task error status * Address feedback * Address feedback (broadinstitute#6997) * Address additional feedback (broadinstitute#7000) * Fix copy/paste error (broadinstitute#7005) * Address additional feedback * Fix copy/paste error * Trigger CI --------- Co-authored-by: Blair Murri <BMurri@users.noreply.github.com> Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> * Centaur reference image test should validate symlinks [VS-796] (broadinstitute#6996) * WX-903 Pre-GHA test suite disablement * WX-877 Update CHANGELOG for release 85 (broadinstitute#7011) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> Co-authored-by: Katrina P <68349264+kpierre13@users.noreply.github.com> Co-authored-by: Chris Llanwarne <cjllanwarne@users.noreply.github.com> Co-authored-by: Christian Freitas <christian.j.freitas@gmail.com> Co-authored-by: Saloni Shah <salonipshah11@gmail.com> Co-authored-by: kshakir <kshakir@broadinstitute.org> Co-authored-by: mspector <michael.l.spector@gmail.com> Co-authored-by: Adam Nichols <anichols@broadinstitute.org> Co-authored-by: Brian Reilly <breilly@broadinstitute.org> Co-authored-by: Adam Nichols <aednichols@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Justin Variath Thomas <JVThomas@users.noreply.github.com> Co-authored-by: Christian Freitas <cfreitas@broadinstitute.org> Co-authored-by: Trevyn Langsford <tlangs@broadinstitute.org> Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com> Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> Co-authored-by: Tom Wiseman <tom.h.wiseman@gmail.com> Co-authored-by: Blair Murri <BMurri@users.noreply.github.com> * Develop aws (#29) * stuck on globbing * efs works, no callcaching * update readme * extended EFS support * fix for globbing in nested scatters * updated config for globbing, to prevent issues with empty folders * efs fixes : support paths with over 127 characters, fix delocalization of efs-based globs (#32) * Develop aws (#34) * efs fixes : support paths with over 127 characters, fix delocalization of efs-based globs * add deployment manual, fix issue with empty disks * update documentation * update documentation * update documentation --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Janet Gainer-Dewar <jdewar@broadinstitute.org> Co-authored-by: Katrina P <68349264+kpierre13@users.noreply.github.com> Co-authored-by: Chris Llanwarne <cjllanwarne@users.noreply.github.com> Co-authored-by: Christian Freitas <christian.j.freitas@gmail.com> Co-authored-by: Saloni Shah <salonipshah11@gmail.com> Co-authored-by: kshakir <kshakir@broadinstitute.org> Co-authored-by: mspector <michael.l.spector@gmail.com> Co-authored-by: Adam Nichols <anichols@broadinstitute.org> Co-authored-by: Brian Reilly <breilly@broadinstitute.org> Co-authored-by: Adam Nichols <aednichols@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Justin Variath Thomas <JVThomas@users.noreply.github.com> Co-authored-by: Christian Freitas <cfreitas@broadinstitute.org> Co-authored-by: Trevyn Langsford <tlangs@broadinstitute.org> Co-authored-by: Miguel Covarrubias <mcovarr@users.noreply.github.com> Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> Co-authored-by: Tom Wiseman <tom.h.wiseman@gmail.com> Co-authored-by: Blair Murri <BMurri@users.noreply.github.com> Co-authored-by: geertvandeweyer <geertvandeweyer@gmail.com>
- Loading branch information
20 people
authored
Sep 18, 2023
1 parent
7043c10
commit 8395859