Skip to content

Test workflow sometimes fails on parallelized initiation #167

Closed
@ajnelson-nist

Description

@ajnelson-nist

This Issue pertains to a Make workflow matter that only occurs on fresh git clones of the CASE repository, and is mitigable with a command repetition (which for some users is a 2-keystroke solution: up-arrow, return). This is not an ontology matter, so is planned to be processed without a vote, merged after the 2024-12-10 meeting.

Disclaimer

Participation by NIST in the creation of the documentation of mentioned software is not intended to imply a recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that any specific software is necessarily the best available for the purpose.

Bug description

A parallel execution issue is present that sometimes arises when running make -j (a.k.a. make --jobs) in a fresh clone of the CASE repository, where a dependent file of the virtual environment wasn't created yet because of a missing dependency on submodule initialization. make without -j, which is how CASE's CI runs, has no problem because of the dependency ordering guaranteeing the dependent file (.../requirements.txt) would exist when the dependencies are addressed in order.

Steps to reproduce

This can be run in a fresh terminal, and is likely to note there is no rule to make dependencies/UCO/requirements.txt.

pushd `mktemp -d` ;
  git clone https://github.com/casework/CASE.git ;
  pushd CASE ;
    make -j 2 >/dev/null ;
  popd ;
popd

Mitigation

Running make -j again immediately after failure (or as a sequenced second command) seems to typically end up working, as the Git submodule initialization recipe runs to completion before make exits from the unsatisfied dependency. That recipe creates the file, satisfying the dependency from the first run.

Resolution

Have dependencies/UCO/requirements.txt depend on the submodule initialization flag file (.git_submodule_init.done.log).

Coordination

  • Tracking in Jira ticket OCCASE-497
  • Administrative review completed, proposal announced to Ontology Committees (OCs) on 2024-11-22
  • Requirements development phase skipped.
  • Solutions development phase skipped.
  • Backwards-compatible implementation merged into develop for the next release
  • develop state with backwards-compatible implementation merged into develop-2.0.0
  • Backwards-incompatible implementation merged into develop-2.0.0 (N/A)
  • Milestone linked
  • Documentation logged in pending release page

Metadata

Metadata

Assignees

No one assigned

    Labels

    Project-BugSomething isn't working.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions