Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restore mononoke getdeps integration tests #951

Closed
wants to merge 3 commits into from

Commits on Sep 19, 2024

  1. project project_install_prefix for generate-github-actions

    Summary:
    fix getdeps generate-github-actions byt adding missing process_project_dir_arguments.  Without this couldn't specify any project_install_prefix other that the default /usr/local
    
    mononoke installs directly to getdeps inst dir so needs a project_install_prefix of / for the artifact discovery to work
    
    regenerated the gh actions with `python3 fbcode/opensource/fbcode_builder/getdeps.py generate-github-actions mononoke --os-type=linux --allow-system-packages --output-dir fbcode/eden/oss/.github/workflows --job-file-prefix mononoke_ --job-name-prefix "Mononoke " --free-up-disk --project-install-prefix mononoke:/`
    
    Differential Revision: D63031271
    ahornby authored and facebook-github-bot committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    a449166 View commit details
    Browse the repository at this point in the history
  2. restore sapling getdeps so eden/mononoke can depend on it (facebook#950)

    Summary:
    Bring a manifest for sapling back so we can test eden fs and mononoke. Their tests need the hg binary and thus a sapling manifest to depend upon
    
    How:
     * started with backout of 73302ef and badf6d1
    
    Pull Request resolved: facebook#950
    
    Test Plan:
    Setup a local ubuntu 22.04 toolbox with:
    ```
    toolbox create --distro ubuntu --release 22.04
    toolbox enter ubuntu-toolbox-22.04
    sudo apt update
    sudo apt install g++ libssl-dev pkg-config
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    ```
    
    Build locally with: `python3 build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. sapling  --project-install-prefix sapling:/usr/local`
    
    Test locally with: `python3 build/fbcode_builder/getdeps.py --allow-system-packages test --src-dir=. sapling  --project-install-prefix sapling:/usr/local`
    
    Generate github actions with:
    
    `./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "Sapling CLI Getdeps " --job-file-prefix=sapling-cli-getdeps_ sapling`
    
    CI
     ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/950).
    * facebook#951
    * __->__ facebook#950
    
    Differential Revision: D62978511
    ahornby authored and facebook-github-bot committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    8a4b799 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. restore mononoke getdeps integration tests (facebook#951)

    Summary:
    X-link: facebookincubator/zstrong#995
    
    Bring back mononoke getdeps integration tests.  This is a refesh of the previously working export-D34186407 branch from 2022
    
    Main changes since:
     * depend on restored sapling manifest from previous commit
     * bring back selected manifests needed by mononoke that were deleted in D51869247. I added the sqlite binary packages from this to the sqlite3 manifest rather than bring back the sqllite3-bin manifest
     * add manifests for new tools used: ripgrep, git-lfs, and zstd cli (we already had the zstd libs)
     * fix a few test expecations that where too closely tied to git cli or TLS version etc (ubuntu 22.04 is on older version)
     * getdeps MakefileBuilder.run_tests() improvements
       * fix error status reporting, was not failing if tests failed
       * pass of --num-jobs to tests to stop it OOMing my machine
       * pass of --filter to tests so can iterate on one test more easily
    
    Can to iterate local execution for one test with:
    ```
    python3 ./build/fbcode_builder/getdeps.py test --num-jobs 4 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration --retry 0 --filter server/test-gettreepack.t
    ```
    
    Pull Request resolved: facebook#951
    
    Test Plan:
    Setup a local ubuntu 22.04 toolbox with:
    ```
    toolbox create --distro ubuntu --release 22.04
    toolbox enter ubuntu-toolbox-22.04
    sudo apt update
    sudo apt install g++ libssl-dev pkg-config
    sudo locale-gen en_US.UTF-8
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    ```
    
    Build dependencies:
    ```
    python3 ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke_integration
    python3 ./build/fbcode_builder/getdeps.py build --allow-system-packages --no-facebook-internal --src-dir=. --no-tests sapling
    python3 ./build/fbcode_builder/getdeps.py build --allow-system-packages --no-facebook-internal --src-dir=. --no-tests mononoke
    ```
    
    then test locally:
    ```
    ./build/fbcode_builder/getdeps.py build --no-deps --num-jobs 16 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration
    ./build/fbcode_builder/getdeps.py test --num-jobs 6 --allow-system-packages --no-facebook-internal --src-dir=. mononoke_integration
    ```
    
    Generate the github actions yaml
    ```
    ./build/fbcode_builder/getdeps.py generate-github-actions --allow-system-packages --test --os-type linux --job-file-prefix mononoke-integration_ --job-name-prefix "Mononoke Integration " --output-dir .github/workflows mononoke_integration --num-jobs 4
    ```
     ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/951).
    * __->__ facebook#951
    * facebook#950
    
    Differential Revision: D62978526
    
    Pulled By: ahornby
    ahornby authored and facebook-github-bot committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    8f4dd4e View commit details
    Browse the repository at this point in the history