Checkout step fails on macOS

[jagthedrummer]

• I have read Contribution Guidelines.
• I have checked for similar issues and haven't found anything relevant.
• This is not a security issue (which should be reported here: https://circleci.com/security/)

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

circleci local execute --job build
...
====>> Checkout code
  #!/bin/bash -eo pipefail
mkdir -p /tmp/circleci/mothership && cd /tmp/_circleci_local_build_repo && git ls-files -z | xargs -0 tar -c | tar -x -C /tmp/circleci/mothership && cp -a /tmp/_circleci_local_build_repo/.git /tmp/circleci/mothership
xargs: tar: terminated by signal 13
Error: Exited with code 125
Step failed

Watching the /tmp dir it doesn't look like /tmp/_circleci_local_build_repo is ever created or populated with anything.

What is the expected behavior?

Checking out the code should work.

Which version of the CLI and OS are you using? Did this work in previous versions?

$ circleci version
0.1.4786+bad101f

$ circleci diagnostic
---
CircleCI CLI Diagnostics
---
Debugger mode: false
Config found: /Users/jgreen/.circleci/cli.yml
API host: https://circleci.com
API endpoint: graphql-unstable
OK, got a token.
Trying an introspection query on API... 
Ok.
Hello, Jeremy Green.

[LachlanStuart]

This is also broken on Linux, even on the latest CLI version (0.1.5314+26252a9) and build-agent version (sha256:65b2102646d5658f892e0ad8253b7912c676126c857c87c8c12460f0aa4f5aa1). The logs look the same.

I've had a previous issue with checkout and the problem was actually in CircleCI's build-agent image, which is in a private repo. The workaround from my previous issue seems to work for this issue as well.

[jagthedrummer]

Thanks for the tip @LachlanStuart! That got me past the checkout step and on to my next hurdle.

[eddiewebb]

@jagthedrummer - thank's for submitting this issue!

1. can you confirm that creating /tmp/_circleci_local_build_repo manually first corrects the issue?
2. please confirm that your user has rights to create folders in tmp (above step without sudo would confirm)

[LastZactionHero]

I'm seeing this issue exactly, but digging in seemed to be an issue with the maximum byte length of xargs. I noticed this by instead saving the tar to a file and seeing what's inside: it was just the last third of my directory structure.

I maxed it out the -s flag. Got me through the day:

- run:
  command: mkdir -p /root/project && cd /tmp/_circleci_local_build_repo && git ls-files -z | xargs -0 -s 2090860 tar -c | tar -x -C /root/project && cp -a /tmp/_circleci_local_build_repo/.git /root/project

[partikus]

@LastZactionHero you are right. I've got the same issue for large project using local executor on linux.

-s max-chars
Use at most max-chars characters per command line, including the command and 
initial-arguments and the terminating nulls at the ends of the argument strings. 
The largest allowed value is system-dependent, and is calculated as the argument 
length limit for exec, less the size of your environment, less 2048 bytes of headroom. 
If this value is more than 128KiB, 128Kib is used as the default value; 
otherwise, the default value is the maximum. 1KiB is 1024 bytes.

https://linux.die.net/man/1/xargs

The simplest workaround is:

checkout: &checkout
  checkout

localCheckout: &localCheckout
  run: |-
    PROJECT_PATH=$(cd ${CIRCLE_WORKING_DIRECTORY}; pwd)
    mkdir -p ${PROJECT_PATH}
    cd /tmp/_circleci_local_build_repo 
    git ls-files -z | xargs -0 -s 2090860 tar -c | tar -x -C ${PROJECT_PATH} 
    cp -a /tmp/_circleci_local_build_repo/.git ${PROJECT_PATH}

# ...
jobs:
  build:
    steps:
      - *checkout
  local:
    steps:
      - *localCheckout

but would be great to have possibility to customize the command.

[esselius]

Make sure that you have this set:

$ cat ~/.circleci/build_agent_settings.json
{"LatestSha256":"sha256:93fc98c6957f7dc9bae552701885c4521d942707e8cc63301a4b05383356f0af"}

[suecharo]

It's running!

version: 2
jobs:
  build:
    machine: true
    steps:
      - run:
          name: "Git clone"
          command: "git clone $CIRCLE_REPOSITORY_URL /tmp/_circleci_local_build_repo"
      - checkout
      - run :
          name: "ls"
          command: "ls -lah"

[alexkli]

Update: This actually checks out the remote code, master branch. It does not use the local code from the local working directory, which is what you normally want from circelci build. Trying to find a better fix...

---

Had the same issue. Here is @suecharo's fix adjusted a bit so that it only applies to local builds:

version: 2
jobs:
  build:
    machine: true
    steps:
      # fix for local builds https://github.com/CircleCI-Public/circleci-cli/issues/281
      - run: |
          if [[ ${CIRCLE_SHELL_ENV} =~ "localbuild" ]]; then
            git clone $CIRCLE_REPOSITORY_URL /tmp/_circleci_local_build_repo
          fi
      - checkout
      - run: echo "hello"

[alexkli]

See #330 (comment) for a working fix, at least for the /tmp/_circleci_local_build_repo: No such file or directory issue.

[marcomorain]

#330 is a duplicate of this issue as far as I can tell.

[marcomorain]

Logged as CIRCLE-25051.