ghorg extremly slow to clone and errors out on many repos

[dasunsrule32]

Describe the bug
Extremely slow at pulling down repos and fails with many 128 or 1 error codes. I can manually clone the repos without issue.

To Reproduce
Steps to reproduce the behavior:

1. ghorg clone org

Expected behavior
It should be fast to iterate through the repos and clone them and should do so without errors.

Terminal output

============ Issues ============

Problem trying to pull master Repo: git@github.com:ORG/repo1.git Error: exit status 1
Problem trying to pull SCHOLAR-3.x Repo: git@github.com:ORG/repo2.git Error: exit status 1
Problem trying to pull master Repo: git@github.com:ORG/repo3.git Error: exit status 1
Problem trying to pull 1_1 Repo: git@github.com:ORG/repo4.git Error: exit status 1
Problem trying to pull master Repo: git@github.com:ORG/repo5.git Error: exit status 1
Problem trying to pull master Repo: git@github.com:ORG/repo6.git Error: exit status 1
Problem trying to pull master Repo: git@github.com:ORG/repo7.git Error: exit status 1
Problem trying to pull master Repo: git@github.com:ORG/repo8.git Error: exit status 1
Problem trying to pull master Repo: git@github.com:ORG/repo9.git Error: exit status 1
Problem trying to pull main Repo: git@github.com:ORG/repo10.git Error: exit status 1
Problem trying to clone Repo: git@github.com:ORG/repo11.git Error: exit status 128
Problem trying to clone Repo: git@github.com:ORG/repo12.git Error: exit status 128

Desktop (please complete the following information):

• OS: Fedora 34
• Go Version: go version go1.16.5 linux/amd64
• Git Version: git version 2.31.1
• Ghorg Version: 1.6.0
• Ulimit: DefaultLimitNOFILE=1024:524288

Additional context
I installed using the following command: go install github.com/gabrie30/ghorg@latest. I have this setup on Amazon Linux 2 as well and it works as expected, quick and efficient. It's using golang 1.15* however. It is oddly enough using ghorg 1.6.1, compared to the version getting pulled to the Fedora install, 1.6.0.

[gabrie30]

What happens if you run ghorg clone org --concurrency=1

It will be slower but wondering if you still see those errors.

[dasunsrule32]

So I was doing --concurrency=10 yesterday and it seemed to be doing better, but it's still extremely slow. I'm starting to think it my be something with open file handles, because when ghorg is running, I can't do a simple git pull in a separate repo that is ignored. I'm looking at that as well, since Fedora doesn't seem to respect /etc/security/limits.conf. I'm open to suggestions as well. I'll also update as I figure out anything as well.

[dasunsrule32]

So I was able to up my file limits in /etc/systemd/user.conf and that didn't help:

DefaultLimitNOFILE=65535:524288

I've verified that I'm using SSH as well. git and gh are both using SSH when I do a pull or clone and they are working quickly.

Adding the following to my ~/.ssh/config helped on the git side of things, but not on the ghorg side.

Host *
   AddressFamily inet

I've watched system metrics as well, and it's barely breaking a sweat during all of this.

I'm also still cannot peform a git pull on other repos when ghorg is running. This is an odd issue...

[gabrie30]

Does it complete successfully when you go all the way down to --concurrency=1

[dasunsrule32]

So I shall hang my head in shame...

I checked my Unifi UDM Pro and Threat Management was blocking lots of outbound ssh calls to Github. I whitelisted Github and it's working as intended now. I apologize for taking up your time.

[gabrie30]

Thanks for following up...I should add that to the troubleshooting section in the readme, no need to apologize for anything!

[dasunsrule32]

Thanks for being understanding haha 😃

Looking at the issue now, I'm actually impressed that ghorg kept on working at all with what it was dealing with. 😅

[tfrijsewijk]

I'm running windows 10 and I also had status error 1, looks like because of "filenames too long". I cloned closer to root using --path D:\g and I got away with it