Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

taplo-cli very slow at collecting files #619

Open
drahnr opened this issue Jun 12, 2024 · 6 comments · May be fixed by #640
Open

taplo-cli very slow at collecting files #619

drahnr opened this issue Jun 12, 2024 · 6 comments · May be fixed by #640

Comments

@drahnr
Copy link
Contributor

drahnr commented Jun 12, 2024

Our project grew and taplo evolved. Trying 0.9.0 and 0.9.1 (unreleased, git) both exhibit a rather slow glob resolution and file collection. It hangs for minutes.

EDIT: it doesn't finish collecting the files at all

@panekj
Copy link
Collaborator

panekj commented Jun 12, 2024

It's hard to act in any way without details about the project, how many files, how is taplo used, etc.

@drahnr
Copy link
Contributor Author

drahnr commented Jun 12, 2024

I think it be great to have some addition tracing::trace! level logs in there.

I added some and it turns out self.env.glob_files_normalized(&pat) doesn't return but gets hung up. There are 71 files that match the regex ^.+\.toml$, the repository contains 5 symlinks, 0 of them are toml files.

@drahnr
Copy link
Contributor Author

drahnr commented Jun 12, 2024

After deleting my arguably hefty target dir, it goes back to normal.

tl;dr it might make sense to honour .gitignore if present to avoid scanning temporary build directories.

@panekj
Copy link
Collaborator

panekj commented Jun 12, 2024

We already have an exclude configuration option that can resolve that.
Reading .gitignore might introduce unwanted behaviour of not formatting files that are ignored but not expected to be.

@drahnr
Copy link
Contributor Author

drahnr commented Jun 13, 2024

Well, now I have to define two places where I have globs of ignores. I personally find that more unexpected vs taking into account .gitignore files.

@ejpcmac
Copy link

ejpcmac commented Jul 22, 2024

I am facing the same issue currently. I am using Nix with direnv in my project, so this creates a .direnv/ directory containing symbolic links to directories that contain a lot of files. This directory is in the .gitignore, and I would have expected taplo not to look into it.

Even adding manually this directory to the taplo.toml does not speed-up the collection: it still goes through all the files and then filters them. The only solution I’ve found to speed-up the collection is to add an include directive with manual addition of all subdirectories where I expect to have formatted TOML files. This is however not optimal since what I’d like is just to actually exclude a few directories, and would expect taplo not to try collecting files from them.

@drahnr drahnr linked a pull request Jul 30, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants