Support globs in CLI

[1j01]

Support e.g. html-beautify -r *.html (or htm in my case) to replace all matching files in a directory.
Also, the commands js-beautify -r, html-beautify -r and css-beautify -r could default to replacing *.js, *.htm?(l) and *.css respectively.
(Use glob)

[1j01]

I've used glob-run as a workaround in the meantime.

[billymoon]

+1

[jwhitmarsh]

+1

[bitwiseman]

Pull requests welcome!

[1j01]

Well, I only needed to beautify some horrible HTML one time, so count me out. 😐

[rogah]

+1

[prettydiff]

Here is how to go about this, presuming Node:

I would provide an option to specify the depth a user want to traverse. In my app I have an option called readmethod that supports values: screen, file, filescreen, directory, and subdirectory. The screen value means to read from input on the console and output to the console. The filescreen value means to read from a file and output to the console. These first two are convenient for testing. The value file means to read from a file and output to a file. The directory value means to read from a single directory only and process those files. The subdirectory option provides some recursion so that subdirectories are traversed and this directory tree is preserved. In my own app I output against these last three values according to an option named output, which requires the user to specify a location even if it is the same location as the input.

First you will need to read the starting directory with fs.readdir. The callback will return an array list of files. You may want to prune this list so that you are only working with certain files you deem acceptable for processing. I don't prune the file list in my own app though. As you are looping through the file list you will want to read the files and gather some data. I gather local relative path, absolute path, file size, read file contents, whether or not it is text or binary, and probably some other stuff that is helpful later on.

You will want to recursively repeat this logic on each child directory if you are crawling directory trees. You probably won't have to collect as much information as I do. In a diff utility I find that I have to deal with two parallel directory structures and be able to compare files appropriately by both path and file name asynchronously, which means accounting for all kinds of extra data to avoid race conditions, collisions, and ensure a comparison occurs as early as possible without waiting or polling.

You will need to determine if the files are binary or text even though your application won't process the contents of binary files. I have found that fs.readFile is much easier and generally faster, but doesn't work well on binary files. For binary files you have to open a stream with fs.open and then read the stream with fs.read only once fs.open is complete. This is necessary because if you are going to clone a given directory into a new location it is wise to bring all the code you are beautifying as well as all the other files you are not beautifying, for integrity, which means writing untouched text and binary into files of the correct file names.

Once you have read the files you will need to beautify them and then write them back into a file. fs.writeFile is great but for the binary files you are safer using fs.write. I just noticed, by the way, in the latest Node documentation fs.write is mentioned twice... weird.

You will also want to know when the operation is complete, which is determined by the writing of the last file. This doesn't mean the last file you read or the last file you populated into some array. It just means when you are writing that one file and no other files are pending a write. This is tricky in a fully asynchronous operation, where various steps execute in unpredictable orders and various operations are occurring simultaneously in wildly different durations. Ultimately, you will want to be just precise enough to know when the fs.write/fs.writeFile operation starts on the last item in queue without regard to whether this is the last item to finish writing. I have tried a couple of times to get this correct, but its pretty challenging.

If you are writing directory structures into a new location that means making directories. You will want to create a system so that new subdirectories are created as needed, but not more than that as it will throw an error.

Something else that has gotten me into trouble is knowing where you are in the file system versus where you want to execute to (reading/writing). You will want to ensure you can read from absolute paths, local paths, and global paths (in the case of an NPM global installation). Getting both the input and output locations correct for each of these three scenarios can be tricky and open lots of regression. Just have good unit tests to account these issues. I kept getting this wrong so much that I had to write a file system simulator to perform all these tasks as unit tests to verify integrity at each step of enhancement.

You could look to my code as an example api/node-local.js but I would hold off on that until you get really frustrated as my code is far more over-engineered that what you probably want.

[1j01]

@prettydiff For traversing the filesystem there's glob (as I mentioned subtly when opening this issue). For async operations in JS (and waiting for the end of all of them) there's async. For making directories when needed there's mkdirp and others, but I was just thinking it would modify files in-place.

[RoHePu]

+1

[taji]

FYI: If using glob-run inside an npm script, make sure to enclose the glob pattern in single quotes as so:

"html-format": "glob-run html-beautify -r '**/*.html'",

As described in detail here:

https://medium.com/@jakubsynowiec/you-should-always-quote-your-globs-in-npm-scripts-621887a2a784

Lost 2 hours on this one critical detail.