Add diff in output of --check (for CI use cases)

[muescha]

More people submit PR for documents in Markdown files or other things via Github web editor. If they make a mistake then the CI fails but there is no chance to inform the user whats wrong.

I see some infos should be with the --debug-check but currently there is no output with this option.

Environments:

• Prettier Version: 1.18.2
• Running Prettier via: cli
• Runtime: node?
• Operating System: linux

Steps to reproduce:

prettier "**/*.{md,css,scss,yaml,yml,ts}" "--check"

Expected behavior:

Show each error and the changes like with --debug-check here in #4415

Actual behavior:

Show nothing about the errors:

> @ prettier /home/circleci/project
> prettier "**/*.{md,css,scss,yaml,yml,ts}" "--check"

Checking formatting...
docs/contributing/environment.md
Code style issues found in the above file(s). Forgot to run Prettier?

Also there are no more infos available with this commands:

prettier "**/*.{md,css,scss,yaml,yml,ts}" "--debug-check"
prettier "**/*.{md,css,scss,yaml,yml,ts}" "--debug-check" "--check"

[dylannz-sailthru]

"Forgot to run Prettier?" is really not a very helpful message...

[thorn0]

The message contains the file names. What else do you need to run Prettier on those files? ¯\_(ツ)_/¯

[lydell]

I personally like the output of --check very much.

CI: "Forgot to run Prettier?"

Me: "Oh yeah, I guess I did!" Runs Prettier.

CI: "All good!"

[muescha]

I like to point again to:

more people submit PR for documents in Markdown files or other things via Github web editor.

That means they have no local IDE installed. That's why they can not run prettier on the file.

[muescha]

Expected behavior would be a detailed error message or a diff

[error] src/index.js: ast(input) !== ast(prettier(input))
[error] Index:
[error] ===================================================================
[error] ---
[error] +++
[error] @@ -17,6 +17,6 @@
[error]                    "method": false,
[error]                    "key": {
[error] -                    "type": "StringLiteral",
[error] -                    "value": "asd"
[error] +                    "type": "Identifier",
[error] +                    "name": "asd"
[error]                    },
[error]                    "computed": false,
[error]
[error] Index:
[error] ===================================================================
[error] ---
[error] +++
[error] @@ -1,1 +1,1 @@
[error] -const test = { "asd": 1 };
[error] +const test = { asd: 1 };
[error]

[alexander-akait]

Why not run prettier \"{**/*,*}.{js,json,md,yml,css}\" --list-different" using npm/yarn script?

[muescha]

thx. I would try on monday the output of --list-different

[lydell]

--list-different also just lists file names – I don’t think it’ll make any difference for @muescha.

@muescha I think this proposal might be solving your problem the wrong way. Forcing people to click into a CI build log, find diffs of expected output and then manually applying them does no sound very friendly. I think we could try several other things:

• Encourage people to clone the project rather than using the web editor.
• Use --prose-wrap preserve or --prose-wrap never to reduce chance of needing to run Prettier.
• Running Prettier in the web editor via https://github.com/prettier/prettier-chrome-extension.
• A bot that automatically runs Prettier on PRs.
• You could pull people’s branches, run Prettier yourself and push to their PR (contributions from maintainers is generally enabled).
• Not use Prettier for markdown files if you get a lot of contributions to them via the web editor.

[muescha]

• Encourage people to clone the project rather than using the web editor.

thats much overhead :(

• Use --prose-wrap preserve or --prose-wrap never to reduce chance of needing to run Prettier.

it should run normal prettier

• Running Prettier in the web editor via prettier/prettier-chrome-extension.

this is only the solution for one user, and not for all

• A bot that automatically runs Prettier on PRs.

this changes the PR - the user should have information about the problems

• You could pull people’s branches, run Prettier yourself and push to their PR (contributions from maintainers is generally enabled).

to much overhead to checkout a branch etc

• Not use Prettier for markdown files if you get a lot of contributions to them via the web editor.

also prettier is a good help

the only thing is that there should be an more verbose error message :/

[lydell]

Thanks for the responses!

You didn’t seem to respond to this one, though:

Forcing people to click into a CI build log, find diffs of expected output and then manually applying them does no sound very friendly.

Thoughts?

[muescha]

since they edit the PR on github and check why the CI build is failing they already looking into the CI logs. and then when they only find a not helpful message - that is frustrating.

example gatsbyjs/gatsby#18719 (comment)

I checked what you suggested and lines where only word to highlight is "false" (without quotes or 0 are highlighted with no issues.

PS. @muescha is there a way to get rid of prettier complains in build errors? I am editing this in the browser.

[dylannz-sailthru]

I was making changes to a repository that'd had prettier added to it without me realising, but I had not installed it in my local development environment. So naturally it blew up in CI and asked if I'd "forgotten to run prettier". But that message came from a program called "prettier", which was pretty confusing - yes I suppose I'd "forgotten" to run prettier (I hadn't actually, I didn't even know I had to. Unnecessarily snarky error message, if you ask me...) but even more annoying was that the prettier program was printing that message!

exit 1 would have been just as helpful, and less condescending. A list of formatting violations would have been a lot better though!

[thorn0]

@dylannz-sailthru Would you really prefer to get a diff instead like @muescha has proposed and apply it manually instead of running the formatter?

[dylannz-sailthru]

@dylannz-sailthru Would you really prefer to get a diff instead like @muescha has proposed and apply it manually instead of running the formatter?

I admit, I've now set up my IDE with prettier and now it doesn't bother me. But it seems there is a use case where someone wants to make a small change and doesn't have prettier installed (either on GitHub or in a local development environment), adds an extra space somewhere, and now has to install a 3rd party program and run it instead of just fixing a small error. And while I'll admit it's a narrow use case, I can't see much of a downside to being more verbose (or at least having a configurable verbose flag). Most CI tools will collapse each CI step so you can view the logs of each one in isolation.

[muescha]

example where i used the GitHub editor and i get for this change gatsbyjs/gatsby@7b78781 in gatsbyjs/gatsby#19822 this CI log message:

https://circleci.com/gh/gatsbyjs/gatsby/283065?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

now i have to checkout ~500MB data from pull request, install prettier only just to see whats wrong in this commit and i can change the file?

[thorn0]

@dylannz-sailthru Locally, extra setup isn't needed. Prettier is installed as a dev dependency and there is usually a command in the project to run all the linters, formatters, etc. (e.g. npm run format). This means we're discussing only the web UI use case here.

[lydell]

Unnecessarily snarky error message, if you ask me...

Sorry, the message is actually not supposed to be snarky, but friendly! Do you have ideas for better wording?

example gatsbyjs/gatsby#18719 (comment)

I don’t think that’s a good example. That’s not a markdown change, but a code change! Is that really something people do in the browser? That sounds terrible to me – how do you debug and test then?

---

Here’s yet one idea: When editing a markdown file in the GitHub editor and are about to hit the Commit button – first copy-paste the entire file into https://prettier.io/playground/ and then copy back the output.

[muescha]

I don’t think that’s a good example. That’s not a markdown change, but a code change! Is that really something people do in the browser?

yes - people do it. so why not also on all files.

Here’s yet one idea

that maybe a workaround but not a solution and also not fit with the project settings for prettier

[dylannz-sailthru]

Sorry, the message is actually not supposed to be snarky, but friendly! Do you have ideas for better wording?

Sure! Something like: You need to run `prettier fix` on these files to ensure they are formatted correctly.

This way it's telling me more specifically what I need to do to get past the error. I think it doesn't matter so much for people who have used prettier before, but as someone who hadn't used it before I found it a little confusing.

Here’s yet one idea: When editing a markdown file in the GitHub editor and are about to hit the Commit button – first copy-paste the entire file into https://prettier.io/playground/ and then copy back the output.

Actually that'd be a great thing to link to in the error message as well - I'd have found it really useful at the time :)

[grncdr]

I would also like to see a diff or more detailed information about why prettier thinks a file isn't formatted. I currently don't run prettier --check in CI/pre-commit hooks because it tells me that files I've just formatted with Prettier are still not formatted. I really don't know how to begin fixing the problem when I can't see what it is.

[lydell]

@grncdr It tells you which files aren’t formatted, so all you need to do is run Prettier on those files. Or on your whole project – probably prettier --write .. Can you expand on how seeing a diff helps?

[thatsIch]

I found, that sometimes there are false positives or new situations where you do not want the code formatted like that. There is no way, that --write is 100% correct all the time. There is no feedback loop if you want to learn the rules.

#12598 would solve the conflict.

It does not modify the default "way" of prettier.

[joonashak]

Struggling with an issue where locally running prettier --check yields no errors and prettier -w yields no diff, yet in GitHub Actions, using Ubuntu 22, prettier --check fails on a single tsconfig.json.

I think this is definitely an example of a legit use case for getting a diff printout straight from Prettier. Yes, it can be achieved with a combination of prettier -w and git diff but that's assuming Prettier is used in a Git repo which might not always be the case.

[siawyoung]

Struggling with an issue where locally running prettier --check yields no errors and prettier -w yields no diff, yet in GitHub Actions, using Ubuntu 22, prettier --check fails on a single tsconfig.json.

I think this is definitely an example of a legit use case for getting a diff printout straight from Prettier. Yes, it can be achieved with a combination of prettier -w and git diff but that's assuming Prettier is used in a Git repo which might not always be the case.

Oh man, this is happening to me now. It only happens for tsconfig.json files too! Not sure why.

EDIT: looks like it's been fixed in 3.2.5, thanks! @joonashak

https://github.com/prettier/prettier/blob/main/CHANGELOG.md#use-json-parser-for-tsconfigjson-by-default-16012-by-sosukesuzuki

[mesmere]

I don't even want this for CI usecases, I'm just trying out prettier and want to see what it'll do to my code before I clobber it all.

I could try to figure out the right incantations to get git to stash the changes I'm working on atm without stashing the prettier config itself, etc etc, so that git can revert --write's changes if I don't like them but it's a little silly that there's not just a dry run option in prettier itself. :x

[alexey-sh]

@mesmere have you considered to opt out prettier?

[webark]

@alexey-sh BLASPHEMER!! 😂🙃

Another option would be to use the eslint plugin, which I think will show the errors when you run it through that?

https://github.com/prettier/eslint-plugin-prettier

[jballands]

Another option would be to use the eslint plugin, which I think will show the errors when you run it through that?

I want to note that some folks feel that integrating Prettier into your linter is suboptimal:

• You end up with a lot of red squiggly lines in your editor, which gets annoying. Prettier is supposed to make you forget about formatting – and not be in your face about it!
• They are slower than running Prettier directly.
• They’re yet one layer of indirection where things may break.

IMHO, whatever works best for your workflow is what you should do. I don't know if there's a perfect answer here, but using the eslint-plugin-prettier, as suggested above, will get you what you're looking for. I would not expect the Prettier team to integrate this functionality; it feels quite intentional.

[vjekoart]

Is there at least an explanation why the Prettier team doesn't want to add --dry-run, --diff or any other similar option to the CLI?

I understand that Prettier is built with certain philosophy in mind, but there were some pretty valid points in this thread why one would benefit from verbose explanation of style errors.

[lvjp]

Hello @lydell,

It appears that @muescha has stopped the fight...

In one of your reply you said :

Forcing people to click into a CI build log, find diffs of expected output and then manually applying them does no sound very friendly.

There are plenty of people who work in this way, what's the matter?

But this is not the only one use case.

When you use a linter collection app like the GitHub super-linter.
If the CI end's up with an error and asks you to use the --write argument.
You can either refactor your CI pipeline to add the --write argument and git diff command to show the diff, or force the developer to install it on the local environment.

What if you would rather not install it in your environment for security reason or simply to not handle updates...

What if you check not yet committed code, you may not want to have some changes done from a linter.

[lydell]

@lvjp I wrote that 5 years ago. Since then I’ve changed my mind:

#6885 (comment)
#6885 (comment)
#6885 (comment)

(You might also have noticed that I’m not actively involved with this project anymore.)

[lvjp]

@lydell Please forgive me for bothering you with problems from the past, and thank you for your swift response.

Maybe @fisker has an opinion on the subject?

[muescha]

It appears that @muescha has stopped the fight...

@lvjp I don't give up, but repeating my arguments every month isn't helpful ;)

[Osmose]

I remain ready to rebase and fix up the PR for this once someone with merge access commits to reviewing it 🫡

[lvjp]

I remain ready to rebase and fix up the PR for this once someone with merge access commits to reviewing it 🫡

@Osmose > can you add the link to the PR here ?

[Osmose]

@Osmose > can you add the link to the PR here ?

It's been linked before above, it's #12598

[mark-wiemer]

Bumping @george-gca's solution from January 2024 as it works great for me, seems to be the best workaround available, and didn't get any attention. I adapted the PR pipeline as that fits my workflow best.

In my case, the issue was that git was replacing LF with CRLF. I should've figured as much, but without this tool it was a blind search (git wasn't warning me on my local machine).

Ref sample run: https://github.com/mark-wiemer-org/vscode-autohotkey2-lsp/actions/runs/11063073280/job/30738602614?pr=29.

I've also adapted the code to my repo and removed some emojis and shopify stuff for simplicity and better practice: https://github.com/mark-wiemer-org/vscode-autohotkey2-lsp/actions/runs/11063073280/workflow?pr=29

I was finally able to overcome this limitation with GitHub actions using prettier --write, git diff, and diff2html. I created 2 GitHub actions: one to run on PRs and another to run on pushes. I used git diff -- . ':(exclude)package-lock.json' ':(exclude)package.json' because installing prettier currently changes these 2 files, so it gives a false positive in the diff.

The on push tests if the code is passing prettier --check. If it isn't it creates a git diff as an html file and upload it as an action artifact. The artifact currently is a zip file with the html diff inside, since viewing artifacts directly in browser is currently not supported.

name: Prettier code formatter (Push)

on:
  push:
    branches:
      - master
      - main
  workflow_dispatch:

jobs:
  check:
    # available images: https://github.com/actions/runner-images#available-images
    runs-on: ubuntu-latest
    steps:
      - name: Checkout 🛎️
        uses: actions/checkout@v4
      - name: Setup Node.js ⚙️
        uses: actions/setup-node@v4
      - name: Install Prettier 💾
        run: npm install --save-dev --save-exact prettier @shopify/prettier-plugin-liquid
      - name: Prettier Check 🔎
        id: prettier
        run: npx prettier . --check
      - name: Show diff 📝
        # https://docs.github.com/en/actions/learn-github-actions/expressions#failure
        if: ${{ failure() }}
        run: |
          npx prettier . --write
          git diff -- . ':(exclude)package-lock.json' ':(exclude)package.json' > diff.txt
          npm install -g diff2html-cli
          diff2html -i file -s side -F diff.html -- diff.txt
      - name: Upload html diff
        if: ${{ failure() && steps.prettier.conclusion == 'failure' }}
        uses: actions/upload-artifact@v4
        with:
          name: HTML Diff
          path: diff.html
          retention-days: 3

The on PR one adds a comment to the PR with the html content of the diff. This one I haven't tested yet since it involves this PR being accepted in the repo I commited it and then a next PR failing prettier test.

name: Prettier code formatter (PR)

on:
  pull_request:
    branches:
      - master
      - main
  workflow_dispatch:

jobs:
  check:
    # available images: https://github.com/actions/runner-images#available-images
    runs-on: ubuntu-latest
    steps:
      - name: Checkout 🛎️
        uses: actions/checkout@v4
      - name: Setup Node.js ⚙️
        uses: actions/setup-node@v4
      - name: Install Prettier 💾
        run: npm install --save-dev --save-exact prettier @shopify/prettier-plugin-liquid
      - name: Prettier Check 🔎
        id: prettier
        run: npx prettier . --check
      - name: Show diff 📝
        # https://docs.github.com/en/actions/learn-github-actions/expressions#failure
        if: ${{ failure() }}
        run: |
          npx prettier . --write
          git diff -- . ':(exclude)package-lock.json' ':(exclude)package.json' > diff.txt
          npm install -g diff2html-cli
          diff2html -i file -s side -F diff.html -- diff.txt
      - name: PR comment with html diff
        # https://docs.github.com/en/actions/learn-github-actions/expressions#failure-with-conditions
        if: ${{ failure() && steps.prettier.conclusion == 'failure' }}
        uses: thollander/actions-comment-pull-request@v2
        with:
          filePath: diff.html

[mark-wiemer]

In short, the best workaround I've found is to run prettier --write . && git diff and go from there. Your favorite diff viewer will help too :) That said, prettier should definitely have its own --diff option, it just feels like such a natural and valuable part of any tool like this

[webark]

@mark-wiemer out of curiosity, what's the benefit of doing to do a git diff over just using one of the prettier lint plugins?

https://prettier.io/docs/en/integrating-with-linters.html

[fisker]

I'm fine to add diff, but better to implement it as a "report", user can choose how the result to be reported.

[mwiemer-microsoft]

@webark I'm no prettier expert--if you find something that works and you like it, go for it. Personally I lint my code for everything except formatting, so I'm not the guy to ask. I found git diff to be the simplest way to get what I wanted, but it may not be best for everyone

(same person, now my work acct)

[george-gca]

If you need Prettier for CI cases, one option I just found out is pr-prettier. Apparently it can add a commit to a PR with the prettified code.