Anonymize secrets data that will end up in report

[plocket]

Does that mean complete anonymity, or do we include the first three chars of email and the last three chars of password, etc?

[BryceStevenWilley]

IMO given that reports are public for public github repos, they should be completely anonymous, just for extra safety. I'm also not sure this needs to be required, as most emails that I give to tests are fake emails, such as example@example.com, and don't need that to be censored. I don't think I'd suggest using live emails in an automated test, especially since we can't verify if the emails been sent from our side.

[michaelhofrichter]

I just noticed this as well in troubleshooting a failed report. Step one of my tests is to log in to the server using two of the github secrets. I just checked the test results and noticed that it listed my username and password in plaintext.

[BryceStevenWilley]

Realizing that this might be a smaller scope than I thought, I thought we were trying to censor all data entered into email and password boxes, but this seems to be an issue in just one line?

@michaelhofrichter I'd be curious about your thoughts here. IMO it's easier (and safer) to just not print that data at all, instead of censoring it by:

1. replacing all characters (except the @ in the email maybe) and still leaking the length of the email / password
2. printing the same length ******** password string, which is misleading if someone really is trying to see if their password is what they expected it to be.

Would you rather have just "Logged in ... successfully", or would you rather have some ********@*** to make it clear it's the email sign in?

[plocket]

Jus to clarify, this is just for the login step.

[plocket]

Also, if we did show some of the names, we definitely wouldn't use the length of the actual secrets. We'd use three stars no matter what.

It occurs to me that we also want to avoid taking a screenshot of the failed page since one of the inputs might be right. That means there's no way for the developer to confirm what they put in.

Maybe that's just the way it has to be. They can always re-enter the secrets to make sure they've put the right ones in.

[We might want to think about how to handle screenshots of errors for setting other secrets. If a secret is set in an input field and the page errors, there will be a screenshot of it. Maybe we can disable screenshots until the page navigates away where that's handled.]

[michaelhofrichter]

@BryceStevenWilley Sorry for the delay in my responses. I certainly don't need a length or anything to show in the report. If the password's wrong, I'm going to have to update the secret. I certainly can't edit it.

[plocket]

Closed by #562