404 Not Found when external PR built on AppVeyor due to incorrect documentation

[Sharparam]

AppVeyor build here: https://ci.appveyor.com/project/chroma-sdk/colore/builds/25970637

Example of working (internal) build with the same config here: https://ci.appveyor.com/project/chroma-sdk/colore/builds/25970154

When building PRs that aren't made within the same repo, the Codecov step on AppVeyor fails with a 404 Not Found error.

The PR in question: chroma-sdk/Colore#267

Relevant error output:

========================================
Codecov
========================================
Running Codecov tool with version 6.1.0-PullRequest0267.524.build.171-ubsecsfk on OpenCover result
              _____          _
             / ____|        | |
            | |     ___   __| | ___  ___ _____   __
            | |    / _ \ / _  |/ _ \/ __/ _ \ \ / /
            | |___| (_) | (_| |  __/ (_| (_) \ V /
             \_____\___/ \____|\___|\___\___/ \_/
                                         exe-1.5.0
            
2019-07-14 19:51:44 INF AppVeyor detected.
2019-07-14 19:51:44 INF Git detected.
2019-07-14 19:51:44 INF Project root: C:\projects\colore
2019-07-14 19:51:45 INF Reading reports.
2019-07-14 19:51:45 INF ./artifacts/opencover-results.xml
2019-07-14 19:51:45 INF Uploading Reports.
2019-07-14 19:51:45 INF url: https://codecov.io
2019-07-14 19:51:45 VRB api endpoint: https://codecov.io/upload/v4?branch=pull/267/merge&commit=e7d854d241eca7d5c2fc23ec2f2d494b0147bf37&build=0y1pu3hapat6x69e&tag=&pr=267&name=&flags=&slug=chroma-sdk%2FColore&token=&package=exe-1.5.0&build_url=&yaml=.codecov.yml&job=chroma-sdk%2Fcolore%2F6.1.0-PullRequest0267.524.build.171-ubsecsfk&service=appveyor
2019-07-14 19:51:45 INF query: https://codecov.io/upload/v4?branch=pull/267/merge&commit=e7d854d241eca7d5c2fc23ec2f2d494b0147bf37&build=0y1pu3hapat6x69e&tag=&pr=267&name=&flags=&slug=chroma-sdk%2FColore&token=&package=exe-1.5.0&build_url=&yaml=.codecov.yml&job=chroma-sdk%2Fcolore%2F6.1.0-PullRequest0267.524.build.171-ubsecsfk&service=appveyor
2019-07-14 19:51:45 INF Pinging Codecov
2019-07-14 19:51:45 VRB Trying to upload using HttpWebRequest.
2019-07-14 19:51:46 VRB One or more errors occurred. (The remote server returned an error: (404) Not Found.)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Codecov.Upload.HttpWebRequest.Post() in C:\projects\codecov-exe\Source\Codecov\Upload\HttpWebRequest.cs:line 32
   at Codecov.Upload.Upload.Uploader() in C:\projects\codecov-exe\Source\Codecov\Upload\Upload.cs:line 28
2019-07-14 19:51:46 VRB Uploader failed.
2019-07-14 19:51:46 VRB Trying to upload using WebClient.
2019-07-14 19:51:46 VRB The remote server returned an error: (404) Not Found.
   at System.Net.HttpWebRequest.GetResponse()
   at System.Net.WebClient.GetWebResponse(WebRequest request)
   at System.Net.WebClient.DownloadBits(WebRequest request, Stream writeStream)
   at System.Net.WebClient.UploadBits(WebRequest request, Stream readStream, Byte[] buffer, Int32 chunkSize, Byte[] header, Byte[] footer)
   at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
   at System.Net.WebClient.UploadString(Uri address, String method, String data)
   at Codecov.Upload.WebClient.Post() in C:\projects\codecov-exe\Source\Codecov\Upload\WebClient.cs:line 26
   at Codecov.Upload.Upload.Uploader() in C:\projects\codecov-exe\Source\Codecov\Upload\Upload.cs:line 28
2019-07-14 19:51:46 VRB Uploader failed.
2019-07-14 19:51:46 FTL Failed to upload the report.
   at Codecov.Upload.Uploads.Uploader() in C:\projects\codecov-exe\Source\Codecov\Upload\Uploads.cs:line 35
   at Codecov.Program.UploadFacade.Uploader() in C:\projects\codecov-exe\Source\Codecov\Program\UploadFacade.cs:line 121
   at Codecov.Program.Run.Uploader() in C:\projects\codecov-exe\Source\Codecov\Program\Run.cs:line 51
   at Codecov.Program.Run.Runner(IEnumerable`1 args) in C:\projects\codecov-exe\Source\Codecov\Program\Run.cs:line 19

[AdmiringWorm]

I see that it uses an invalid branch name for the url, and I noticed you are setting the branch name yourself in the cake build script here: https://github.com/chroma-sdk/Colore/blob/develop/build.cake#L358 (which ends up being pull/267/merge on the PR, this needs to be the base branch getting merged into).

I believe this is the cause of the failure. Btw, you should not need to set the branch name yourself when you are running on appveyor (yes I know there was an issue in the past that required to do that, but that bug should have been fixed).

[Sharparam]

Ah, thanks. I'll test without an explicit branch set and see if it resolves it.

…

On Mon, 15 Jul 2019, 08:18 Kim J. Nordmo, ***@***.***> wrote: I see that it uses an invalid branch name for the url, and I noticed you are setting the branch name yourself in the cake build script here: https://github.com/chroma-sdk/Colore/blob/develop/build.cake#L358 (which ends up being pull/267/merge on the PR, this needs to be the base branch getting merged into). I believe this is the cause of the failure. Btw, you should not need to set the branch name yourself when you are running on appveyor (yes I know there was an issue in the past that required to do that, but that bug should have been fixed). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#76?email_source=notifications&email_token=AAEGRNUIPZHFHURWXGNXQEDP7QJEJA5CNFSM4IDR2WZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ4ZQUY#issuecomment-511285331>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEGRNT7ZA3UI5R6S5OK43DP7QJEJANCNFSM4IDR2WZA> .

[AdmiringWorm]

👍

Please do keep me posted, the problem could be elsewhere, but that is the only thing I could see that could cause the problem.

[Sharparam]

The problem seems to persist even when letting the branch be detected automatically.

This AppVeyor build works correctly (non-PR build with the explicit branch setting removed).

This AppVeyor build still fails, even though the branch name now resolves to the same as in the working build.

[AdmiringWorm]

@Sharparam I see now that you are not using the latest version of Codecov-exe, so the first thing I would recommend is to update the version used to 1.7.0.

While I don't think I have fixed any bugs that will help with this, it is better to make sure.
The latest version also outputs the response message from codecov.io if the response is in plain text, which also may give us some clue of why the failure is happening.

However, what I actually think is the problem is that I am not correctly encoding the branch name value, which then causes codecov.io to return a 404 response code. I will put together a build with that assumption when I get home later today (will be around 8-9 hours until I can).

[Sharparam]

Thought for sure I had upgraded to 1.7.0, but evidently not. Updating that now and waiting for a rebased PR to see if it changes anything.

If the issue is the branch name encoding, then shouldn't the same issue happen on my regular release/6.0.0 builds?

[Sharparam]

Updating to 1.7.0 seems to not make it run at all: https://ci.appveyor.com/project/chroma-sdk/colore/builds/26394710

With an error: "The specified executable is not a valid application for this OS platform."

[AdmiringWorm]

Oh right, I have completely forgotten to update the Cake.Codecov addin with the new paths.

Ever since version 1.6.0 the normal package also includes Linux and OSX binaries, and the (yet to be updated) version of Cake.Codecov is incorrectly picking up the linux or osx binaries before the windows binary.

You can work around that until I can get the addin updated by passing in the path to the exe yourself.

Like this for instance:

var ccVersion = $"{version.FullSemVer}.build.{BuildSystem.AppVeyor.Environment.Build.Version}";

Information("Running Codecov tool with version {0} on OpenCover result", ccVersion);
var codecovPath = Context.Tools.Resolve("codecov.exe");

Codecov(new CodecovSettings
{
    Files = new[] { "./artifacts/opencover-results.xml" },
    Required = true,
    ////Branch = version.BranchName,
    EnvironmentVariables = new Dictionary<string, string>
    {
        ["APPVEYOR_BUILD_VERSION"] = ccVersion
    },
    Verbose = true,
    ToolPath = codecovPath
});

I am sorry about that, I completely forgot to update the addin.

EDIT: Issue tracking the status of Cake addin is here: cake-contrib/Cake.Codecov#40

[Sharparam]

Ah, I see. Setting the tool path made it work again for regular builds. Remains to see what the result of PR builds is.

[AdmiringWorm]

If the issue is the branch name encoding, then shouldn't the same issue happen on my regular release/6.0.0 builds?

I would think so, but it could be the combination of a PR and non-encoded branch name that is the problem.

[Sharparam]

Here's the build of the PR with 1.7.0 of Codecov

Still failing, though there seems to be a bit more output this time.

[AdmiringWorm]

I see.

There is still a little while until I get home, but I will take a closer look at what is going on (I may need to fork your repository to get to the bottom of this).

The additional output is mostly due to the implementation of the fallback uploader that was implemented in v1.6.0, and the actual response from codecov.io that was implemented in v1.7.0.

Unfortunately the response isn't helpful in this case, but hopefully I will get to the bottom of this anyhow.

[Sharparam]

Feel free to fork and make a PR to experiment in. Editing the build scripts in your fork should apply those changes in AppVeyor since all the configuration is stored in-repo.

[AdmiringWorm]

Unfortunately some things got in the way today, so I haven't been able to look into this further, and I am about to sign off for tonight.

I believe I should have some time before work tomorrow, so I will try my best to look into it then. Otherwise I won't be able to do that until Saturday or Sunday.

[AdmiringWorm]

Hmm, it unfortunately seems I was wrong. It have nothing to do with the unescaped branch name.

It did notice some difference between with one of my other projects and your project, however this seem to be the result of wrong documentation in the Cake.Codecov addin.
I have made some changes to the PR I am using to test this out to see if the change have anything to say, which it really should.

[AdmiringWorm]

This did indeed turn out to be a documentation issue on the Cake.Codecov addin. The PR I used to test out the changes have been changed to only include the necessary changes, and you can decide if you want to merge it in or incorporate it yourself (chroma-sdk/Colore#272).

Issue have also been opened regarding the incorrect documentation (the existing documentation only works if Pull Requests also increment the build number) here: cake-contrib/Cake.Codecov#54

I will be closing this issue as it turned out to not be in the application itself.