Build/continuous deployment

[seesharper]

This PR adds support for continuous deployment so that we can release with less effort and without the risk of human errors during packaging and deployment.

Deployment is triggered by creating a new tag on the master branch and pushing the tags to the origin.

The following example shows how to create a new release.

git checkout master 
git pull
git tag <new tag>
git push --tags

The two first command are just to ensure that we have the latest from origin before creating the tag.

The build script detects that we are on the master branch by checking the BuildEnvironment.IsSecure property. More on this later.

The build script then checks to see if this is a tag commit and if that is the case, the deployment process begins.

For dotnet-script this means the following steps.

• Produce a zip file of the publish folder (release). ex dotnet-script.0.19.0.zip

• Create a release note to be published as part of the GitHub release.

• Produce a Chocolatey package. ex dotnet.script.0.18.0.nupkg

• Produce NuGet packages for Dotnet.Script, Dotnet.Script.Core, Dotnet.Script.DependencyModel and Dotnet.Script.DependencyModel.NuGet

  ​

When the artifacts has been created the build script will deploy these artifacts to their respective destinations. The zip file along with the release note goes to GitHub releases, the NuGet packages to NuGet and finally the Chocolatey package goes to Chocolatey.

Artifacts are also uploaded to the AppVeyor cloud service in case we need to point somebody to a specific build of some package not yet released.

As mentioned the Release note is automatically generated and will be posted to GitHub Releases as the default release note. This is merely just a suggestion as to the content of the release note, but it should if not usable as it is, provide enough information about the changes that are included in the release.

For instance this is how it would have looked like for the 0.19.0 release.

Change Log

0.19.0 (06.03.2018)

Full Changelog

Merged Pull Requests

• version bump to 0.19, build using 0.18 (22.01.2018) #222 (filipw)
• Remove Global.json (14.02.2018) #223 (Dispersia)
• Remove dependency on System.Runtime.Loader. (16.02.2018) #224 (SlowLogicBoy)
• Feature/release configuration (02.03.2018) #229 (seesharper)
• Enforce TLS 1.2 in powershell (06.03.2018) #232 (filipw)
• Clear MSBuild related env variables before dotnet restore (06.03.2018) #233 (seesharper)

Closed Issues

• support custom filename in init (10.01.2018) #218 (filipw)
• Very large delays between command entry and execution (18.01.2018) #220 (nesteruk)
• Is possible to run script in RELEASE mode? (02.03.2018) #221 (wk-j)
• Intellisense not working for inline dependencies (06.03.2018) #230 (vincentlabatut)
• PowerShell installation script not working (06.03.2018) #231 (fmajestic)

--> End of release note sample

The release note is generated using github-changelog which is a script package that I have been working on for some time now.

Another script package, dotnet-build simply contains a collection of build related script such as NuGet.csx, Command.csx, FileUtils.csx and so on.

In order for all this to work properly, we need to provide the necessary encrypted environment variables to AppVeyor.

This is the appveyor.yml files as is stands in this PR.

environment:
    IS_SECURE_BUILDENVIRONMENT:
        secure: The encoded value of 'true' (not including the apostrophes)
    GITHUB_REPO_TOKEN:
        secure: The encoded value of your GitHub Access Token.
    NUGET_APIKEY:
        secure: The encoded value of your NuGet ApiKey.
    CHOCO_APIKEY:
        secure: The encoded value of the Choco API Key (dotnet-script account).

These values are ONLY decoded when a build is started from the master branch and hence the IS_SECURE_BUILDENVIRONMENT will only evaluate to true for the master branch.

You might wonder why we don't simply check if we are on the master branch? The reason is that AppVeyor never builds anything directly of the master branch. It build from a particular commit provided in the web hook. The reason for not checking the AppVeyor specific environment variables is that I wanted the build script to be completely unaware of the underlying build server.

The other three variables are just access keys to the deployment targets.

For local builds running on your machine, these values as just present as environment variables in plain text.

So, I need you (@filipw ) to encrypt these values using the Encrypt Data feature of AppVeyor.

Even though I do have access to AppVeyor, I am not entirely sure how it would be if I encrypted the variables. Figured best if we did it from one account (yours:))

I'll mail you the password for the dotnet-script account on Chocolatey so you can get the API key needed for pushing to Chocolatey.-

With this in place we'd never need to do a "manual" again apart from maybe adding some TLC to the generated release notes. That's it.

[filipw]

🏆

[filipw]

Looks great! Looks like we will avoid publishing a wrong dll like I did last time 😂

As mentioned the Release note is automatically generated and will be posted to GitHub Releases as the default release note.

doesn't this also create a tag?

Also, given that Nuget doesn't allow packages to be deleted, I think I'd like to run it once without auto-publish and just create artifacts, and only from then onwards enable auto-publish. Would that make sense?

[seesharper]

doesn't this also create a tag?

Not exactly. A tag and a release are two different things. A GitHub release is its own thing that is associated with a given tag. In this case it is the latest tag.

That being said, for 0.19.0 there is already an existing draft that probably should put aside while we run this for the first time. We'll coordinate that when are ready to release. No worries there 😄

Also, given that Nuget doesn't allow packages to be deleted, I think I'd like to run it once without auto-publish and just create artifacts, and only from then onwards enable auto-publish. Would that make sense?

In the dotnet-script solution we produce multiple NuGet packages, but we don't necessarily bump the version for all the packages. That is why you see the NuGet.TryPush instead of NuGet.Push which would fail if it tries to publish an existing package.

We should be good with the existing stuff in the build script. The only missing piece is the various API keys that needs to be encrypted and put into the appveyor.yml file.

Drop me a mail at bernhard.richter@gmail.com and I'll provide the account information for Chocolatey 👍

[filipw]

Not exactly. A tag and a release are two different things. A GitHub release is its own thing that is associated with a given tag. In this case it is the latest tag.

sorry for not being clear. I meant that if I go to releases on GH, I can create a new release and give it a tag. That tag is then publicly created when I publish the release.
So what happens, if I do something like this now? It seems that the script would run on appveyor and create a new release draft?

Or in other words, I could trigger the release process via GH release UI, without dropping to command line at all, which is even more enticing scenario I would say.

[seesharper]

Ah, okay. I understand. That should be quite easy to accomplish. Just need to check if the AppVeyor build is triggered when we create a tag from the GitHub UI. If that works we should be okay. I also need to check the logic for actually creating the release when there is already a release associated with the tag. I'll get back at you 👍

[seesharper]

Confirmed. The AppVeyor build is triggered when creating the tag from the GitHub UI. Same as git push --tags Just need to check if we can update the release notes and add the assets 👍

[filipw]

great. let's do this release 0.19.0 manually since I already drafted it last week and then from next one we automate everything 👍

[seesharper]

Sounds good to me. Creating the release from the GitHub UI totally possible. Making the necessary changes now 👍

[seesharper]

With that we can release either from the command line or from the GitHub Release UI 👍 Fully automated

[filipw]

I have released 0.19.0 manually, can you push the choco release? then we can merge this and move to a more structured world of managing releases!

[seesharper]

The choco package has been pushed and is awaiting moderation. Should be available soon. I'll need to make some minor changes to this PR to make the GitHub UI scenario work. Let you know once it's ready 👍

[seesharper]

I've made the necessary changes in Dotnet.Build to handle creating a new tag/release from the GitHub UI.
The workflow will be.

1. Create a new release with a new tag in the GitHub Releases UI.
2. Leave the name and body empty as it will be overwritten by the build script that pushes the generated changelog.
3. After the AppVeyor build completes and the release notes have been updated, we are free to edit the release notes if needed.

Note: git push --tags will also work if we want to do it from the command line

Important
@filipw I need you to fill inn the missing keys in the AppVeyor.yml file before we merge to master.
You have all the information needed except for the Choco api key. Ping me on twitter and we'll exchange that info somehow. 👍

Since there are parts of the build script that only executes when building on master, we might encounter issues as the code has never been executed before. If that happens, I'll fix it immediately. That would require an "in-flight push to master". Nothing to worry about. I have implemented this on several of my other projects here on GitHub and its working perfectly 👍

[filipw]

ok let's go