Add Azure Pipelines CI #2299
Add Azure Pipelines CI #2299
Conversation
|
This is actually something I've been considering. I won't have any time today to dig in but I'll try to find time soon. Can you remove the package-lock file that was generated? I haven't yet added the npmrc file that stops them from being generated. |
|
Whoops, didn't realized I'd added that, taken care of 😄 |
.azure-pipelines-steps.yml
Outdated
|
|
||
| - task: PublishTestResults@2 | ||
| inputs: | ||
| testResultsFiles: '**/xunit.xml' |
There was a problem hiding this comment.
So azure pipelines only speaks xunit?
There was a problem hiding this comment.
With this input it does, but it can also use JUnit, NUnit, VSTest, xUnit, or cTest, we'd just need to change testResultsFiles to something like **/junit.xml. I believe if we want we could also do something like **/TEST-**.xml, this input really just works as a pattern matcher I think.
I used xunit since that matches the custom reporter I added (more on that below).
There was a problem hiding this comment.
@damccorm It looks like mocha (even the version we use) ships with an XUnit reporter. Can we just use that? We can have an azure script that does mocha --async-only --reporter xunit
That way we can drop the extra reporter.
There was a problem hiding this comment.
That works, the only downside is we don't get great output in the console.
What we get with multi-reporter:
What we get with xunit reporter:
I have this change in PR, if you're comfortable with that tradeoff I'll merge just let me know.
There was a problem hiding this comment.
Oh actually, on closer inspection, this doesn't get written to a file which is necessary for us to report results. We can pipe it out to a file, but that would mean we would get no output at all on the console.
There was a problem hiding this comment.
We can use the -O output=test.xunit flag to specify the output file. I was thinking we could just run the test suites as separate phases, one with the spec reporter and one with the xunit reporter.
There was a problem hiding this comment.
Makes sense to me! Updated accordingly
|
@damccorm Just a few more things I followed up on. Also, do you have any guidance on how I signup and get this configured to work under a gulpjs organization for Azure Pipelines? |
|
Oh, also - does Azure Pipelines support a |
Sure, I moved it in there in my PR.
The easiest way to get this done is probably to install the GitHub app after you've merged in this PR. That will walk you through the process of creating an organization and project. From there you can follow the getting started docs here. One thing to call out is that since we moved the yaml to the Let me know if anything is unclear as well, happy to help however I can! |
|
One more thing - I noticed that both the organization names Regardless, installing the GitHub app is still the best way to get going. |
|
@damccorm yeah, I did set one up a long time ago but I gave up when I ran into tons of XML. Is the YAML stuff new?
|
Yeah, it was added a little more than half a year ago as the default experience. It was part of a larger push (which is still ongoing) to offer a better OSS experience in particular.
Yep! It should give you an opportunity to sign into an existing organization. |
|
Sorry for the delay. This is looking great! |
|
@damccorm I'm looking at the docs for secure variables and it seems to indicate there should be a 🔒 icon in the variables panel but I don't have that. How do I secure this coveralls token? Also, by using the coveralls token as a secure token, it says it won't be available to PRs - won't that break our coveralls Status API integration for PRs? |
Are you looking at this page or somewhere else? This is what I see (with the secure checkbox column highlighted):
If not could you provide a little more info on what you're looking at? To get to that page, you can follow these steps: Edit pipeline Choose
I missed that you were doing this. The answer is yes by default, but you can configure this behavior from the same menu (under the triggers tab)
If you do this, I would be careful though. The reason that secrets aren't enabled on forked builds is that someone could change a script that a secret is mapped into and do something malicious (e.g. send themselves the token). They can't change the yaml that gets executed though (or at least it won't get executed until merged), so a change you could make to ensure this doesn't happen is instead of calling |
|
@damccorm thanks for following up. I have totally different variables view than you. Is it possible that I'm on a newer/older version of the UI because I signed up so long ago? I also noticed that other stuff in the documentation didn't match my UI. |
|
@damccorm this is what I'm seeing for my "variables" screen |
|
Huh, that's weird, I haven't seen that view before (I joined Pipelines relatively recently though, so there's a chance it predates me). One thing that you could do is check to make sure the new UI didn't somehow get disabled in the preview features. For me you can access this here, hopefully this works for you as well:
|
The Change
This PR adds Azure Pipelines to perform CI builds and PR builds. Pipelines offers several key advantages:
Disclaimer
Full disclosure, I'm an engineer on the Pipelines team, but I also think Pipelines provides some pretty significant advantages here.
Additional Info
You can see that this works in my pipeline.
With test reporting!