Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Would you like an C.I. to test the builds? #22

Closed
eduncan911 opened this issue Oct 8, 2019 · 18 comments
Closed

Would you like an C.I. to test the builds? #22

eduncan911 opened this issue Oct 8, 2019 · 18 comments

Comments

@eduncan911
Copy link
Contributor

eduncan911 commented Oct 8, 2019 •
edited
Loading

With the addition of the Dockerfile in #21 , I can put together a Travis or maybe even a Github Tasks build environment to test for and report the state of master, and all PRs.

I have GitHub Actions enabled in my account; however, I think everyone has to request enrollment with Github.

Let me know and I will try to whip it up. Most likely will test all 3 build targets of the Makefile, and test that the files exists with no build errors.

Note: an Administrator will have to setup the tasks and join it to Travis or alike. I'll try to keep the instructions short and simple for the Assignee here to minimize the work.
@abarisani
Copy link
Contributor

I tried Travis already (see dc3b758), the problem is that the build takes too long and so it is not compatible with Travis time limits.

@eduncan911
Copy link
Contributor Author

eduncan911 commented Oct 8, 2019 •
edited
Loading

Ah yes, it does take a long time.

Let me see what the restrictions are around GitHub Actions as I've forked it over there on my accounts.

@eduncan911
Copy link
Contributor Author

I'm making some good headway with Github Actions (Github's native, free, unlimited CICD tool).

https://github.com/features/actions

I'll start on the usbarmory builds this weekend or next week.

@eduncan911
Copy link
Contributor Author

eduncan911 commented Oct 12, 2019 •
edited
Loading

Looks like Github Actions gives you up to 6 hours to run! https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes

Almost done. I'm getting an error running on Ubuntu 18.04 and compiling for the Mark II. Notice Mark I passes though! Woohoo!

Could you take a look? The errors are in the finalizer make target, something around u-boot make not finding a target on the uSD build:

https://github.com/eduncan911/usbarmory-debian-base_image/runs/257866845

However, on the eMMC build, i get a missing pgp key on the verify u-boot.bin:

https://github.com/eduncan911/usbarmory-debian-base_image/runs/257866843

Maybe no one has built this on Ubuntu 18.04? I only have options of Ubuntu 16.04 (too old gcc I ran into with u-boot) and Ubuntu 18.04 (giving the errors above).

Also, could you give me a set of checks and/or processes to perform AFTER make all completes? Something I can do to verify the image is valid or ready? (maybe the finalizer does that already?)

In addition, GitHub Actions can do anything - I can sign the build and make a full Release automatically (if pushing to master obviously). That would involve a private signing key though. Maybe you can upload a short-expiring sub-key (GitHub now has "Secrets" for uploading things like this).

I could also create a bash script you can run on your local machine to push a 24 hour signing sub-key if you want directly to GitHub (which is what I would do if it was me, before I "merge" any PR to master - run bash script, merge).

@abarisani
Copy link
Contributor

The README highlights some keys that need to be imported in the pre-requisites section, have you imported that in your action?

@eduncan911
Copy link
Contributor Author

eduncan911 commented Oct 13, 2019 •
edited
Loading

Thanks @abarisani. Yep, I was importing but had a bug in my retry logic (as PGP keyservers are unreliable). Fixed and now they always get both keys.

Switching to make all seems to have resolved it all as well! Across all 5 builds (i added imx6ul and imx6ull as separate builds). So we're ready to go!

I'll clean up the branch and will submit a PR. However, next steps is that you need to file a request to get access to Github Actions to enable it for your Github Org or it will never work:

https://github.com/features/actions

More info...

I was previously building each make target, so it's easier to debug. However, using the individual targets kept giving the errors during the finalizer target mentioned earlier. u-boot config and target issues. I suspect the Makefile needs some tweaking to allow for mk-ii individual target builds.

    - run: make ... check_version
    - run: make ... linux-deb
    - run: make ... debian 
    - run: make ... u-boot 
    - run: make ... finalize

Besides, having so many steps would actually make things more fragile if we were ever to change the Makefile, which I'm assuming is still in flux? :)

@eduncan911 eduncan911 mentioned this issue Oct 13, 2019
Merged
@abarisani
Copy link
Contributor

Thanks for all the effort! I just signed up for Actions beta for our org, we will review your commits while we wait on that! Cheers.

@abarisani
Copy link
Contributor

Ok, I now have Actions enabled.

@eduncan911
Copy link
Contributor Author

Nice!

https://github.com/inversepath/usbarmory-debian-base_image/actions

@abarisani
Copy link
Contributor

I wonder if something like this can be used to directly upload to a new release?

https://github.com/JasonEtco/upload-to-release/blob/master/upload-to-release

It would be nice if the release would remain a draft, so that we can download and check the image and manually set it to non draft (the linked script does the opposite).

@eduncan911
Copy link
Contributor Author

eduncan911 commented Oct 21, 2019 via email

@abarisani
Copy link
Contributor

I do want to verify each binary first, which is why I mentioned that I would be happy with upload to draft releases only. Because this allows me to download the published file on the draft release (which won't be public), validate it and then mark manually the release as non-draft. Does that make sense?

I'd like to peak only two releases, imx53 on uSD and imx6ULL on uSD, do you think this is feasible?

Thanks!

@eduncan911
Copy link
Contributor Author

eduncan911 commented Oct 21, 2019 via email

@abarisani
Copy link
Contributor

Ideally it would be one release with two attachments, but if that's not possible then two separate releases is fine. I'd keep both .zip and .xz files consistently with previous releases.

@eduncan911
Copy link
Contributor Author

eduncan911 commented Oct 21, 2019 via email

@abarisani
Copy link
Contributor

I was thinking of one release with just 2 .xz files (uSD for mark-one and mark-two) and SHA256s, but if there are no constraints on Github why not making all valid combinations (uSD for mark-one and mark-two, eMMC for mark-two, SHA256 file and .xz/.zip for all) ?

@eduncan911
Copy link
Contributor Author

eduncan911 commented Oct 21, 2019 •
edited
Loading

Heh, that's what I was originally proposing: attach all to one release, including sha hash files. :)

I'll try to whip up something this week.

Note one restriction with GitHub Actions: their triggers for builds are kind of limited.

So I was going to trigger a "Release" purely from a Tag. You make a Tag, and push the tag up, and it will trigger a DRAFT release build and push all files up.

Note x2: since I have to build all of this in one pipeline, I have to do it synchronously. That is, one at a time. So the build will take about 1.5 to 2 hours to produce the final Release.

Btw, hit me on keybase.io/eduncan911 if like. Or another messenger (PM twitter as well)

@abarisani
Copy link
Contributor

Ok, and sorry for the ping-pong. I was thinking out loud ;).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eduncan911 @abarisani