try moving to Github Actions from Travis for CI #1783

jywarren · 2020-12-30T00:02:01Z

based on similar work in publiclab/plots2#8795 and publiclab/plots2#8793

In-depth discussion here: https://www.jeffgeerling.com/blog/2020/travis-cis-new-pricing-plan-threw-wrench-my-open-source-works

gitpod-io · 2020-12-30T00:02:04Z

jywarren · 2020-12-30T00:02:44Z

Working from:

image-sequencer/.travis.yml

Lines 1 to 49 in fc799bf

    
           sudo: required 
        
           language: node_js 
        
           node_js: 
        
             - '10' 
        
             - '12' 
        
           env: 
        
             - CXX=g++-4.8 
        
           before_script: 
        
             - npm install grunt-cli -g # for "grunt build" 
        
             - curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter 
        
             - chmod +x ./cc-test-reporter 
        
             - ./cc-test-reporter before-build 
        
           jobs: 
        
             include: 
        
               - name: "Base istanbul/tape node tests" 
        
                 script: npm test 
        
               - name: "Benchmark tests" 
        
                 script: npm run benchmark 
        
               - name: "Gif tests" 
        
                 script: npm run gif-test 
        
               - name: "Browserify core tests and run" 
        
                 script: grunt tests && npm run core-tests 
        
               - name: "Jasmine UI tests (mocked browser env)" 
        
                 script: npm run test-ui 
        
               - name: "jest-puppeteer UI tests (full browser env)" 
        
                 script: npm run test-ui-2     
        
               - name: "CLI tests" 
        
                 script: npm run test-cli 
        
               - name: "Grunt build test of dev environment" 
        
                 script: grunt build 
        
           after_success: 
        
             - bash <(curl -s https://codecov.io/bash) 
        
           after_script: 
        
             - ./cc-test-reporter after-build --exit-code $TRAVIS_TEST_RESULT 
        
           addons: 
        
             apt: 
        
               sources: 
        
                 - ubuntu-toolchain-r-test 
        
               packages: 
        
                 - g++-4.8 
        
                 - xvfb # for tape-run 
        
           before_install: 
        
             - sudo apt-get update 
        
             - sudo apt-get install xserver-xorg-dev libxext-dev libxi-dev 
        
             - sudo apt-get install -y build-essential libxi-dev libglu1-mesa-dev libglew-dev pkg-config libglu1-mesa-dev freeglut3-dev mesa-common-dev 
        
           install: 
        
             - export DISPLAY=':99.0' # for tape-run 
        
             - Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 & # for tape-run 
        
             - npm install # for tape-run

jywarren · 2020-12-30T00:11:14Z

Looking good for a first attempt! We got all the way to gif tests and failed 2:

  Failed Tests: There were 2 failures

    blur module works correctly

      ✖ blur module works correctly with Gif


    convolution module works correctly

      ✖ convolution module works correctly with Gif

  total:     31
  passing:   29
  failing:   2
  duration:  30.9s


npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! image-sequencer@3.6.0 gif-test: `node test/core/gifs/gif-test.js | tap-spec;`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the image-sequencer@3.6.0 gif-test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2020-12-30T00_07_06_164Z-debug.log
Error: Process completed with exit code 1.

I wonder if these relate to setup - do we need... chromedriver... headless-gl... other stuff from .travis.yml?

cc @harshkhandeparkar @publiclab/is-maintainers just so you can see my progress here! Any help welcome!

jywarren · 2020-12-30T00:12:10Z

Thinking some of this stuff we may have to port over:

image-sequencer/.travis.yml

Lines 35 to 49 in fc799bf

    
           addons: 
        
             apt: 
        
               sources: 
        
                 - ubuntu-toolchain-r-test 
        
               packages: 
        
                 - g++-4.8 
        
                 - xvfb # for tape-run 
        
           before_install: 
        
             - sudo apt-get update 
        
             - sudo apt-get install xserver-xorg-dev libxext-dev libxi-dev 
        
             - sudo apt-get install -y build-essential libxi-dev libglu1-mesa-dev libglew-dev pkg-config libglu1-mesa-dev freeglut3-dev mesa-common-dev 
        
           install: 
        
             - export DISPLAY=':99.0' # for tape-run 
        
             - Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 & # for tape-run 
        
             - npm install # for tape-run

harshkhandeparkar · 2020-12-31T04:51:09Z

Thinking some of this stuff we may have to port over:

We need those dependencies for some of the modules.

I wonder if these relate to setup - do we need... chromedriver... headless-gl... other stuff from .travis.yml?

If it works for non-GIFs, it should work for GIFs...

jywarren · 2021-01-05T20:59:15Z

OK we should be able to use this format to do this:

- name: Install prerequisites
  run: | 
   - sudo apt-get update 
   - sudo apt-get install g++-4.8 xvfb
   - sudo apt-get install xserver-xorg-dev libxext-dev libxi-dev 
   - sudo apt-get install -y build-essential libxi-dev libglu1-mesa-dev libglew-dev pkg-config libglu1-mesa-dev freeglut3-dev mesa-common-dev 
   - export DISPLAY=':99.0' # for tape-run 
   - Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 & # for tape-run 
   - npm install # for tape-run

.github/workflows/continuous-integration.yml

Co-authored-by: Barun Acharya <47106543+daemon1024@users.noreply.github.com>

jywarren · 2021-01-05T22:46:01Z

OK, we got as far as Gif tests!

daemon1024 · 2021-01-05T22:48:00Z

It's the same as before, the same two tests were failing without the prerequisites too.

daemon1024 · 2021-01-10T16:01:35Z

I tried debugging it. It seems like the blur module is working fine but the generated base64 string is not matching the provided benchmark. Both seem to be blurred out when I converted them to image. What could be the reason behind it 🤔

harshkhandeparkar · 2021-01-10T16:36:50Z

It is possible that the two blurred images are very slightly off. This could happen even due to precision issues! We need to use a better way of comparing images. We used to use the looks-same library to do it before. I think we ditched it at some point because this used to work just fine.... Until now.

daemon1024 · 2021-01-10T19:39:47Z

That might be the case. looks-same seems to be limited to PNGs tho.

Both blur and convolution seems to be dependent on

image-sequencer/src/modules/Blur/Blur.js

Line 51 in fc799bf

    
           const convolve = require('../_nomodule/gpuUtils').convolve; // GPU convolution function.

which convolutes the rgb channels.

It is a gpu accelerated task, is it some limitation of the ci environment in github resulting in that precision issue?

harshkhandeparkar · 2021-01-11T02:55:40Z

GPU precision can vary very slightly from device to device and in the case of modules like a blur, it won't even be noticed. We really need a looks-same alternative.

jywarren · 2021-01-11T04:00:45Z

Oh thanks for digging into this! Could we try using this? https://github.com/mapbox/pixelmatch Dunno if it would help...?

…

On Sun, Jan 10, 2021, 9:55 PM Harsh Khandeparkar ***@***.***> wrote: GPU precision can vary very slightly from device to device and in the case of modules like a blur, it won't even be noticed. We really need a looks-same alternative. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF6JYVK67D7RL2WAR2Y7TSZJSDRANCNFSM4VNXUEIQ> .

daemon1024 · 2021-01-11T10:01:42Z

So I tried using https://github.com/rsmbl/Resemble.js since it looked more convenient to use than pixelmatch. I set the mismatch percentage to be less than 5% to pass the tests.

Here's the CI run on my PR with resemble.js
https://github.com/daemon1024/image-sequencer/pull/1/checks?check_run_id=1680335832
and my PR daemon1024#1

The gif tests seems to passing now tho it fails at the next step but it shouldn't be related to the scope of what we are trying to figure out right now.

Update: Pixelmatch seems to need height and width of image as input, we would need to process the image for getting that information. Let me know if you know of a way to do that.

jywarren · 2021-01-12T16:19:50Z

This is awesome, @daemon1024 thank you! Pulling in your commits now. Let's see what's next.

jywarren · 2021-01-12T16:28:56Z

OK great - past Gif tests! Now:


npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! image-sequencer@3.6.0 core-tests: `cat ./output/core-tests.js | tape-run --render="tap-spec"`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the image-sequencer@3.6.0 core-tests script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2021-01-12T16_23_44_822Z-debug.log
Error: Process completed with exit code 1.

note that ./output/core-tests.js is the browserified output of the tests themselves, from the grunt tests command. Some googling shows ELIFECYCLE sometimes relates to outdated dependencies and sometimes to permissions issues (https://stackoverflow.com/questions/42308879/how-to-solve-npm-error-npm-err-code-elifecycle and facebook/create-react-app#5980)... hmm. I tried restarting the test run to see if this was intermittent, but will keep trying...

jywarren · 2021-01-12T18:37:28Z

OK, reproduced this in GitPod, full log copied here: https://gist.github.com/jywarren/eef2a7419c8a465842149554598e4045

harshkhandeparkar · 2021-01-12T19:46:50Z

Are Github actions faster or have higher caps?

harshkhandeparkar · 2021-01-12T19:48:15Z

Github, NPM, Azure, those PRs to the Linux kernel, WSL, Github One.... It's all falling into place now 😂

jywarren · 2021-01-12T19:50:08Z

I dunno about faster but travis was cut drastically so it has almost no runs for open source projects so we needed an alternative. 😭

…

On Tue, Jan 12, 2021, 2:48 PM Harsh Khandeparkar ***@***.***> wrote: Github, NPM, Azure, those PRs to the Linux kernel, WSL, Github One.... It's all falling into place now 😂 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF6J2B6PPXMZUABIGDKYDSZSRQ5ANCNFSM4VNXUEIQ> .

harshkhandeparkar · 2021-01-12T19:55:47Z

Judging by what they are doing, they probably want to erase GitLab from the face of the planet. CI/CD, Web IDE, Collaboration... All of these were Gitlab's main features...

jywarren · 2021-01-18T21:54:18Z

For what it's worth, here is a complete log of running npm run setup:

https://gist.github.com/jywarren/c1573f8ceac5ecb45db49c945a6d7115

It does seem related to canvas but... it's possible the error from npm run setup is unrelated to the error from npm run core-tests

jywarren · 2021-01-18T21:55:08Z

Here's a note on using tape-run in github actions! https://github.com/juliangruber/tape-run#github-actions

jywarren · 2021-01-18T22:12:03Z

What do you both think? Can you offer a review?

daemon1024

🎉

daemon1024 · 2021-01-18T22:22:58Z

Xvfb (short for X virtual framebuffer) is an in-memory display server for UNIX-like operating system (e.g., Linux). It enables you to run graphical applications without a display (e.g., browser tests on a CI server)

Interesting.

jywarren · 2021-01-19T00:11:24Z

Yeah, seems like that was it -- i hope that by splitting out, we can see more clearly what is required for which job!

jywarren · 2021-01-19T02:29:36Z

Also noting this runs in 1/2 the time of Travis because it's parallel -- 2m 50s instead of 6m 26s in https://github.com/publiclab/image-sequencer/runs/1343663692 (for example)

harshkhandeparkar · 2021-01-19T02:58:52Z

It works finally? 🥳

jywarren · 2021-01-19T03:03:56Z

Yes, wanna look through? I think we can merge!

…

On Mon, Jan 18, 2021, 9:59 PM Harsh Khandeparkar ***@***.***> wrote: It works finally? 🥳 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1783 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF6JZFLLPRR3G2HW6ADJ3S2TYPVANCNFSM4VNXUEIQ> .

.github/workflows/tests.yml

jywarren · 2021-01-19T03:59:41Z

Let's do it in a follow-up, what do you think?

…

On Mon, Jan 18, 2021, 10:48 PM Harsh Khandeparkar ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In .github/workflows/tests.yml <#1783 (comment)> : > @@ -0,0 +1,209 @@ +name: tests +on: [pull_request] +jobs: + base-tests: + runs-on: ubuntu-18.04 + steps: + - uses: ***@***.*** + - name: Setup node + uses: ***@***.*** + with: + node-version: '12' Should we add v10 and v14 also? (At least v14 since it is the latest) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1783 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF6J4TF5WSVHLRBKROAQDS2T6JZANCNFSM4VNXUEIQ> .

harshkhandeparkar · 2021-01-19T04:52:35Z

Sure, np!

jywarren · 2021-01-19T12:48:36Z

Hooray! Thank you both a TON for helping get this merged! I couldn't have done it without you!

try moving to Github Actions from Travis for CI

1fcc522

jywarren requested a review from a team as a code owner December 30, 2020 00:02

npm install

78620c7

jywarren added 2 commits January 5, 2021 16:01

add apt-get prereqs

b42249f

Update continuous-integration.yml

474a3f8

daemon1024 reviewed Jan 5, 2021

View reviewed changes

.github/workflows/continuous-integration.yml Outdated Show resolved Hide resolved

Update .github/workflows/continuous-integration.yml

6c63cc7

Co-authored-by: Barun Acharya <47106543+daemon1024@users.noreply.github.com>

use resemblejs in gif-tests

cd1aeda

jywarren requested a review from a team as a code owner January 12, 2021 16:19

name change for brevity

0d38f0c

jywarren added 2 commits January 18, 2021 21:56

add recommended deps for tape-run

db180e9

xvfb-run --auto-servernum npm run core-tests

7715fa2

jywarren requested review from daemon1024 and harshkhandeparkar January 18, 2021 22:12

daemon1024 approved these changes Jan 18, 2021

View reviewed changes

harshkhandeparkar reviewed Jan 19, 2021

View reviewed changes

.github/workflows/tests.yml Show resolved Hide resolved

harshkhandeparkar approved these changes Jan 19, 2021

View reviewed changes

jywarren merged commit 580a154 into main Jan 19, 2021

jywarren mentioned this pull request Jan 21, 2021

Travis taking forever to finish..move to git-actions? publiclab/PublicLab.Editor#676

Closed

cesswairimu mentioned this pull request Jan 31, 2021

Move Travis to Github actions publiclab/mapknitter#1489

Closed

jywarren mentioned this pull request Feb 2, 2021

switch to npm ci for GitHub Action CI instead of npm install #1800

Closed

jywarren mentioned this pull request Mar 25, 2021

Checklist and coordination for v3.7.0 minor release #1751

Closed

11 tasks

jywarren mentioned this pull request May 18, 2021

Convert all publiclab repositories' automated tests/checks from TravisCI to GitHub Actions publiclab/plots2#9641

Open

36 tasks

aliciapaz mentioned this pull request Jul 30, 2021

try migrating to GitHub Actions from Travis for CI publiclab/webjack#189

Closed

This was referenced Jan 11, 2022

Bump to v3.7.0 (trouble with pangocairo lib) #2034

Merged

bump to 3.7.1 w /dist/ files compiled, yarn.lock removed, updated release workflow issue template #2036

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

try moving to Github Actions from Travis for CI #1783

try moving to Github Actions from Travis for CI #1783

jywarren commented Dec 30, 2020 •

edited

Loading

gitpod-io bot commented Dec 30, 2020 •

edited

Loading

jywarren commented Dec 30, 2020

jywarren commented Dec 30, 2020

jywarren commented Dec 30, 2020

harshkhandeparkar commented Dec 31, 2020

jywarren commented Jan 5, 2021

jywarren commented Jan 5, 2021

daemon1024 commented Jan 5, 2021 •

edited

Loading

daemon1024 commented Jan 10, 2021

harshkhandeparkar commented Jan 10, 2021

daemon1024 commented Jan 10, 2021 •

edited

Loading

harshkhandeparkar commented Jan 11, 2021

jywarren commented Jan 11, 2021 via email

daemon1024 commented Jan 11, 2021 •

edited

Loading

jywarren commented Jan 12, 2021

jywarren commented Jan 12, 2021 •

edited

Loading

jywarren commented Jan 12, 2021

harshkhandeparkar commented Jan 12, 2021

harshkhandeparkar commented Jan 12, 2021

jywarren commented Jan 12, 2021 via email

harshkhandeparkar commented Jan 12, 2021

jywarren commented Jan 18, 2021

jywarren commented Jan 18, 2021

jywarren commented Jan 18, 2021

daemon1024 left a comment

daemon1024 commented Jan 18, 2021

jywarren commented Jan 19, 2021

jywarren commented Jan 19, 2021

harshkhandeparkar commented Jan 19, 2021

jywarren commented Jan 19, 2021 via email

jywarren commented Jan 19, 2021 via email

harshkhandeparkar commented Jan 19, 2021

jywarren commented Jan 19, 2021

try moving to Github Actions from Travis for CI #1783

try moving to Github Actions from Travis for CI #1783

Conversation

jywarren commented Dec 30, 2020 • edited Loading

gitpod-io bot commented Dec 30, 2020 • edited Loading

jywarren commented Dec 30, 2020

jywarren commented Dec 30, 2020

jywarren commented Dec 30, 2020

harshkhandeparkar commented Dec 31, 2020

jywarren commented Jan 5, 2021

jywarren commented Jan 5, 2021

daemon1024 commented Jan 5, 2021 • edited Loading

daemon1024 commented Jan 10, 2021

harshkhandeparkar commented Jan 10, 2021

daemon1024 commented Jan 10, 2021 • edited Loading

harshkhandeparkar commented Jan 11, 2021

jywarren commented Jan 11, 2021 via email

daemon1024 commented Jan 11, 2021 • edited Loading

jywarren commented Jan 12, 2021

jywarren commented Jan 12, 2021 • edited Loading

jywarren commented Jan 12, 2021

harshkhandeparkar commented Jan 12, 2021

harshkhandeparkar commented Jan 12, 2021

jywarren commented Jan 12, 2021 via email

harshkhandeparkar commented Jan 12, 2021

jywarren commented Jan 18, 2021

jywarren commented Jan 18, 2021

jywarren commented Jan 18, 2021

daemon1024 left a comment

Choose a reason for hiding this comment

daemon1024 commented Jan 18, 2021

jywarren commented Jan 19, 2021

jywarren commented Jan 19, 2021

harshkhandeparkar commented Jan 19, 2021

jywarren commented Jan 19, 2021 via email

jywarren commented Jan 19, 2021 via email

harshkhandeparkar commented Jan 19, 2021

jywarren commented Jan 19, 2021

jywarren commented Dec 30, 2020 •

edited

Loading

gitpod-io bot commented Dec 30, 2020 •

edited

Loading

daemon1024 commented Jan 5, 2021 •

edited

Loading

daemon1024 commented Jan 10, 2021 •

edited

Loading

daemon1024 commented Jan 11, 2021 •

edited

Loading

jywarren commented Jan 12, 2021 •

edited

Loading