Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove 'travis*' prefix #3176

Merged
merged 30 commits into from
Jul 9, 2021
Merged

Conversation

vkatsuba
Copy link
Contributor

@vkatsuba vkatsuba commented Jul 8, 2021

Proposed changes include:
Remove travis* prefix from all places.

Roadmap of renaming:

From To
tools/travis-upload-to-s3.sh tools/upload-to-s3.sh
tools/travis-upload-to-gdrive.sh tools/upload-to-gdrive.sh
tools/travis-test.sh tools/test.sh
tools/travis-setup-rmq.sh tools/setup-rmq.sh
tools/travis-setup-ldap.sh tools/setup-ldap.sh
tools/travis-setup-humio.sh tools/setup-humio.sh
tools/travis-setup-db.sh tools/setup-db.sh
tools/travis-secure-erase.sh tools/secure-erase.sh
tools/travis-pull-db.sh tools/pull-db.sh
tools/travis-publish-github-comment.sh tools/publish-github-comment.sh
tools/travis-prepare-log-dir.sh tools/prepare-log-dir.sh
tools/travis-helpers.sh tools/helpers.sh
tools/travis-db-versions.sh tools/db-versions.sh
tools/travis-common-vars.sh tools/common-vars.sh
tools/travis-check-tests-branch.sh tools/check-tests-branch.sh
tools/travis-build-tests.sh tools/build-tests.sh

TODO's:

tools/travis-setup-db.sh    -> tools/setup-db.sh
tools/travis-setup-ldap.sh  -> tools/setup-ldap.sh
tools/travis-setup-rmq.sh   -> tools/setup-rmq.sh
tools/travis-test.sh        -> tools/test.sh
tools/travis-build-tests.sh -> tools/build-tests.sh
* tools/travis-test.sh        -> tools/test.sh
* tools/travis-build-tests.sh -> tools/build-tests.sh
* tools/travis-setup-db.sh    -> tools/setup-db.sh
* tools/travis-test.sh -> tools/test.sh
* TRAVIS_JOB_ID        -> GITHUB_RUN_ID
* travis_fold/2        -> github_actions_fold/2
* Travis -> GitHub Actions
* tools/travis-setup-db.sh -> tools/setup-db.sh
* tools/travis-test.sh     -> tools/test.sh
* Travis                   -> GitHub Actions
* ./tools/travis-test.sh -> ./tools/test.sh
* TravisInfo  -> CIInfo
* travis_info -> ci_info
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-build-tests.sh -> tools/build-tests.sh
* tools/travis-setup-db.sh    -> tools/setup-db.sh
* tools/travis-test.sh        -> tools/test.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* TRAVIS_COMMIT_MESSAGE       -> CI_COMMIT_MESSAGE
* TRAVIS_PULL_REQUEST -> CI_PULL_REQUEST
* mongoose-travis-commenter -> mongoose-commenter
* travis -> CI
* TRAVIS_BUILD_NUMBER -> GITHUB_RUN_NUMBER
* TRAVIS_OTP_RELEASE  -> CI_OTP_RELEASE
* TRAVIS_BRANCH       -> CI_BRANCH
* TRAVIS_PULL_REQUEST -> CI_PULL_REQUEST
* tools/travis-helpers.sh -> tools/helpers.sh
* tools/travis-helpers.sh -> tools/helpers.sh
* TRAVIS_PULL_REQUEST     -> CI_PULL_REQUEST
* TRAVIS_COMMIT           -> GITHUB_SHA
* TRAVIS_JOB_ID           -> GITHUB_RUN_ID
* TRAVIS_JOB_NUMBER       -> GITHUB_JOB
* JOB_URL="https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$GITHUB_JOB" -> JOB_URL="https://github.com/esl/MongooseIM/actions/runs/$GITHUB_RUN_ID"
* echo "export TRAVIS_BUILD_NUMBER=$TRAVIS_BUILD_NUMBER" -> removed
* TRAVIS_REPO_SLUG   -> GITHUB_REPOSITORY
* TRAVIS_OTP_RELEASE -> CI_OTP_RELEASE
* Travis             -> GitHub Actions
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-db-versions.sh -> tools/db-versions.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-db-versions.sh -> tools/db-versions.sh
* Travis                      -> CI
* travis-common-vars.sh       ->  common-vars.sh
* tools/travis-setup-ldap.sh  -> tools/setup-ldap.sh
* tools/travis-setup-rmq.sh   -> tools/setup-rmq.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-common-vars.sh -> tools/common-vars.sh
* tools/travis-helpers.sh -> tools/helpers.sh
tools/travis-helpers.sh -> tools/helpers.sh
* tools/travis-helpers.sh -> tools/helpers.sh
* tools/travis-upload-to-s3.sh           -> tools/upload-to-s3.sh
* tools/travis-upload-to-gdrive.sh       -> tools/upload-to-gdrive.sh
* tools/travis-test.sh                   -> tools/test.sh
* tools/travis-setup-rmq.sh              -> tools/setup-rmq.sh
* tools/travis-setup-ldap.sh             -> tools/setup-ldap.sh
* tools/travis-setup-humio.sh            -> tools/setup-humio.sh
* tools/travis-setup-db.sh               -> tools/setup-db.sh
* tools/travis-secure-erase.sh           -> tools/secure-erase.sh
* tools/travis-pull-db.sh                -> tools/pull-db.sh
* tools/travis-publish-github-comment.sh -> tools/publish-github-comment.sh
* tools/travis-prepare-log-dir.sh        -> tools/prepare-log-dir.sh
* tools/travis-helpers.sh                -> tools/helpers.sh
* tools/travis-db-versions.sh            -> tools/db-versions.sh
* tools/travis-common-vars.sh            -> tools/common-vars.sh
* tools/travis-check-tests-branch.sh     -> tools/check-tests-branch.sh
* tools/travis-build-tests.sh            -> tools/build-tests.sh
@codecov
Copy link

codecov bot commented Jul 8, 2021

Codecov Report

Merging #3176 (9ee015a) into master (5624bf9) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3176      +/-   ##
==========================================
- Coverage   80.23%   80.23%   -0.01%     
==========================================
  Files         396      396              
  Lines       32317    32317              
==========================================
- Hits        25931    25928       -3     
- Misses       6386     6389       +3     
Impacted Files Coverage Δ
src/gen_mod.erl 74.00% <0.00%> (ø)
src/mod_roster_rdbms.erl 88.61% <0.00%> (-7.32%) ⬇️
src/mongoose_tcp_listener.erl 76.59% <0.00%> (-4.26%) ⬇️
src/ejabberd_s2s_out.erl 58.99% <0.00%> (-1.14%) ⬇️
src/wpool/mongoose_wpool.erl 84.00% <0.00%> (-1.00%) ⬇️
src/ejabberd_admin.erl 59.24% <0.00%> (-0.95%) ⬇️
src/muc_light/mod_muc_light.erl 85.13% <0.00%> (-0.75%) ⬇️
src/auth/ejabberd_auth_ldap.erl 67.88% <0.00%> (-0.73%) ⬇️
src/auth/ejabberd_auth.erl 82.38% <0.00%> (-0.52%) ⬇️
src/ejabberd_sm.erl 84.26% <0.00%> (-0.33%) ⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5624bf9...9ee015a. Read the comment docs.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Jul 8, 2021

small_tests_21_3 / small_tests / 9ee015a
Reports root / small


dynamic_domains / pgsql_mnesia / 9ee015a
Reports root/ big
OK: 1565 / Failed: 0 / User-skipped: 121 / Auto-skipped: 0


small_tests_22 / small_tests / 9ee015a
Reports root / small


small_tests_23 / small_tests / 9ee015a
Reports root / small


ldap_mnesia_21 / ldap_mnesia / 9ee015a
Reports root/ big
OK: 1514 / Failed: 0 / User-skipped: 359 / Auto-skipped: 0


ldap_mnesia_22 / ldap_mnesia / 9ee015a
Reports root/ big
OK: 1514 / Failed: 0 / User-skipped: 359 / Auto-skipped: 0


internal_mnesia / internal_mnesia / 9ee015a
Reports root/ big
OK: 1587 / Failed: 0 / User-skipped: 286 / Auto-skipped: 0


ldap_mnesia_23 / ldap_mnesia / 9ee015a
Reports root/ big
OK: 1514 / Failed: 0 / User-skipped: 359 / Auto-skipped: 0


pgsql_mnesia / pgsql_mnesia / 9ee015a
Reports root/ big
OK: 3084 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


mssql_mnesia / odbc_mssql_mnesia / 9ee015a
Reports root/ big
OK: 3084 / Failed: 0 / User-skipped: 184 / Auto-skipped: 0


elasticsearch_and_cassandra / elasticsearch_and_cassandra_mnesia / 9ee015a
Reports root/ big
OK: 1890 / Failed: 0 / User-skipped: 282 / Auto-skipped: 0


mysql_redis / mysql_redis / 9ee015a
Reports root/ big
OK: 3067 / Failed: 0 / User-skipped: 201 / Auto-skipped: 0


riak_mnesia / riak_mnesia / 9ee015a
Reports root/ big
OK: 1737 / Failed: 0 / User-skipped: 285 / Auto-skipped: 0

@vkatsuba
Copy link
Contributor Author

vkatsuba commented Jul 8, 2021

GitHub Actions Result

@NelsonVides
Copy link
Collaborator

This is a good idea, but it's not really the direction I was thinking of moving forwards. We've been using a paid account of CircleCI for almost a couple of years, and some long time ago, I was working on #3001, which was trying to move the CircleCI scripts from VM machines to docker containers.

It had some advantages, like, in the VM erlang needs to be installed and that takes around 10min, while the container can simply be spawned as-is. Also, general execution time was faster, and it allows for some tricks with parallelism (we have unlimited parallel containers), etc etc.

I had a challenge, and it's that our backends (pgsql, redis, ldap, etc), are started as separate containers that CircleCI doesn't allow you to provision them for security reasons, so the spawned containers had to be used as-they-are, and we were testing them with ssl. So I thought, that the ssl connection could be tested separately, we could have a VM job that would rotate the backends and run simple tests to check that they work, and a big bunch of docker jobs testing everything else. But I didn't have the capacity to reconfigure all of our scripts.

This PR has some good ideas, but I don't know if Github actions have some limitations per organisation or so, or that of any other kind. Maybe it's a good discussion on how to move forwards, what use are we giving to each CI, would love to have a chat on this 😄

@vkatsuba
Copy link
Contributor Author

vkatsuba commented Jul 9, 2021

This is a good idea, but it's not really the direction I was thinking of moving forwards.

You see, the main idea of do those changes is remove prefix from all scripts what is used al in CI as in locally for build/run/prepare some stuff. In fact, dead scripts are also present here and it would be great if MIM team will provide info about scripts which not used at all as in CI as in locally, then we will remove it in scope of current PR. Regarding mentioning of GitHub Actions - we can replace to CI and keep them for using in any CI.

Copy link
Collaborator

@DenysGonchar DenysGonchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DenysGonchar
Copy link
Collaborator

I think the main confusion comes from GH Action report for the last commit, but it was manually triggered. This PR doesn't enable automatic GH Actions execution for PRs, it still starts automatically for master branch changes only.

indeed, after introduction of that CircleCI changes we won't use that container provisioning scripts for a docker based CircleCI env, but we still need them for testing SSL connection in CircleCI machine, GH Actions builds and local execution.

Copy link
Collaborator

@NelsonVides NelsonVides left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I got it wrong, I thought that this PR was also enabling GH Actions on PRs, it was a manual run 🙂

I have an example of a file that could use some unification: tools/publish-github-comment.sh and tools/circle-publish-github-comment.sh. Just to mention an example, I don't really know all the so many files we have in the tools/ dir, but it's actually a good idea to have a cleanup soon.

But making our CI great is even more important 😁

@NelsonVides NelsonVides merged commit 9741296 into master Jul 9, 2021
@NelsonVides NelsonVides deleted the refactoring/remove-travis-prefix branch July 9, 2021 16:21
@Premwoik Premwoik added this to the 5.0.0 milestone Oct 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants