Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GH Actions CI support #991

Merged
merged 12 commits into from
Apr 25, 2020

Conversation

michdolan
Copy link
Collaborator

This adds GH Actions CI as a replacement for AZP, per the ASWF decision to switch, and in preparation for adding GPU CI support.

Some notes to help in reviewing the changes:

  • GH Actions currently has no support for templates like AZP had, so everything exists in the one yml workflow file. I could have moved some of the commands into .sh scripts, but opted to leverage the dynamic nature of the job matrices, so the workflow is mostly self-contained. The benefit is that it is easier to see what's going on at a glance.
  • GH Actions matrices are quite different in design from AZP. In theory their approach is quite nice, as you get the cross product of one or more sequences of options, but I found in practice that it makes it harder to keep the job budget down while running CI. It's easy to end up having 16 jobs when trying to test a number of different configurations. My approach to solving this was to force an AZP style matrix by having the only matrix sequence be a list of build indices, and then using the include feature (designed for tacking additional variables onto specific matrix builds) to define each of the index builds independently. The result is fewer builds, but still good representation of OCIO build options.
  • Each platform has Release, Debug, Python 3, Python 2, SSE, no SSE, static, shared, and on Linux GCC and Clang. Some commented workarounds are present which will be removed after a ci-ocio:2020 image is added to the aswf-docker project. I will submit issues/PRs there for needed changes and patch our workflow in later PR(s).
  • I have disabled CI when the only changes are .md files to ease PRs for TSC notes and documentation which does not need to be built. Hopefully that will not cause issues with required checks for these PRs. We can easily change that if it does.
  • I tested all the builds in my fork successfully, including the latest and SonarCloud builds. Having CI directly in GH is quite nice!

Signed-off-by: Michael Dolan michdolan@gmail.com

Signed-off-by: Michael Dolan <michdolan@gmail.com>
@michdolan
Copy link
Collaborator Author

Here is the successful full GH Actions build for reference, since it will not trigger in this PR:
https://github.com/michdolan/OpenColorIO/actions/runs/80819230

Signed-off-by: Michael Dolan <michdolan@gmail.com>
@michdolan michdolan closed this Apr 18, 2020
@michdolan michdolan reopened this Apr 18, 2020
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
.github/workflows/ci_workflow.yml Outdated Show resolved Hide resolved
Signed-off-by: Michael Dolan <michdolan@gmail.com>
@michdolan
Copy link
Collaborator Author

@michdolan
Copy link
Collaborator Author

I am now going to test a full build again by temporarily enabling the scheduled jobs. I will disable them in a follow up commit.

Signed-off-by: Michael Dolan <michdolan@gmail.com>
Signed-off-by: Michael Dolan <michdolan@gmail.com>
@michdolan
Copy link
Collaborator Author

All jobs completed with new changes: https://github.com/michdolan/OpenColorIO/actions/runs/85424974

Signed-off-by: Michael Dolan <michdolan@gmail.com>
@hodoulp
Copy link
Member

hodoulp commented Apr 23, 2020

[Question] The current build image is the ASWF 2019 one, and you plan to transition the build system to the 2020 one. However, I think that the build system must support the latest VFX Platform recommandation plus 1 or 2 older ones.

[Comment] The new build description is better. It would be nice to have the compiler version if that's possible.

@michdolan
Copy link
Collaborator Author

michdolan commented Apr 23, 2020

When I add CY2020 my plan was to keep CY2019 as well. We could add CY2018, but in terms of OCIO dependencies there is not a difference (compilers have not changed between the three years). Even between 2019/2020 the major difference for us is only the Python version. [edit] OpenEXR version has also changed.

I could re-add compiler version, but it will need to be hard-coded since there is no way to inspect it from the docker container. As a counter argument, if you really want to know the specific compiler version, the job is broken up into really granular steps and within the top lines of the "Configure" step you can see the exact compiler version reported by CMake. But I can add it back in.

@michdolan
Copy link
Collaborator Author

Also, I just remembered why I was using C++14. It's because VFX ref platform mandates it since CY2018. We can keep with C++11, but just mentioning it.

@hodoulp
Copy link
Member

hodoulp commented Apr 23, 2020

When I add CY2020 my plan was to keep CY2019 as well. We could add CY2018, but in terms of OCIO dependencies there is not a difference [...]

Right, and CY2019 & CY2018 are for OCIOv1.

Also, I just remembered why I was using C++14. [...]

So, one build per compiler with C++14 is then needed.

Compiler version in the description

If that's not easy, then forget.

Signed-off-by: Michael Dolan <michdolan@gmail.com>
@michdolan
Copy link
Collaborator Author

Signed-off-by: Michael Dolan <michdolan@gmail.com>
@hodoulp
Copy link
Member

hodoulp commented Apr 24, 2020

Few comments using the latest CI build:

  • The cxx is empty in the build title
    Linux CentOS 7 latest ext packages <Clang 7, config=Release, shared=ON, sse=ON, cxx=, python=3.7, docs=OFF>.
  • The docs build cost is about 3 mins per build so one docs build per platform is enough.
  • The Python 3 install cost is almost 3 mins so it would be great to have it by default in the ASWF docker image (Python 2 & 3 in the ASWF docker images).
  • In the 'latest' builds, there are issues:
    • eXpat problems are: builds tests & tools & examples, build type is unknown and builds a shared library.
    • OpenEXR problems are: builds tests & examples, and PYILMBASE_ENABLE is enabled (it's causing trouble between boost & python3 but that's known OpenEXR issue 684)

Signed-off-by: Michael Dolan <michdolan@gmail.com>
Signed-off-by: Michael Dolan <michdolan@gmail.com>
@michdolan
Copy link
Collaborator Author

@michdolan
Copy link
Collaborator Author

The most recent commit responds to all comments except resolving the Python 3 install in the aswf-docker container. That will be done in a future PR since it will take time to submit that change and roll it into available containers.

Copy link
Member

@hodoulp hodoulp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job @michdolan

@michdolan michdolan merged commit 4b0c538 into AcademySoftwareFoundation:master Apr 25, 2020
@michdolan michdolan deleted the gh_actions_init branch October 21, 2020 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants