Better CI required for the ioc-template - AND ec-services-template

[gilesknap]

I've just had this build failure after adopting the latest ioc-template into ioc-pmac which I believe to be related to the ibek 1.7.0 update to the template. https://github.com/epics-containers/ioc-pmac/actions/runs/8146873609/job/22266338651#step:5:2443

I have confused things by making further changes to the template before trying to adopt it but I'm pretty sure it is the ibek changes that are having an issue here.

@GDYendell I think there is a bit of a problem with pushing changes to the template as it does not test that applying them to a repo works. I'm sure you tested them in the devcontainer but unfortunately it is the runtime stage that has failed.

I'm pretty sure the issue is that there is no globbing of the path passed to ioc extract-runtime-assets call. Best to fix this in ibek - but we also need to think about how CI could do a test update of a generic IOC repo. @coretl I think python copier already addresses this in some fashion??

[gilesknap]

This one is directly update to template 3.1.0 (the ibek 1.7.0 changte).

But it is failing in the developer stage testing https://github.com/epics-containers/ioc-adsimdetector/actions/runs/8147676875/job/22268918354#step:6:115

[gilesknap]

@GDYendell The issue with ioc-adsimdetector is that your lovely changes to ibek-support did not get applied by the template. I've taken a look at this and the version of the submodule that you currently have exists in your .git folder. Therefore updating the contents of the repo will not update which submodule version is in use.

I think I might be able to fix this in a copier task.

This build succeeded
https://github.com/epics-containers/ioc-adsimdetector/actions/runs/8152420647

Therefore my original assertion that it was the ibek 1.7.0 update must be false and my more recent updates to the template must be at fault.

[gilesknap]

All fixed: https://github.com/epics-containers/ioc-pmac/actions/runs/8153269759

But this highlighted a few issues:

1. I had a branch of ibek-support with motor record work and another with pvi latest schema changes, they had not yet met. FIXED and merged into ibek-support main.

• but this does highlight some difficulties with ibek-support management and its many subtle effects on generic IOCs.
• TODO: get some decent CI working for ibek support that covers all the issues seen in this discussion (at least)

2. ibek-support issues were exacerbated by my local build script re-initialising the sub module every time and thus confusing me with resepect to what I had changed in ibek-support. FIXED and in ioc-template 3.2.0b3
3. pushing changes to ioc-template does not do any verification that adopting that template won't break the CI of a generic IOC. TODO: need to adopt a similar mechanism to python copier
4. building a generic IOC that has zero pvi references in the support yaml files it uses will blow up on the runtime asset copying stage because the folder ibek-defs is missing. TODO: fix this in ibek by allowing this folder to be missing.
5. updating ibek-support version in the ioc-template has no effect when adopting it. TODO: I think we should leave updating of ibek-support version as a separate documented step for upgrading generic IOCs. The idea of using a task in copier is going to be fiddly and explicit control of the ibek-support version will be less error prone.

TLDR: I'm still not convinced I haven't created an over-complex monster - it is definitely a good idea to get the pvi and ibek schemas locked down before we let this into the wild.

[coretl]

I'm pretty sure the issue is that there is no globbing of the path passed to ioc extract-runtime-assets call. Best to fix this in ibek - but we also need to think about how CI could do a test update of a generic IOC repo. @coretl I think python copier already addresses this in some fashion??

The python-copier-template repo has a set of example answers, and keeps the python-copier-template-example repo up to date with them on push to main and the tests test that:

1. The template expands with the example answers
2. The example repo updated with the latest version of the template gives the same output as the freshly expanded version
3. tox -p in the expanded template works

[coretl]

Note that the example repo is a pristine expanded template, so 2. is really testing the copier framework rather than the template, so I'm not convinced it adds much unless the example repo is filled with some user changes, but that would make checking it against a pristine expansion more tricky to write

[gilesknap]

@coretl I think we want something like that. I'm imagining using a real generic ioc like ioc-pmac and then checking that its CI works. Do you verify the CI of the example repo somehow?

[coretl]

I'm imagining using a real generic ioc like ioc-pmac and then checking that its CI works

I'd say that your tutorial ec-services repo is a good candidate for this, but not sure if there's any testing in that?

Do you verify the CI of the example repo somehow?

No, didn't have time to implement that. I just keep an eye on the CI of the example repo.