Automate regression updates

[APJansen]

Aim

Be able to add a red-regressions label and have the regressions be recomputed in the CI and automatically committed.

Status

It seems to be working for the extra regression tests, the result of the workflow applied to this PR as a test is the commit d142744

I don't know about the fitbot, if we want to automate that too I guess it would be a different workflow but using the same label. And what would need to be updated exactly? The developing_weights are a starting point, so I assume those stay the same. Would you want to update the reference fit that it compares against?

Also, I expected the bot's push to trigger the other tests again, but it seems this doesn't happen. I don't know if that's necessary or not.

[scarlehoff]

Thanks!

I think git bot commits cannot trigger again the checks. Let's see whether having the previous check is enough for the branch rules to consider the branch as ok.

Also, I like that we will basically have a redo-regression tag that would allow to quickly know whether they were regenerated in a particular branch!

[github-actions]

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

• Fit Name: NNBOT-24d9e678c-2024-02-16
• Fit Report: https://vp.nnpdf.science/mIR5gNpnS5yZ0tpYwr6Drw==
• Fit Data: https://data.nnpdf.science/fits/NNBOT-24d9e678c-2024-02-16.tar.gz

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

[github-actions]

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

• Fit Name: NNBOT-0eef06ee9-2024-02-16
• Fit Report: https://vp.nnpdf.science/vjzseWbeRoKmjqSPB7CBmA==
• Fit Data: https://data.nnpdf.science/fits/NNBOT-0eef06ee9-2024-02-16.tar.gz

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

[APJansen]

Interesting that the .exportgrid didn't change for any of the fits while the integrability and arc-lenghts did. arc-length changes are due to scipy's integrate module, fine, but the integrability surprises me.

They just changed again, so there is still some source of randomness somewhere. Not really an issue though I think, changes are very small.

[scarlehoff]

Yes, sure. I'm just surprised that integrability changed while the .exportgrid (which contains hundreds of values) didn't.

The integrability asks the NN for a bunch of points and sums them together, while the .exportgrid asks the NN for hundreds of values...

(in any case I agree, it doesn't really matter. I just found it curious).

[APJansen]

Yes, sure. I'm just surprised that integrability changed while the .exportgrid (which contains hundreds of values) didn't.

To be sure everythings ok I checked that if I run it locally, those do get changed.

What about the fit-bot?

[APJansen]

They just changed again, so there is still some source of randomness somewhere. Not really an issue though I think, changes are very small.

Actually that's not true, looking a bit more carefully, only the timings changed now.

[scarlehoff]

What do you mean?

[APJansen]

Earlier I thought the regression data was changed twice, so there was some randomness still, but I think I wasn't looking carefully enough, it was only the timings that changed the second time.

Or if you mean on the fit bot:

I don't know about the fitbot, if we want to automate that too I guess it would be a different workflow but using the same label. And what would need to be updated exactly? The developing_weights are a starting point, so I assume those stay the same. Would you want to update the reference fit that it compares against?

[scarlehoff]

Ah, no, don't update the fitbot like this.
That's a more conscious choice (and it's really just changing the reference in the workflow, nothing needs to be rerun)

[APJansen]

Ok, then this is done I think? I just rebased to master.

[RoyStegeman]

Thanks! This is nice!