Fix Travis Cron monorepo split

[TomasVotruba]

No description provided.

[JanMikes]

Hey, what is the problem? Is there a way i could help you with this one @TomasVotruba ?

[TomasVotruba]

Hey, I'd pleasured! It might be an easy pick.

This line doesn't run: https://github.com/Symplify/Symplify/blob/master/.travis.yml#L53

Maybe there shall be only cron condition and no need for the MONOREPO_SPLIT

Could you try? We'll see if that works when cron is run. Once daily by Travis, cannot be speed up

[TomasVotruba]

It looks like the "CRON" condition is not detected: https://travis-ci.org/Symplify/Symplify/jobs/391229218#L636

$TRAVIS_EVENT_TYPE is probably empty

[JanMikes]

$TRAVIS_EVENT_TYPE is probably empty

Can we debug it somehow? etc echo $TRAVIS_EVENT_TYPE ? it does not necessarily be during the cron but trigger it manually and just see if it displays something

[TomasVotruba]

Maybe, I'm not sure. I'd try Googling it + PR

[JanMikes]

@TomasVotruba this seems to be weird to me, both variables are set: https://travis-ci.org/Symplify/Symplify/jobs/391288480#L609 (note that pull_request vs cron, because of the trigger type)

[TomasVotruba]

Same here. That's why it's broken I guess :)

[JanMikes]

I would suggest then adding the echo and waiting for next run, to check out what is going on.

Now we are like blind men ⬛️ because do not know the output of these variables during the cron job

[JanMikes]

Depends what is the desired behaviour -> do you want to split monorepo on any trigger or the cron only?

[TomasVotruba]

On cron only, because it takes around 4-6 minutes

[TomasVotruba]

I would suggest then adding the echo and waiting for next run, to check out what is going on.

Ok. Have you tried Googling that var?

[JanMikes]

Yeah - https://docs.travis-ci.com/user/environment-variables/ - TRAVIS_EVENT_TYPE: Indicates how the build was triggered. One of push, pull_request, api, cron. which seems to be fine.

[TomasVotruba]

Great!

I think it would be better to include this link: https://docs.travis-ci.com/user/environment-variables/#Default-Environment-Variables as reference to that varible use, so we don't have to look for that anymore.

I'd also keep echo $TRAVIS_EVENT_TYPE to test this for the future, in case Travis changes API, so we don't have to debug it this waiting way anymore.

[JanMikes]

Nice idea!

[JanMikes]

Do we have any updates on this? ;) Results, outcomes?

[TomasVotruba]

I'm not working on this. Results are in Travis logs, every day a new one :)

[JanMikes]

Oh yea, i had issues finding out the log from cron job, finally i was successful.

The log is pretty clear about that what is wrong: https://travis-ci.org/Symplify/Symplify/jobs/394061090#L696 git: 'subsplit' is not a git command. See 'git --help'.

Trying to figure out how to fix it.

If you were ever about looking for it, it can be found like this:

[TomasVotruba]

Oh, that's great finding! Good job 🏹

This command is missing: https://github.com/dflydev/git-subsplit

[JanMikes]

Hopefully will be fixed by #916

[JanMikes]

Oh men,... https://travis-ci.org/Symplify/Symplify/jobs/394514575#L702

Let's create composer package for it? :-)

[JanMikes]

Opened issue for it - dflydev/git-subsplit#34

[TomasVotruba]

Damn.

That project is unmaintained for years. Maybe a new SplitCommand could be added to packages/MonorepoBuilder

Reference: https://github.com/DeprecatedPackages/Monorepo

[TomasVotruba]

On related note: https://medium.com/@cedric.thibault/how-to-manage-asynchronous-process-in-php-with-symfony-4-2f0039d76fba

[TomasVotruba]

Work in progress...

[TomasVotruba]

One more step finished, it looks it's working: https://travis-ci.org/Symplify/Symplify/jobs/398788080#L782

But the build is too fast, only 20 secs. It takes 3 minutes on my machine, so we need to still verify these data are pushed to the repositories.

[JanMikes]

Seems good, right? 👍 And 20 secs is super lightning-speed, in case it works 😄

[TomasVotruba]

Never trust something that is way too slower/faster on another machine :)

I merged #934 and got new data:
https://travis-ci.org/Symplify/Symplify/jobs/399089255#L695

I'll try to get more debug output

[TomasVotruba]

@JanMikes Could you check if the split is working now? I'm very near burnout on this 🗡️

[TomasVotruba]

I've added test line to Travis: https://travis-ci.org/Symplify/Symplify/jobs/401029257#L886

Current state

WIP...

[JanMikes]

Hi! Sorry for such delay, i would love to help you and will do so, if this can wait like 1 more day, right now im literally saving someone's ass 😸

[TomasVotruba]

Cocaine overdose again? :D

[JanMikes]

Well, i wish 😄 TLDR. 3rd party mail provider's system failed to send 900+ emails, they have absolutely no evidence of that, guess what, i have all of those in my logs so i am able to recover and resend them 😄 this reminds me of my "how logging saved our ass" talk 😄

[TomasVotruba]

Hehe, nice job 👍 That sounds you're like "crisis solver". "When you fuck up, call me" 👍

[TomasVotruba]

Almost there.

The last issue to solve at the moment is: #937

Since Travis clones repository without tags: https://travis-ci.org/Symplify/Symplify/jobs/401449760#L407

we have to find a way how to get all tags of specific repository.

Maybe something like this could work (Stackoverflow answer)?

git ls-remote --tags https://github.com/Symplify/Symplify.git

So not bind to a local repository, but to the remote one

[TomasVotruba]

Still a bit struggle: https://travis-ci.org/Symplify/Symplify/jobs/402213335#L717

Try running command locally

[JanMikes]

Hey! Im back 😄 So you want to split only those repos that were not split before, right?

Without it does it work? Because it is only "enhancement" and we should focus on get it working first.

[TomasVotruba]

Hey, I'm not sure what you mean :), but rather than expalining all what happened here, take another issue please.
I think I can handle it and it would take me more energy to explain it than finish it.

[TomasVotruba]

Done:

commit pushed by split httpss://github.com/Symplify/MonorepoBuilder/commit/041322c83d986ed3f975a56b7e26767e79f42441

• successful master Travis run: https://travis-ci.org/Symplify/Symplify/jobs/402334538#L1224

[TomasVotruba]

Thanks for your help Honza. You saved me many steps that I saw as a blocker