Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate migrator PRs #1586

Closed
jakirkham opened this issue Jan 6, 2023 · 15 comments
Closed

Duplicate migrator PRs #1586

jakirkham opened this issue Jan 6, 2023 · 15 comments

Comments

@jakirkham
Copy link
Contributor

Earlier today a CUDA arch migrator PR ( conda-forge/faiss-split-feedstock#60 ) was opened. This was then followed by another migrator PR ( conda-forge/faiss-split-feedstock#62 ) recently opened. Both for the main branch.

Meanwhile the conda-forge status page (not sure how often this is updated) still shows faiss-split as needing a CUDA arch migrator PR.

Screen Shot 2023-01-06 at 12 41 15 PM

@jakirkham
Copy link
Contributor Author

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

Great. Leave them both open so I can look at the backend data and logs.

@jakirkham
Copy link
Contributor Author

jakirkham commented Jan 6, 2023

Thanks Matt! Yep not changing anything 😄

Edit: Also commented in those PRs to leave them open

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

Well the good news is that I figured this out. It appears the bot failed to push its metadata back to github and so lost track of the PRs it made. It then remade them the next run.

The bad news is that I cannot tell why it lost track. Github appears to have rejected the push for the specific bot job but the same code is working on other jobs.

Given how intermittent this error is, I doubt we'll be able to solve it without more detailed information. It could also be on github's end. We do try 10 times to push but that may not be enough...

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

One possible workaround would be to have the bot look for a PR with the same title into the same branch and it if finds one, it can assume that PR was lost in a previous run and recover it.

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

For now, close the older one and keep the newer one for each of the two above.

@jakirkham
Copy link
Contributor Author

jakirkham commented Jan 6, 2023

Ah ok. That makes sense. So did the metadata end up getting updated for both of these migrations? Or may this occur again?

Are runs here synchronous? Or is there a risk of having two jobs push around the same time? Wondering if a conflict might be occurring?

Also took a quick look at GitHub status history, but the last event there is from Jan 1st. So not seeing anything relevant.

Lastly what should we do with the PRs? Close the old ones? Ask the bot to re-run the old ones? Something else? Edit: Didn't see your answer. Will close

@jakirkham
Copy link
Contributor Author

One possible workaround would be to have the bot look for a PR with the same title into the same branch and it if finds one, it can assume that PR was lost in a previous run and recover it.

Presumably we would also want to confirm author (to avoid picking up maintainer PRs)

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

Presumably we would also want to confirm author (to avoid picking up maintainer PRs)

We could check that the PR is made by the bot itself too.

@jakirkham
Copy link
Contributor Author

Closed out the old PRs. Commented on the new ones that we are keeping them.

Checked the status page and am not seeing faiss-split as "in-pr" for either migration. When would we expect this to update?

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

The updates take maybe 30 minutes to 1 hour. If you don't see it, lmk but they usually appear.

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

Ah ok. That makes sense. So did the metadata end up getting updated for both of these migrations? Or may this occur again?

Metadata only got updated for the second. This could occur again if the bot fails to push its metadata.

Are runs here synchronous? Or is there a risk of having two jobs push around the same time? Wondering if a conflict might be occurring?

The runs do not appear to be synchronous. I don't see any git conflicts in the logs. I see

fatal: could not read Password for 'https://~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~@github.com': No such device or address

in the logs. If you have ideas on how that could be generated LMK. The same code worked another time.

@beckermr
Copy link
Contributor

beckermr commented Jan 6, 2023

Oh crap. I rotated a bot token right then due to the circleci security issue. I deleted the old token too soon. I think that explains this. 🤦‍♂️

Sorry for the trouble!

@jakirkham
Copy link
Contributor Author

Oh that caused git push issue? Makes sense

No worries

@jakirkham
Copy link
Contributor Author

Looks like the status page recently updated. These are both shown as in PR now

Given this is resolved, let's close this out

Thanks Matt! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants