Document the developer process for "pushing" submissions (p6) [13,5,2.6]

[stenington]

This doc describes some submission gotchas and mentions that a developer needs to intervene to address them. Expand this documentation with a developer section that details the processes and tools for resolving the possible issues.

[stenington]

From #2280

[stenington]

Last year I believe we ran fix_submissions periodically, which does a few things:

1. looks for onboarding students that can be marked as onboarded
2. looks for onboarding teams that can be marked as onboarded
3. looks for submissions that only need to be published and publishes them

I think the task would go ahead and make any changes it deemed appropriate and output info about what it was doing, which I saved as logs each time in case we needed to go back and audit later.

In the planning for this year, we're currently looking at a different way of doing things where the program team would pull

1. a list of submissions that can be published, and
2. a list of submissions that should be unpublished

and then hand those lists to a developer to make it happen.

There's a discrepancy here though, in that the admin won't properly report submissions that can be published or should be unpublished unless we're somehow making sure that student and team onboarding is all marked correctly in the database. #2381/#2388 is potentially handling that for students, but we would still need a team piece in place, and then we'd need a tool that takes a list of teams and only publishes those.

Alternatively, we can adapt the April plan to do something more like last year: let the tool(s) determine the list of students/teams/submissions to adjust, and let the program team audit that list to call out any problems they see after the fact.

[stenington]

I added two Developer note sections in the current docs outlining what we have, but I think what I'm seeing here is that we need to coordinate with the program team and see if there are new tools to build.

[stenington]

I want to keep the scope on this one fairly tight, but not lose track of the bigger issue. I opened #2392 to capture the overall theme here.

[stenington]

I think I've documented the process as it stands, at least at a high level, and wrote tests around the fix_submissions rake task as well. I've opened a theme ticket to track the expanded scope of being ready to support the program team overall through the season, and so I want to consider this one complete, if possible.

@shaunxp20 can I ask you to review the tests in PR #2393, and review the documentation here? I'm admittedly taking a "minimum viable" sort of approach to the docs, and not specifying the exact steps for running the tool in production, whether we want to require local test runs first, etc. I'm open to you saying that the docs don't feel complete enough, especially if you can enumerate the additions you would want to see, although I don't think we should be duplicating Heroku docs on the actual mechanics of heroku run ... or anything like that.

[shaun-technovation]

@shaunxp20 can I ask you to review the tests in PR #2393, and review the documentation here? I'm admittedly taking a "minimum viable" sort of approach to the docs, and not specifying the exact steps for running the tool in production, whether we want to require local test runs first, etc. I'm open to you saying that the docs don't feel complete enough, especially if you can enumerate the additions you would want to see, although I don't think we should be duplicating Heroku docs on the actual mechanics of heroku run ... or anything like that.

I reviewed #2393, and the documentation. I think I'm good as far as understanding the issues/solutions. 👍

Just a random 2 cents, I think it would be nice to reduce some of the factors/moving parts though. Like, it'll be nice to get #2388 running (and ultimately address/fix #2372), and do something similar for addressing the team's has_students/all_students_onboarded issue. 🤷 😸

[stenington]

Yep, I agree 100%. I opened #2392 to track that as a theme, because I agree there are several moving parts and it would be nice to feel like they're moving in sync as opposed to being a big grab-bag of possibly broken tools and processes that we can try. I'm hoping that theme ticket can get refined and generate more work in that direction over the next iterations. Feel free to use it to capture relevant thoughts!