Bug Assessment Trial Service

[acrymble]

As per issue #1790 we've had a suggestion for a 'big fixer / bug tester' as a paid service provided by ProgHist. This service would support managing editors (MEs) by assessing and making recommendations on bugs that are reported by users, drawing upon our guidelines for retirement.

In order to asses if this is helpful for MEs (the goal is to both resolve bug tickets faster and reduce the effort required of MEs) we are going to try a test of this service on an existing bug ticket.

I will be leading on the test, but the relevant ME will have full control over whether or not to accept the advice.

I hope to have this test completed to share feedback with the team as soon as practical.

[acrymble]

Possible bug candidates for trial:

• Additional step suggested for Mac users of "Building Static Sites with Jekyll" Additional step suggested for Mac users of "Building Static Sites with Jekyll" #1669
• Python lesson sequence and accessing files as binary Python lesson sequence and accessing files as binary #1777

I could not tell if the others were or were not covered by the pull request @walshbr has in progress re Python lessons. Maybe he can let me know if there are outstanding python bugs to add to the list.

[drjwbaker]

Small note: can we please make sure the bug fixer/tester is made aware of knock on implications of their recommendations (e.g. https://github.com/programminghistorian/jekyll/wiki/The-Programming-Historian-Digital-Object-Identifier-Policy-%28April-2020%29)

[walshbr]

In the case of #1672, the actionable they decided on was just to remove a particular paragraph. And #1662 the suggestion similarly was to modify the text (we should decide what that means re: DOIs before those changes are made). So I think #1669 or #1777 could be candidates for this (neither of those are covered by the PR I have open, which only closes #1768).

[ZoeLeBlanc]

Agree with @walshbr about potential trial issues. @acrymble Just to clarify would this bug tester be part of the technical team at all? It seems like there's some overlap in regards to work, so would be helpful to delineate the responsibilities.

[acrymble]

@ZoeLeBlanc no they would not be a technical team member. As previously agreed, fixing bugs is editorial work not technical work. This would be a service for editors and would only apply to bugs within published lessons. Not wider problems with the website.

[acrymble]

We're moving towards a test on either #1777 or #1669 and will report to everyone when we know more about how it has gone. @svmelton will receive a recommendation in her role as ME and it will be her decision on how to proceed (fix or retire). We'll be doing this with our retirement and DOI policies in mind.

[acrymble]

Just keeping track of this old page on troubleshooting which is live but not linked anymore: https://programminghistorian.org/troubleshooting

A possible solution for reducing false bug reports, if reworked.

[acrymble]

The trial will take place on ticket #1669 and @mariechristineb is going to conduct it. She will make a recommendation to @svmelton as the Managing Editor. We will then report to the team how this test went.

[acrymble]

@mariechristineb has now submitted her recommendation to @svmelton who will make a decision on the particular issue. We'll then be able to report to the team on the bug assessment process.

[acrymble]

The pull request with the fix has now been merged.

[acrymble]

Some suggestions for how we might take this process forward as a clear workflow:

1. Bug reported to ME. They triage reports and close any that are obviously not real bugs or problems with the lesson (eg, people wanting to customize a skill for their own workflow, or reports without enough detail to actually understand the problem). If bug appears genuine, ME contacts Finance Director to check for budget for a fix/assessment. If approved, move to step 2. Else, put bug fix on hold until finances are available. ME alerts all other MEs that there may be a bug in a lesson and if they have or are translating it, they may need to follow the bug ticket.
2. ME contacts an approved Bug Assessor (list to be kept on Wiki) to ask them to assess bug. Outlining £ and standard terms (eg, payments will be made by paypal within 30 days of invoice).
3. Bug assessor first assesses the bug and provides a recommendation to the ME (I can fix / can be fixed but not by me / suggest retiring lesson), drawing upon our retirement policy and new DOI policy. Bug assessors should generally be able to fix problems themselves, so 'can be fixed but not by me' should be rare.
4. ME decides and Bug assessor fixes / someone else fixes / ME retires lesson, as appropriate. Fixer adds a link to the bug ticket on the Wiki page for new DOI requests to give examples of cases we've deemed 'minor' and 'major' changes (https://github.com/programminghistorian/jekyll/wiki/The-Programming-Historian-Digital-Object-Identifier-Policy-%28April-2020%29). This should ensure consistency in future.
5. Bug ticket closed.

[acrymble]

I've added the proposed workflow for this to our Wiki using the copyediting workflow as a guide: https://github.com/programminghistorian/jekyll/wiki/Bug-Assessment

If this is approved at the ProgHist board meeting #1824 I will set everything up for Managing Editors and we can start using it.

[acrymble]

I'm pleased to say we have majority support to offer this service on an on going basis.

Thanks in particular to @mariechristineb and @svmelton for helping us test and refine the process.

I've added instructions on using this service to the wiki and I will email MEs so they can ask any questions.

https://github.com/programminghistorian/jekyll/wiki/Bug-Assessment