Make compensation requests programattically parsable

[m52go]

This is a Bisq Network project. Please familiarize yourself with the project management process.

Description

Currently, in order to determine how funds were allocated in a particular cycle, compensation requests must be manually analyzed and aggregated. This is time-consuming and error-prone.

This mini-project seeks to establish and implement a new structure for compensation requests that can be parsed by a script.

Rationale

The project reorganization implemented in Cycle 10 established a budgeting structure with team leads. This has helped the project look forward and plan how resources should be allocated. But planning is useless if one cannot look backward and evaluate results.

Criteria for delivery

This project should result in:

• a new template for compensation requests
• a bot that "lints" compensation requests as they are made (and edited) to ensure they fit the new template and can indeed be parsed
• a bot that parses compensation requests after a cycle's voting period ends, and adds a breakdown of issuance by functional team as a comment on each compensation request issue

The existing compensation request template and wiki documentation will need to be updated to reflect the new requirements, along with announcements in all major Keybase channels to ensure contributors are aware (#compensation, #dev, #chinese, #transifex, etc).

The project will be complete when the items above are complete: linter, parser, and related communications.

Measures of success

Contributors must make correctly-formed compensation requests on their own (this will demonstrate awareness of the initiative). The linter must alert compensation request makers of mistakes. The parser must make comments on approved compensation request issue with issuence numbers broken down by team.

The project can be considered a success if team leads actually use the issuance numbers provided by the parser bot for budgeting and tracking issuance over time.

Risks

Not applicable as no Bisq code is touched.

The most significant risk is probably a bot that reports incorrect numbers for some reason, but such a mistake should be discovered quickly and only impacts reporting (not issuance or software or anything else).

Tasks

I don't think this project is complex enough to warrant a whole GitHub board, so here's a checklist.

• Finalize compensation request format (Markdown table, YAML, etc)
• Clarify planned results (e.g., issuance breakdown by functional team, anything else?)
• Create and test linting bot
• Create and test parsing bot
• Determine ops for bots (who will host them, where, costs, etc)
• Edit wiki documentation and compensation request template to reflect changes
• Announce changes to contributors
• Follow implementation for 1 cycle: 1 proposal phase and 1 results phase
• Hand off ownership of bots to contributor who can host and maintain the code over time (and establish new roles if necessary)
• Ensure results are used for budgeting

Estimates

Since this is largely a reporting initiative, it probably makes most sense to come out of the growth budget. Ongoing server costs should come from ops.

Maybe 1500 USD is sufficient for the whole project, as described above (initial implementation and documentation)? This is based on it taking a day to create the bots. Ongoing costs for the bots should be very low/negligible. Open to feedback if it any of this is off.

Notes

Tracking issuance in a more automated way is the first step of a bigger drive to report issuance, burn, and trading volume better.

Compensation request details are an important first step to enabling other reporting, so a new project can be created to pursue further reporting once this project has been successfully completed.

I'd like to work on this!

[First edit was an exploration into a yaml formatted compensation request. Unfortunately yaml is extremely sensitive to whitespace indenting and therefore considered troublesome for use in user-generated compensation requests.]

After discussing with @m52go here is a second attempt at understanding how a parsable compensation request might look, using markdown tables. Additional context could be included in the compensation report for human consumption, outside of the table. The bot would sum up the amounts by team, convert to BSQ at the cycle's rate and report issuance by team as well as a total issuance (the user would claim the total in the Bisq DAO).
_{[NB. normally the links would go to specific issues or PRs, but in order to avoid polluting bisq github I removed issue numbers for this example]}

Team	Contribution Link	Amount
Dev	reviewed: Add signed witness filter	308 USD
Dev	reviewed: Tor 0.4.2.6	210 USD
Dev	reviewed: Added graceful shutdown hook	148 USD
Dev	many small reviews	150 USD
Dev	testing v1.3.0, v1.3.1, Tor v2/v3 upgrade, and v1.3.2	540 USD
Dev	release v1.2.9, v1.3.0, v1.3.1	2700 USD
Dev	developed PR: Improve handling of filtered offers and unsupported payment methods	150 USD
Dev	developed PR: Make prompt text readable in dark mode	0 USD
Dev	developed PR: Only use InputValidation when formatting voting stake	0 USD
Ops	Created Tor V3 seednode and testing Tor V3	200 USD
Ops	2x Bisq Seednode Operator	200 USD
Ops	3x Bitcoin Node Operator	150 USD
Support	Mediator	1000 USD
Support	Monitoring the support channel duing my L2 times	480 USD
Support	burning man March 15th 2020	150 USD

[cbeams]

Glad to hear your're interested @jmacxx. What you've posted above is similar to what I had in mind. A couple notes:

1. I think it's important we have a 'front matter' section before the table where the contributor can self-report the total USD expected. The parser will validate that the totals actually add up to that, and will act as a check on both the contributor and the parser adding things up correctly.

2. No need to have 'USD' in every row. Just title the column 'USD Amount' and leave the row values as plain integers.

3. I just want to make sure that the requirement about the bot posting comments back to the issue is clear. It's quite important that we do this for transparency and fulfilling the actual budget tracking requirement that's driving this project. I ask because I think this bit below is kind of easy to miss in the project description above.

The linter must alert compensation request makers of mistakes. The parser must make comments on approved compensation request issue with issuence numbers broken down by team.

Regarding the bot commenting on CRs with issuance numbers broken down by team, I did experiment with that in https://github.com/jmacxx/compensation-bot/issues/2 where you can see the bot making issuance comments. @m52go reminded me that such issuance comments are only to be made after the cycle is complete.

Alerting users of mistakes -> am I correct in understanding the only way to do this is to have the bot comment on the CR whenever it is updated? (if mistakes are detected).

I have put some work into the linter/parser and come up with an HTML/JavaScript test tool that enables compensation requests to be checked. It comes with 4 sample CRs and an edit box where you can try it out with your own CR text.

Screenshot:

Updated the linting tool to the latest 5 column format received from @cbeams & @m52go.
See example @ https://github.com/jmacxx/compensation-bot/issues/5

[m52go]

@cbeams we might have something worth testing. @jmacxx has created bots that evaluate compensation requests that follow this template.

Please check to see if it looks acceptable to you.

It should largely conform to the ideas you and I discussed earlier, with one small difference:

• the "roles performed" section is removed, as discussed, and role items are included in the main contributions table, but line-items for roles include an asterisk in the team name column to designate them as role items (idea being it might be helpful in the future to separate contributions from "overhead"...see template linked above for an example)

For what it's worth, I also drafted bullet points for documentation that would need to be added according to the proposed template and bots:

• compensation requests are parsed programmatically for budgeting, so please make sure your compensation request adheres to the template as closely as possible. there are comments in the template for guidance, and you can always check past requests for reference.
• a linter will check your request once you've removed [WIP] from your issue title and will notify you of any errors.
• all contributions should specify values in usd. only the top-line number in the summary section should be denominated in bsq.
• role reports and related compensation should both be included in the "contributions delivered" table. role items should include an asterisk in the team column (so a regular growth contribution would specify the team as "growth", and a growth team role would specify the team as "growth*").
• multiple smaller contributions can be added into the "contributions delivered" column with just the link field filled in and a total specified below (see example in compensation request template).
• the "contributions in progress" section is ignored by the parser.

If this format is ok, I think it would make sense to proceed with testing...perhaps provision an API key to let @jmacxx test the bots with real requests. Maybe team leads could make their requests using the new format in this cycle.

[cbeams]

I've only just taken a brief look at the template. Looks pretty good; I'm afraid it might end up being confusing for folks, but having real world examples out there should go a long way to making it clear.

I just submitted my compensation request, and unfortunately didn't get it together in this format. While I'm away for the next couple weeks, I'd say just keep going. Sorry I didn't have more time to check this out in depth!

[m52go]

With the new compensation request issue template now merged, all contributors should now use the new format that came about as a result of this project.

Here I'll run through each of the objectives that has been completed so far.

Finalize compensation request format (Markdown table, YAML, etc)

The new format, in a nutshell, simplifies all line-items for contributions and roles into a single table.

• There is no longer a separate section for roles performed.
• Each contribution should specify the team it belongs to.
• As soon as [WIP] is removed from the issue title, the issue will be linted by a bot to make sure it can be evaluated later for budgeting purposes.
  • If a request is valid, it's labeled as parsed:valid and no further action is necessary from the contributor.
  • If it's invalid, it's labeled as parsed:invalid, and the bot will make a comment indicating the error. Then the contributor should correct the request until it's marked as parsed:valid.

The updated issue template should include all details a contributor would need to compose a valid compensation request.

Clarify planned results

Currently the bot will post an issuance breakdown by team after DAO voting when the request is marked as accepted or rejected by the compensation maintainer. This will help achieve the near-term goal of determining total issuance by team. For now, results will be posted in a spreadsheet that will be made public.

The next step is to add context to the raw totals by aggregating line-item titles to give more context to team issuance numbers.

Create and test linting bot
Create and test parsing bot

@jmacxx took care of all of this. Thank you! It's been a pleasure working with you.

Edit wiki documentation and compensation request template to reflect changes

See tentative documentation in the wiki. That article will be reworked so that "Making a compensation request" is a standalone article, but the linked section covers the basics. Aside from the new format (detailed in the issue template) and the linting (errors are posted in the issue), there isn't anything additional users need to know or do.

I'll make announcements on Keybase and in https://github.com/orgs/bisq-network/teams/dao to draw attention to this new format and encourage contributors to test it in this cycle.

[m52go]

Next step is to follow usage and performance for the remainder of this cycle (Cycle 14) to ensure the bot performs as expected.

Once it goes well, it probably makes sense to discuss devops / ownership of the bot.

Then we'll require the new format in Cycle 15 so that all requests can be parsed, so that results are actually useful (only a portion of Cycle 14 requests will be in the new format, so the results won't be useful).

Those numbers will go in a public spreadsheet to help team leads with budgeting and reporting.

At that point, this project will be completed as currently defined, and further efforts (to add context to numbers, etc) can either be added to this project or to a new project.