Decide on a scoring rubric to evaluate travel fund requests

[tobie]

In his proposal, @bensternthal suggested for the CPC to adopt a scoring rubric to evaluate travel fund requests.

This has the benefit of reducing bias in the approval process and making it easier to delegate part of the approval process to Foundation staff if needed.

Tasks

Preview Give feedback

1. Decide whether the CPC agrees with this recommendation
2. Create scoring rubric
3. Get CPC consensus about the data that needs to be collected for scoring rubric (shouldn’t be contentious)
4. Get CPC consensus about the weights for the scoring rubric (potentially contentious)

[tobie]

If we decide to go down the scoring rubric route, here's are some considerations to get us started:

Criteria	Description	Score
Occasion
	In person working session of OpenJS project, collab space, or CPC	x
	OpenJS related event	x
	Other	x
Relevance to OpenJSF activities
	Relevant to multiple projects, collab spaces, or CPC	x
	Relevant to 1 project, collab space, or CPC	x
	Not relevant	x
Individual investment level in relevant activities
	is core contributor	x
	is occasional contributor	x
	is new contributor	x
	hasn't contributed	x
Individual role for the occasion
	Facilitator of working session	x
	Core contributor to working session	x
	Presenter / speaker	x
	Panelist	x
	Other	x
Travel
	intercontinental	x
	domestic	x
Diversity and inclusion
	contributor self-assesses as representative of an underrepresented minority or protected class	x
Employment status
	Employer is OpenJSF member	x
	Employer is not OpenJSF member	x
	Unemployed	x
	Self-employed	x
Nature of contributions
	contributions are made on behalf of OpenJSF member	x
	contributions are made on behalf of employer which is not a member	x
	contributions aren't related to employment, employer is OpenJSF member	x
	contributions aren't related to employment, employer is not member	x
Are contributions done as part of individual's job?
	Yes	x
	No	x
If contributions are done as part of one's job
Has request been filed with employer?
	request has been filed with employer but only got partial funding	x
	request has been filed with employer but got rejected for understandable reasons (e.g. employer is nonprofit or SME)	x
	request has been filed with employer but got rejected	x
	request has not been filed	x
Employer type
	employer is nonprofit / educational	x
	employer is government institution	x
	employer is SMB / coop	x
	employer is funded startup	x
	employer is large enterprise	x
Employer location
	low income country	x
	lower-middle income country	x
	upper-middle income country	x
	high income country	x

[ovflowd]

I like the idea of scoring, but the list above might sound too complex and overboard. Initially, I was thinking of something way simpler...

I'm unsure how I feel about adding employer location and employer type. I feel that for the nature of the requests we get, we don't need to go with such a robust and complex scoring, but that might be just me overthinking here 🤔

Mainly because we don't get that many requests, and if we do, I believe a simple scoring based on the meaningful participation of said individual in said event and how important that event is for us or the member project should be the scoring done here.

But genuinely speaking, I'm not super versed and knowledgeable in this matter. Open to be proven wrong.

[tobie]

The location questions are is in relation to comments made in the Standards WG that giving domestic flights more points than intercontinental ones would favor folks living in tech hubs at the expense of folks living in less privileged regions.

[ovflowd]

The location questions are is in relation to comments made in the Standards WG that giving domestic flights more points than intercontinental ones would favor folks living in tech hubs at the expense of folks living in less privileged regions.

That is a fair point. Thank you for explaining!

[tobie]

Yeah, my hunch is will need to do some serious tuning to get to a solution that's fair and meets our goals, so I don't think we'll be able to avoid digging into folk's situation a little more substancially than we have done so far.

[LeaVerou]

contributor self-assesses as representative of an underrepresented minority or protected class

How would this be assessed? Would they need to list why, or would it basically be a checkbox?

[mikesamuel]

All else being equal, would we score a travel request to an international conference that has no domestic equivalent?

[tobie]

@LeaVerou wrote:

contributor self-assesses as representative of an underrepresented minority or protected class

How would this be assessed? Would they need to list why, or would it basically be a checkbox?

I'm not sure. Do you have a suggestion? I remember you writing on an adjacent topic not long ago.

[LeaVerou]

@LeaVerou wrote:

contributor self-assesses as representative of an underrepresented minority or protected class

How would this be assessed? Would they need to list why, or would it basically be a checkbox?

I'm not sure. Do you have a suggestion? I remember you writing on an adjacent topic not long ago.

I don't think it's a good path to go down to ask people to list what makes them URR. For one, many don't feel comfortable publicly sharing this information. But even if that were not a factor, what will we do, score how diverse someone is?
IMO we should just list examples and have it be a boolean (self-assessed).

[ovflowd]

@LeaVerou wrote:

contributor self-assesses as representative of an underrepresented minority or protected class

How would this be assessed? Would they need to list why, or would it basically be a checkbox?

I'm not sure. Do you have a suggestion? I remember you writing on an adjacent topic not long ago.

I don't think it's a good path to go down to ask people to list what makes them URR. For one, many don't feel comfortable publicly sharing this information. But even if that were not a factor, what will we do, score how diverse someone is?

IMO we should just list examples and have it be a boolean (self-assessed).

Note that they wouldn't be publicly sharing as it would be on a form 👀

[dylans]

Apologies for potentially jumping into the middle of this without having attended any of the discussion calls.

While URR and financial situation are important things, I personally wouldn't consider them factors for the general travel fund. This may seem like a surprising opinion coming from me as I'm generally very in favor of investing money towards diversity and URR.

I would say it's better to have a general travel fund and a separate fund to sponsor URR or provide scholarships, etc.

I probably also would not add weight/value in an employer covering part of the person's travel expenses (though it's certainly appreciated).

Why does that matter? It's highly likely that someone super valuable to the foundation and its projects is fully employed and well paid and they're probably asked to attend many events. Their employer is happy (or grudgingly accepts) to provide that person's time to attend the event, but maybe the value of the event isn't all that important to the company.

When someone asks for travel funds to represent the foundation, it's because their company is not going to cover their costs, and so they're deciding whether they should cover it personally or have the foundation cover it. (Or maybe their company recently had a round of layoffs and it would feel insensitive to ask for budget after their colleagues were let go.)

At that point the question is really "does the foundation find sufficient value in this person attending an event given the costs?" And that is likely a function of how much they contribute to the foundation and/or its projects, how important the event is to the foundation and its projects.

So I guess my suggestion is to peel away most criteria and focus on whether that person should attend that event given the cost to the foundation and the value their contribution provides to the foundation and its projects. And split travel funds for URR/scholarship into a separate bucket with different criteria. Fewer clearer options will make it easier to review/approve and to know what's expected when applying for funds.

[tobie]

Thanks for your input, @dylans. Also totally OK for you to jump in with comments despite not having been able to attend discussions. We want to broadest possible input, and that's what being written + async first enables.

[bensternthal]

Here is the V1 Rubric from the discussion in early December with the very preliminary weights we discussed on December 19th, next step will be validating the below with some use cases. Note the CPC will still have to decide how the Rubric is used.

Item	Score
Reason For Travel: OpenJS Project, Collab Space, or CPC, OpenJS Related Event	+3
Primary Reason For Attendance: Leading, Presenting, Or Panelist	+5
What OpenJS Hosted Project(s) Do You Contribute To (1 or more)	+1
Role: TSC, Core, or Active	+1
DEI: Identifies as from any underrepresented group	+3
Total Amount Requested Less Than X (e.g. $1,500)	+1

[mhdawson]

For -> Role: TSC, Core, or Active, its not clear to me what would merit the +1 ?

[mcollina]

What OpenJS Hosted Project(s) Do You Contribute To (1 or more)

If I recall, somebody needs to be a member of an OpenJS project to access this (this was the initial purpose). I this is a prerequisite.

I would actually bump it to +3.

[mcollina]

For -> Role: TSC, Core, or Active, its not clear to me what would merit the +1 ?

In case of a collab summit, If TSC/TC/Leads members are not there, I suspect not much would happen.

[tobie]

Can we close this? Is the current rubric documented anywhere?

[bensternthal]

Good call, I just created #1260 to add the rubric to the readme. Once that merges we can close.