Q4 2022 Goal: Billing

[damianavila]

Context

We acknowledge the fact we need to effectively bill our communities to be sustainable in the long term.

And there are some aspects of the billing process where Engineering can help with getting the usage data, aggregating that information, and automating the delivery.

We have different sorts of deployments (ie. dedicated vs shared) where usage computation is different and we should account for and develop stuff taking into consideration those differences.

Proposal

• [Billing] Automatically figure out how much a dedicated GCP project spends in a month #1921
• Automatically figure out how much a separate AWS project spends in a month #1922
• [Billing] Automatically figure out how much a shared GCP cluster spends in a month (and map those costs to inner hubs) #1923

Updates and actions

Created child issues to work on that will be further specified by @yuvipanda.

[consideRatio]

Related issues

• Expose cloud billing dashboards for customers with dedicated clusters #1575
• [Billing] Automatically figure out how much a dedicated GCP project spends in a month #1921

[choldgraf]

Question: What's the status of our cloud cost recovery?

I know that last year, @jmunroe undertook a big effort to define the cloud costs for each of our communities and send invoices to them via CS&S. What has happened since then? Note that we now track our cloud cost summary stastics in the KPIs page. So that indicates we're spending between $4-8k a month on cloud (which is almost a full FTE!)

Here are some specific questions / to-dos we should make sure to tackle:

• Have we sent cloud cost invoices for December and January? If not, can we do so?
• Can we document the process we currently follow for tracking cloud costs, even if it is manual?
• Can we give CS&S recommendations for what they should adjust to help us in this process? (Ask Code for Science and Society to have a dedicated category and invoice type for our re-billable cloud costs team-compass#663)
• Who is responsible for this effort? Is it engineering? Or community/partnerships? I think we need somebody to spearhead leading this effort to make sure we don't drop it.

[damianavila]

Have we sent cloud cost invoices for December and January? If not, can we do so?

@jmunroe should have that answer, I don't know.

Can we document the process we currently follow for tracking cloud costs, even if it is manual?

@jmunroe, did you take notes of the manual process the last time you did it?

Can we give CS&S recommendations for what they should adjust to help us in this process? (2i2c-org/team-compass#663)

+1

Who is responsible for this effort? Is it engineering? Or community/partnerships? I think we need somebody to spearhead leading this effort to make sure we don't drop it.

IMHO, the leading actor in the billing process "as a whole" should be the Community/Partnership side because they have a more direct interaction with our communities and CS&S. This issue tried to capture some of the Eng-related pieces (and aspects) of that whole billing process.

[damianavila]

During Q4 there was some progress at #1921.
During Q1, I expect that one finally land in our infrastructure.
The remaining items (child issues) should be addressed in the next billing theme iteration (realistically, proto-Q2 content).

I am closing this one now (as Q4 has long ended) but I know there is an ongoing discussion above, so feel free to keep commenting on this issue in you are inclined to.