Ubbagent does not seem to be sending data

[maelvls]

It seems like there is something wrong with the ubbagent container that runs inside the cert-manager controller pod in 1.1.0-gcm.3.

Error 403: Permission 'servicemanagement.services.check' denied on service '<redacted_3rd_party_service>'

Full log:

I0311 16:34:28.003237       1 servicecontrol.go:87] ServiceControlEndpoint:Send(): serviceName:  jetstack-secure-for-cert-manager.jetstack-public.appspot.com  body:  {"operations":[{"consumerId":"project:pr-2e981f0d7882bc90","endTime":"2021-03-11T16:34:28Z","metricValueSets":[{"metricName":"jetstack-secure-for-cert-manager.jetstack-public.appspot.com/time","metricValues":[{"endTime":"2021-03-11T16:34:28Z","int64Value":"60","startTime":"2021-03-11T16:33:28Z"}]}],"operationId":"423286a5-edaa-45de-80cb-69dca73814d3","operationName":"jetstack-secure-for-cert-manager.jetstack-public.appspot.com/report","startTime":"2021-03-11T16:33:28Z","userLabels":{"goog-ubb-agent-id":"10663b6a-7a4b-4015-abd4-a4f6e2fdc1e1"}}]}
E0311 16:34:28.016559       1 retry.go:216] RetryingSender.maybeSend [*googleapi.Error - will NOT retry]: googleapi: Error 403: Permission 'servicemanagement.services.check' denied on service '<redacted_3rd_party_service>'., forbidden

converted to YAML, the body is:

operations:
  - consumerId: "project:pr-2e981f0d7882bc90"
    endTime: "2021-03-11T16:34:28Z"
    metricValueSets:
      - metricName: jetstack-secure-for-cert-manager.jetstack-public.appspot.com/time
        metricValues:
          - endTime: "2021-03-11T16:34:28Z"
            int64Value: "60"
            startTime: "2021-03-11T16:33:28Z"
    operationId: 423286a5-edaa-45de-80cb-69dca73814d3
    operationName: jetstack-secure-for-cert-manager.jetstack-public.appspot.com/report
    startTime: "2021-03-11T16:33:28Z"
    userLabels:
      goog-ubb-agent-id: 10663b6a-7a4b-4015-abd4-a4f6e2fdc1e1

It might be due to the fact that I did not know what to put in the Application deploy-info annotation. Currently, the annotation looks like this:

apiVersion: app.k8s.io/v1beta1
kind: Application
metadata:
  annotations:
    marketplace.cloud.google.com/deploy-info: '{"partner_id": "partner", "product_id": "jetstack-secure", "partner_name": "Jetstack"}'

[maelvls]

I noticed that the service name we have in billing-agent-config.yml has:

# billing-agent-config.yml
- name: servicecontrol
  servicecontrol:
    identity: gcp
    # The service name is unique to your application and will be
    # provided during onboarding.
    serviceName: jetstack-secure-for-cert-manager.jetstack-public.appspot.com  # ❌ is different from below
    consumerId: $AGENT_CONSUMER_ID

But the jetstack-secure-for-cert-manager.yaml sent by Google (see https://github.com/jetstack/platform-board/issues/347#issue-830052570) shows:

# Generated by //cloud/marketplace/tools/price_ingest/build_service_config.py
type: google.api.Service
config_version: 3
name: jetstack-secure-for-cert-manager.mp-jetstack-public.appspot.com            # ❌ is different from above
title: "Jetstack Ltd. Jetstack Secure for cert-manager Reporting Service"
producer_project_id: mp-jetstack-public

control:
  environment: servicecontrol.googleapis.com

metrics:
- name: jetstack-secure-for-cert-manager.mp-jetstack-public.appspot.com/time
  metric_kind: DELTA
  value_type: INT64
  unit: h

billing:
  metrics:
  - jetstack-secure-for-cert-manager.mp-jetstack-public.appspot.com/time
  rules:
  - selector: "*"
    allowed_statuses:
    - current

So I changed this line in our jetstack-secure-for-cert-manager.yaml and it seemed to have started working:

I0312 12:55:21.000486       1 servicecontrol.go:87] ServiceControlEndpoint:Send(): serviceName:  jetstack-secure-for-cert-manager.mp-jetstack-public.appspot.com  body:  {"operations":[{"consumerId":"project:pr-2e981f0d7882bc90","endTime":"2021-03-12T12:55:21Z","metricValueSets":[{"metricName":"jetstack-secure-for-cert-manager.mp-jetstack-public.appspot.com/time","metricValues":[{"endTime":"2021-03-12T12:55:21Z","int64Value":"60","startTime":"2021-03-12T12:54:21Z"}]}],"operationId":"47518ebf-c084-4168-a536-8a117ffec655","operationName":"jetstack-secure-for-cert-manager.mp-jetstack-public.appspot.com/report","startTime":"2021-03-12T12:54:21Z","userLabels":{"goog-ubb-agent-id":"00e22795-be22-4839-b022-acf3120b8a5e"}}]}
I0312 12:55:21.100157       1 servicecontrol.go:111] ServiceControlEndpoint:Send(): success