Export site metrics in Prometheus

[redblom]

sciencemesh/sciencemesh#30

[redblom]

Develop a metrics package that exports site metrics to prometheus.

[redblom]

@ishank011 So I am able to do grpc calls with basic authentication (user/pass). Now how do I get a token (authenticate my request) if I don't have user. Which is the case for the metrics package. I guess I should use this shared jwt_secret somehow ... ?

[ishank011]

@redblom The way other packages do it is that the HTTP or GRPC services pass the context which contains the token to the package methods like done here. But with the metrics package, since the methods execute in the background as goroutines, this won't be possible.

What would be the use case for this? The metrics methods should ideally be unprotected anyway.

[redblom]

@ishank011 The metrics should be retrieved from the underlying EFSS/storage. The CS3 api has useful methods to get this data so why not use that (I would think it's the preferred way). But the CS3 api requires authentication.
Or is there another way ???

[redblom]

@hugo Hi Hugo, could you enlighten me with your view on the above?

[labkode]

Hi @redblom, using the CS3APIS will be an option, note that currently there are not such method to obtain this information so the APIS need to be extended. I'll discuss it with @ishank011 today and let you know by noon.

[ishank011]

@redblom, apologies for the delay. I was on leave for the last couple of days. After discussing with @labkode, we decided to go with not adding the stats to CS3APIs, since the primary method to gather these stats would be to execute external scripts which would need to be executed on the same host.

So I've refactored the metrics package using which you can define multiple drivers to populate stats, and these can register their views themselves, making it more flexible to add new stats. The next step is to complete the logic for the script driver, which would execute external scripts to gather the stats. PR: #934

[redblom]

@ishank011 Ishank, could you tell me precisely what we are trying to achieve here?
Are drivers and external scripts that same thing? If not what are their roles?
Are the drivers or external scripts supposed to implement the Reader interface? (I understand how an interface works)
Are we, or the vendors, supposed to write these external scripts or drivers?

[ishank011]

@redblom, the drivers represent the multiple ways in which metrics can be read, one of which is to read those using external scripts, which ties with this task: sciencemesh/sciencemesh#125. All the drivers are expected to implement the Reader interface. Also, this change enables us to register metrics from multiple packages and services, whereas previously each of these had to be imported in the prometheus service and registered separately.

@labkode must have discussed with you how to proceed with the above task and how exactly these scripts would be executed. I'm not familiar with how that is to be done so you might want to discuss that with him.

[labkode]

@redblom you're supposed to write a driver that reads the metrics from a file, I'd suggest JSON.
Every site will then use their mechanisms to put this file into a place that Reva can read so the metrics can be exported.

[redblom]

PR #973 created. @ishank011 : ready for review.