Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add telemetry for application usage #52737

Closed
alexfrancoeur opened this issue Dec 11, 2019 · 17 comments · Fixed by #58401
Closed

Add telemetry for application usage #52737

alexfrancoeur opened this issue Dec 11, 2019 · 17 comments · Fixed by #58401
Assignees
Labels
Feature:Telemetry Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v7.7.0

Comments

@alexfrancoeur
Copy link

alexfrancoeur commented Dec 11, 2019

We need a consistent way to monitor the usage of an application. There are a number of applications in our navigation, and with the changes we plan to do to the navigation, it'll be more important than ever to understand what apps get used over others.

I realize that "usage" of an application can be defined differently for each app, but if we have a metric that is consistent across application we can easily compare across them. For instance, how many clusters are using the Logs UI vs. Discovery? What ratio of clusters are using Observability and Security apps? These are questions we can't really answer today, at least in a consistent manner. The most obvious metric that comes to mind is clicks, I could see a format like the one we've used for Lens being useful.

"stack_stats": {
      "kibana": {
        "plugins": {
           "observability": {
             "logs": {
                 "available": true,
                 "enabled": true,
                 "clicks_total": 1000,
                 "clicks_30_days": 10,
                 "clicks_90_days": 100
               },
             "apm": { ...},
             ...
       }
  }

@bmcconaghy it feels as if the future core UI team would implement this type of telemetry, but we need these type of metrics sooner than later, particularly with the nav changes around the corner. Is this something the @elastic/pulse team might be able to think through for now? It might be a good exercise to familiarize some of the newer members of the team with our telemetry collection.

cc: @tbragin @AlonaNadler as I know you've both also been thinking about this. What do you think?

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-stack-services (Team:Stack Services)

@bmcconaghy
Copy link
Contributor

Given the expected timeline for delivery, it seems the only way this would get done is if the Pulse team takes it on, so that seems like the way forward to me for now.

@TinaHeiligers
Copy link
Contributor

@epixa Would this request fall under #52515?

@epixa
Copy link
Contributor

epixa commented Dec 11, 2019

@TinaHeiligers That's unclear to me as well. @alexfrancoeur is going through and making a bunch of issues, but he said he'd make sure to track the ones that are critical for that.

@alexfrancoeur
Copy link
Author

I would say that this is not directly tied to the usage data we want to capture in #52515. I'll provide more context in that issue once we've ironed out internally. We are planning to make some changes to the navigation in 7.6 and while not a hard requirement, it would be great to target for 7.6 inline with the nav changes. While it'd be amazing to have this level of information, we shouldn't treat this telemetry as a blocker for those changes.

@alexfrancoeur
Copy link
Author

@tbragin @VijayDoshi @hmnichols @mriley this is something we'd like to still prioritize for 7.6 (though it may be tight), any insight into the metrics that would be most helpful to you to understand application usage would be helpful. No pressure, just thought I'd ping to see if any others had ideas, particularly around general solution usage monitoring.

@VijayDoshi
Copy link

This slipped by me... just now seeing the mention, apologies!

@alexfrancoeur
Copy link
Author

For anyone interested, we're working in this PR #53849 and are now targeting 7.7. Ideally, we'll merge in line with the nav changes being worked on here #53313

@afharo
Copy link
Member

afharo commented Jan 21, 2020

I'm having a hard time to get this done because I'm finding many different ways in which the apps are registered/mounted, so I have to add listeners to each one of them to report that app is been used:

  1. The first one is the NP's applicationService.currentAppId$.
    But, when running in legacyMode (basically, always), it's constantly undefined.
  2. This leads to the Legacy rendering service (actually the chrome service).
    This covers the apps:
    • discover
    • visualize
    • dashboard
    • maps
    • dev_tools
  3. There are some other apps that overwrite the chrome's rootTemplate.
    This covers the apps:
    • apm
    • uptime
    • infra (no differentiation between logs and metrics)
  4. We also have these 2 apps registered via the NP way:
    • ml
    • siem
      So you would expect them to be logged in the subscription set in point 1. But it always logs undefined in any case.

@elastic/kibana-platform any idea what else can I check or why undefined gets emitted over and over in the observer currentAppId$?
Is there any way we can simply rely on the currentAppId$ observer and properly update it with the right app.id whenever does it change?

@pgayvallet
Copy link
Contributor

@afharo as discussed on slack, this is not an expected behavior from currentAppId$. It should be the single source of truth regarding currently mounted/displayed app. Created #55444

@afharo
Copy link
Member

afharo commented Jan 22, 2020

Thank you @pgayvallet! I tested your changes and they work great! This greatly simplifies my job now :)

@afharo
Copy link
Member

afharo commented Jan 22, 2020

@alexfrancoeur I am wondering whether it might be a good idea to have a tick every N seconds that stores the application the user is in, instead of relying on number of clicks.
Counting clicks might be a bit misleading because clicks/key events might have a huge spike when the user is analysing some specific bits of information or even designing some visualisations/dashboards. But we also have other types of users that will simply keep a specific visualisation/dashboard/metrics view opened and the auto-refresh "on" to monitor their cluster/applications/infra. If we only rely on the clicks, these users are not taken into account.

@epixa epixa added the v7.7.0 label Jan 27, 2020
@rayafratkina
Copy link
Contributor

Would this apply to monitoring something like Lens usage too? (I don't see it in the list above, but we have slightly different requirements so I wanted to make sure we are not missing these, assuming they apply)

@alexfrancoeur
Copy link
Author

@rayafratkina the requirements for this issue were meant to track applications available in the nav in a generic fashion so we can compare usage of apps. I could see any application being applied here though. Lens would be an interesting one to track (with the same metrics) to compare our users time in visualize vs. Lens. I'll leave it to @afharo to weigh in on the complexity of implementing for additional applications out of the original scope, but I'm happy to have a quick zoom to discuss if it's easier.

@afharo
Copy link
Member

afharo commented Feb 4, 2020

I've done a quick test and lens is essentially reported as an app ID different to visualize, so, initially, it will be analysed as a different app.

@alexfrancoeur
Copy link
Author

🎉 🎉 🎉 nice one @afharo !!

@lukeelmers lukeelmers added the Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc label Oct 1, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Telemetry Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v7.7.0
Projects
None yet
10 participants