Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dashboard stops working properly after 50 000 hits #2456

Closed
kyyberi opened this issue Apr 14, 2017 · 9 comments
Closed

Dashboard stops working properly after 50 000 hits #2456

kyyberi opened this issue Apr 14, 2017 · 9 comments

Comments

@kyyberi
Copy link

kyyberi commented Apr 14, 2017

Description

I generated traffic to avoindata.net API with some scripts (10 pcs running in parallel generating >some thousand hits per hour). In the morning I noticed that charts are not shown, I have no idea of the real call amount (since all I see is 50 000 hits, which is not true).
50000

@kyyberi
Copy link
Author

kyyberi commented Apr 14, 2017

In addition in the charts I only see Fri 14 even though I demanded to see 11.4.2017-14.4.2017. Expected: show the requested timeframe in the chart.

@kyyberi
Copy link
Author

kyyberi commented Apr 14, 2017

I took offline a few scripts and thus dropped the traffic amount. After that I was able to see chart with hour granularity.
kuvakaappaus 2017-04-14 08-26-18

This 50k limit seems a bit ridiculous. For example one of our prospect customers has 14 000 000 API calls / month. We must be able to show charts for high traffic APIs. Find a way.

@brylie
Copy link
Contributor

brylie commented Apr 28, 2017

We must be able to show charts for high traffic APIs. Find a way.

After discussing dashboard performance in issue #1340, we agreed to use Elasticsearch directly for dashboard data aggregation. @frenchbread ran several experiments with Elasticsearch aggregation which gave promising results.

The follow-up work means re-writing our dashboard code, including each chart and related data aggregation. We will continue the work, but have to see where it fits in the project roadmap/priorities (@bajiat, @ccsr), which currently includes Epic level work such as:

screenshot_20170428_120030

@frenchbread
Copy link
Contributor

This one: apinf/emq-analytics-dashboard#7

It's for EMQ dashboard though, but we can use it as a reference. Moving towards using ES aggregated data on APInf dashboard means getting rid of data-table at some point.

@philippeluickx
Copy link
Contributor

So basically we need to have a prioritization discussion. This has been on the table for way to long.
I'd rather have no analytics than the current implementation. I consider this feature broken.

@bajiat @kyyberi would this get top priority over the other tasks? I've been +1 on this for almost a year now.

@brylie
Copy link
Contributor

brylie commented May 3, 2017

I'd rather have no analytics than the current implementation. I consider this feature broken.

That is a really strong statement with very little foundation. It doesn't really suggest any helpful way forward. Lets keep the conversation and feedback constructive.

@philippeluickx
Copy link
Contributor

I am being constructive here and clearly own my statement. For me this feature is not done. It allows me to more easily prioritize it when looking at it this way.

As to the investigations done so far, I am very happy to see we are making progress to a workable solution. I would like to understand how come we did not push to the end and managed to close this issue? It's been on the radar for months.

@brylie
Copy link
Contributor

brylie commented May 3, 2017

I am being constructive here

By constructive, I mean 'suggest a way forward' rather than simply stating 'the current solution is inadequate and might be better to throw away'.

Some ways to give constructive feedback:

  • Be focused. No hinting. No generalizations. Be specific. Use concrete examples
  • Don't ask too much. Set realistic goals.
  • Be a guide. Supply feedback and suggest a course of action

Inc. Magazine How to Give Constructive Feedback


I would like to understand how come we did not push to the end and managed to close this issue?

We have some process and prioritization challenges to overcome. For the past year we have been scrambling to keep up with incoming, Epic level, requirements from several sources. This is still ongoing, and perhaps growing stronger, as can be evidenced from our Roadmap above.

@marla-singer
Copy link
Contributor

@preriasusi This bug is resolved in a new Dashboard and can be closed
31215001-8e272e00-a9b6-11e7-8bf0-885d00aba7ba.png

@Nazarah Nazarah closed this as completed Dec 8, 2017
@ghost ghost removed the backlog label Dec 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants