Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin goes down when used with Grafana Docker Container. #133

Closed
ghost opened this issue Nov 11, 2022 · 12 comments
Closed

Plugin goes down when used with Grafana Docker Container. #133

ghost opened this issue Nov 11, 2022 · 12 comments

Comments

@ghost
Copy link

ghost commented Nov 11, 2022

Hi!
So, I observed something weird when working with this plugin and using the grafana docker container.

I started my grafana docker instance using this command,

docker run -d --name=grafana -p 3000:3000 grafana/grafana-enterprise

Grafana started up without any issues. I was able to access it on port 3000 via the browser.

Then, I went to the plugin's section via the gear icon on bottom left of the homepage. Screenshot shown below

Screenshot 2022-11-11 at 2 51 34 PM

After this, I installed this plugin.
Screenshot 2022-11-11 at 2 52 38 PM

Now, when try to create a panel, I get this error: "An error occurred in the plugin". This happened before I can type anything.

Screenshot 2022-11-11 at 2 55 32 PM

I see these logs from the grafana container,

logger=installer.fs t=2022-11-11T09:24:56.7883905Z level=info msg="Downloaded and extracted grafadruid-druid-datasource v1.4.0 zip successfully to /var/lib/grafana/plugins/grafadruid-druid-datasource"

logger=plugin.loader t=2022-11-11T09:24:57.0169687Z level=info msg="Plugin registered" pluginID=grafadruid-druid-datasource

logger=plugin.grafadruid-druid-datasource t=2022-11-11T09:25:25.693593Z level=info msg="DRUID EXECUTE QUERY" _________________________GRAFANAQUERY___________________________="map[Interval:1.5e+10 JSON:map[datasource:map[type:grafadruid-druid-datasource uid:9-6KULD4k] datasourceId:1 intervalMs:15000 maxDataPoints:1488 refId:A] MaxDataPoints:1488 QueryType: RefID:A TimeRange:map[From:2022-11-11T03:25:25.648Z To:2022-11-11T09:25:25.648Z]]"

logger=context userId=1 orgId=1 uname=admin t=2022-11-11T09:25:25.6984401Z level=error msg="Internal server error" error="[plugin.downstreamError] failed to query data: Failed to query data: rpc error: code = Unavailable desc = error reading from server: EOF" remote_addr=172.17.0.1 traceID=

logger=context userId=1 orgId=1 uname=admin t=2022-11-11T09:25:25.6985415Z level=error msg="Request Completed" method=POST path=/api/ds/query status=500 remote_addr=172.17.0.1 time_ms=12 duration=12.3085ms size=116 referer="http://localhost:3000/dashboard/new?editPanel=2&orgId=1" handler=/api/ds/query

After this, any attempts to add a query result in continuous errors
Screenshot 2022-11-11 at 2 59 10 PM

logs says this,

logger=context userId=1 orgId=1 uname=admin t=2022-11-11T09:28:44.0246852Z level=error msg="Internal server error" error="[plugin.unavailable] plugin unavailable" remote_addr=172.17.0.1 traceID=

logger=context userId=1 orgId=1 uname=admin t=2022-11-11T09:28:44.0248356Z level=error msg="Request Completed" method=POST path=/api/ds/query status=500 remote_addr=172.17.0.1 time_ms=2 duration=2.1185ms size=98 referer="http://localhost:3000/dashboard/new?editPanel=2&orgId=1" handler=/api/ds/query

I am assuming that an empty query was issued by grafana to the plugin. This caused the plugin to crash and not recover after that. However, I am not sure why this happens only on the grafana container. These exact steps seem to work fine when I run grafana locally.

An interesting thing is the difference in the log statement, the docker plugin has this _________________________GRAFANAQUERY___________________________ whereas, when run locally, there is a space between GRAFANA and QUERY. I can also see that the log being printed here shows a space between GRAFANA and QUERY

Could it be that the versions getting installed on the container and locally are different? I verified these and found them to be the same.

The same behavior was also observed when I hosted this docker instance on AWS EKS.

I will be happy to fix this myself with a few pointers.

@ghost
Copy link
Author

ghost commented Nov 11, 2022

@jbguerraz I'll be happy to fix this with some pointers.

@saketbairoliya2
Copy link
Contributor

@MohammadGhazanfar - I was facing similar issue but installing it using cli fixed the issue. Also, you can configure the druid credentials directly in docker/grafana/provisioning/datasources/druid.yml. Let me know if this helps.

@ghost
Copy link
Author

ghost commented Nov 18, 2022

Thanks @saketbairoliya2 !
I placed a config file in provisioning/datasources and that seems to work around this!

@ghost
Copy link
Author

ghost commented Nov 18, 2022

Keeping this open because it still tracks an issue with the plugin.

@alvaro-ponce
Copy link

Any news?
Facing the same problem after upgrading from grafana 8.5.15 to 9.3.2

@jbguerraz
Copy link
Contributor

Hello @alvaro-ponce , @MohammadGhazanfar,

How is the datasource configured ? does the connection test pass there ?

@ghost
Copy link
Author

ghost commented Jan 12, 2023

Hey @jbguerraz!
Configured the data source from the Grafana UI. Yeah, the connection test passes.

@ghost
Copy link
Author

ghost commented Jan 12, 2023

@alvaro-ponce you can work around this by provisioning the datasource using a file. See @saketbairoliya2 's above.

@jbguerraz
Copy link
Contributor

I indeed reproduced the issue @MohammadGhazanfar

Interesting is that if you go to explore at first, then run a query, it works.
Or if you restart the docker container, then it also kinda works (plugin crash but get restarted).

The bug on the plugin side is here: https://github.com/grafadruid/druid-grafana/blob/master/pkg/druid.go#L399
somehow Grafana sends a query without Builder (despite https://github.com/grafadruid/druid-grafana/blob/master/src/QueryEditor.tsx#L16 ) and the code is not ready for that: we better make sure there is map and not a nil value before using it.

Maybe here https://github.com/grafadruid/druid-grafana/blob/master/pkg/druid.go#L392 , with something like:

if err != nil || q.Builder == nil || q.Settings == nil

Feel free to fix it @MohammadGhazanfar

@ghost
Copy link
Author

ghost commented Jan 16, 2023

Thanks @jbguerraz ! I will look into this and submit a PR!

@ghost
Copy link
Author

ghost commented Jan 24, 2023

@jbguerraz Created a PR here for this: #138

@jbguerraz
Copy link
Contributor

Thank you @MohammadGhazanfar ! Closing this issue since #138 is merged now 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants