Build Chart Data dependency issue

[pranshu1995]

Hi @nreese
I've been trying to migrate your plugin to Kibana v6.1.2
Now the problem is, when trying to import Build Chart Data from 'ui/vislib_vis_type/build_chart_data'
The file doesn't exist in Kibana v6.1.2 at the mentioned path.
I even tried to explicitly copy the required file at the same path, but it gives an error :

TypeError: Expected private module "undefined" to be a function
    at identify (commons.bundle.js?v=16363:50)
    at Private (commons.bundle.js?v=16363:50)
    at Object.<anonymous> (kibana.bundle.js?v=16363:117)
    at Object.invoke (commons.bundle.js?v=16363:29)
    at extend.instance (commons.bundle.js?v=16363:29)
    at nodeLinkFn (commons.bundle.js?v=16363:29)
    at compositeLinkFn (commons.bundle.js?v=16363:29)
    at commons.bundle.js?v=16363:29
    at Object.link (kibana.bundle.js?v=16363:67)
    at commons.bundle.js?v=16363:29 "<vis-editor-vis-options vis="vis" vis-data="visData" ui-state="uiState" visualize-editor="visualizeEditor" editor="tab.editor">"

Respective code :
import { VislibVisTypeBuildChartDataProvider } from 'ui/vislib_vis_type/build_chart_data'; const buildChartData = Private(VislibVisTypeBuildChartDataProvider); // This throws the following error

And it originates here in the bundled file :

var buildChartData = Private(_build_chart_data.VislibVisBuildChartData)

Is it possible that maybe this file exists somewhere else Kibana 6 and I can load it from there.
Or some changes that I can make in the same file to make it work?

Please help out.
Thank you

[nreese]

Thanks for looking into migrating the plugin to 6.0. You should be able to completely get ride of buildChartData. The only reason it was needed was that kibana 5.x did not support filter agg bounding the aggregation call to the map view area. This was added in 6.0 so that functionallity is no longer needed.

You can safely remove the following modifyToDsl, aggFilter, vis.aggs watch, and respProcessor.

Then esResponse should just have table already defined.

Where is your fork? Give the above a try and let me know if you get stuck.

[pranshu1995]

Hey @nreese

I made the changes you suggested and some others in utils.js to make the plugin work.
The visualisation is now loading without any errors.
But now the problem is that data is not being plotted on the map.
I checked the query being sent in Dev-Tools and the data fetched is fine.
Here is my fork : https://github.com/pranshu1995/enhanced_tilemap

Thank you

[carakel]

I happen to be trying to migrate this to Kibana 6.x as well.

The issue pranshu1955 is seeing is because chartData is null in draw() after removing the code recommended by nreese and so it exits without drawing.

I do not understand what should be done here in draw() to set the previously uninitialized chartData to something which holds the data from the esResponse -

  map.addMarkers(
    chartData,
    $scope.vis.params,
    tooltipFormatter,
    _.get(chartData, 'valueFormatter', _.identity),
    collar);
}

I tried tracing through the former buildChartData code to understand what structure should go there, and I have read various plugin-building blogs and documentation, but I can't find anything that shows how to set up this field when adding markers.

Thank you for any help.

[carakel]

Ahh. So you meant to pass esResponse rather than chartData. I tracked down that callbacks.js was using chart.geohashGridAgg and that kibana's maps_visualization.js makes use of esResponse.geohashGridAgg and realized that is what you meant.
Now I have to go through and change all default exports to named exports to see if it will run, due to this error - TypeError: _modules2.default is undefined (http://lXXXX:5601/bundles/kibana.bundle.js?v=16371:117)

[pranshu1995]

Hey @carakel
Can you provide me your fork so I can have a look to understand better.
Thank you

[pranshu1995]

@carakel I am stuck at the same as you are.
ChartData consists of keys geoJson and geohashGridAgg.
The data in geoJson can be used from the esResponse which we are getting.
But geohashGridAgg gets it's data from BuildChartData ( dependency which is not availavle in Kibana 6).
This geohashGridAgg is futher being used in :

 TileMapMap.prototype._attachEvents -->  self._callbacks.mapMoveEnd -->
  mapMoveEnd: function (event) {
        const vis = _.get(event, 'chart.geohashGridAgg.vis');

@nreese Is there a way through which BuildChartData can be used in Kibana 6 because it doesn't seem to work without it.

Thanks

[carakel]

Because of my workplace requirements, I cannot post code/intellectual property publicly without going through a two-month release review cycle.

So, all I can post is that I commented out this chartData call -

$scope.$watch('esResponse', function (resp) {
  if(_.has(resp, 'aggregations')) {

// chartData = respProcessor.process(resp);
draw();

And changed draw() to this --

function draw() {

// if(!chartData) return;

  //add overlay layer to provide visibility of filtered area
  let fieldName = getGeoField().fieldname;
  if (fieldName) {
    map.addFilters(geoFilter.getGeoFilters(fieldName));
  }

  drawWmsOverlays();

  map.addMarkers(
    esResponse,
    $scope.vis.params,
    tooltipFormatter,
    _.get(esResponse, 'valueFormatter', _.identity),
    collar);
}

When it drops through to TimeMapMap's use of chart.geoHashGridAdd, it will be referring to the one in esResponse, not chartData.

I still don't have it running - whenever I run kibana, I get the module2.default error when it loads. I have gone through all of the code in the plugin trying to change the default references to named references but I still get the error. I'm not sure why you don't see it; I am using 6.1.3 and I had to edit lots of files including ones you don't show changing such as - public/lib/jquery.minicolors/minicolors.js, e.g. -

var module = require('ui/modules').get('kibana/enhanced_tilemap');

vs

import { uiModules } from 'ui/modules';
var module = uiModules.get('kibana/enhanced_tilemap')

[nreese]

@pranshu1995 Try moving responseConverter property in vis.js to a top level property. Right now it is under editorConfig but that changed in 6.x.

[pranshu1995]

@nreese The real problem is that chartData in :
map.addMarkers is not available, since you asked to comment out respProcessor and therefore chartData is null which is further used in many modules like : addMarkers, createMarker, etc.

I moved responseConverter as a top level property, and even tried to pass explicitly pass the resp to AggResponseGeoJsonProvider and resp.tables too.
Please tell how to use it exactly and what to pass.
Also I noticed that resp is different in Kibana 5 and Kibana 6.

@carakel Can you look into this too if it works out?

Thanks

[SSa-KrVf47FD]

Hey guy,
I've been trying to get the plugin to work on 6.x and i'm having a hard time understanding some points. First of all, when @carakel said that he replaced chartData with esResponse in the draw function, what's esResponse and where is it initiated ?
Also, i noticed that "if(_.has(resp, 'aggregations'))" located in "$scope.$watch('esResponse', function (resp)" is always false since resp only has the following:

• aggConfig
• key
• tables(array)
• title

[nreese]

@SSa-KrVf47FD Thanks for looking into getting the plugin to run in 6.x. 6.x changed the way plugins interact with the elasticsearch results. In 5.x, the plugin was getting the raw JSON response from elasticsearch which included the aggregation results under aggregations. In 6.x, the plugin is getting the response as formated by https://www.elastic.co/guide/en/kibana/6.x/development-visualization-response-handlers.html, so the if statement should check that tables exists.

[SSa-KrVf47FD]

Thanks @nreese for the quick reply ! I figured as much. But what i still can't figure is what the draw function will use as data. Should the resp from esResponse be processed before calling draw() or should the rest of the functions that use the old chartData be modified to support the new structure ?

[SSa-KrVf47FD]

Hey
I would like to thank you all for your efforts to migrate this very useful plugin to Kibana 6.x ! I managed to get it working on Kibana 6.1.3 (here goes my fork https://github.com/SSa-KrVf47FD/enhanced_tilemap ).
Nevertheless, the visualization in popups is not working. I figured that the vis.type changed in Kibana 6.x and no longer provides a createRenderbot method and i think it's safe to assume that Renderbots no longer exist and that the logic behind them has moved somewhere else. Can anyone please point me in the right direction ?