Skip to content

ZEPPELIN-830 Improve table display to handle large data #858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
r-kamath wants to merge 12 commits into from
Closed

ZEPPELIN-830 Improve table display to handle large data #858

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
c833f6e
ZEPPELIN-830 Improve table display to handle large data
r-kamath Apr 26, 2016
4e6beb8
fix selenium test failure
r-kamath Apr 26, 2016
7bb508e
ZEPPELIN-830 fix number formatting for table display
r-kamath Apr 28, 2016
de6d134
ZEPPELIN-830 License update
r-kamath Apr 28, 2016
b87245a
ZEPPELIN-830 fix jshint error
r-kamath Apr 28, 2016
b365e7f
ZEPPELIN-830 fix test
r-kamath Apr 29, 2016
a96c94a
ZEPPELIN-830 fix rendering without angular/html directives
r-kamath Apr 29, 2016
adb8380
ZEPPELIN-830 remove unused floatThead dep
r-kamath Apr 29, 2016
2e422ea
ZEPPELIN-830 update licence doc
r-kamath Apr 30, 2016
4fbb22b
Merge branch 'master' of https://github.com/apache/incubator-zeppelin…
r-kamath Apr 30, 2016
dd0ced7
ZEPPELIN-830 fix wrong variable name
r-kamath May 4, 2016
f13ba91
Merge branch 'master' of https://github.com/apache/incubator-zeppelin…
r-kamath May 10, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions zeppelin-distribution/src/bin_license/LICENSE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,10 @@ The text of each license is also included at licenses/LICENSE-[project]-[version
(The MIT License) lodash v3.9.3 (https://lodash.com/) - https://github.com/lodash/lodash/blob/3.9.3/LICENSE.txt
(The MIT License) angular-filter v0.5.4 (https://github.com/a8m/angular-filter) - https://github.com/a8m/angular-filter/blob/v0.5.4/license.md
(The MIT License) ngToast v1.5.5 (http://tamerayd.in/ngToast/) - http://tameraydin.mit-license.org/
(The MIT License) Handsontable v0.24.2 (https://github.com/handsontable/handsontable) - https://github.com/handsontable/handsontable/blob/master/LICENSE
(The MIT License) Zeroclipboard v2.2.0 (https://github.com/zeroclipboard/zeroclipboard) - https://github.com/zeroclipboard/zeroclipboard/blob/v2.2.0/LICENSE
(The MIT License) Moment v2.9.0 (https://github.com/moment/moment) - https://github.com/moment/moment/blob/2.9.0/LICENSE
(The MIT License) Pikaday v1.3.2 (https://github.com/dbushell/Pikaday) - https://github.com/dbushell/Pikaday/blob/1.3.2/LICENSE
(The MIT License) slf4j v1.7.10 (org.slf4j:slf4j-api:jar:1.7.10 - http://www.slf4j.org) - http://www.slf4j.org/license.html
(The MIT License) slf4j-log4j12 v1.7.10 (org.slf4j:slf4j-log4j12:jar:1.7.10 - http://www.slf4j.org) - http://www.slf4j.org/license.html
(The MIT License) bcprov-jdk15on v1.51 (org.bouncycastle:bcprov-jdk15on:jar:1.51 - http://www.bouncycastle.org/java.html) - http://www.bouncycastle.org/licence.html
Expand Down
5 changes: 2 additions & 3 deletions zeppelin-server/src/test/java/org/apache/zeppelin/integration/SparkParagraphIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,9 +167,8 @@ public void testSqlSpark() throws Exception {
WebElement paragraph1Result = driver.findElement(By.xpath(
getParagraphXPath(1) + "//div[@class=\"tableDisplay\"]"));
collector.checkThat("Paragraph from SparkParagraphIT of testSqlSpark result: ",
paragraph1Result.getText().toString(), CoreMatchers.equalTo("age job marital education balance\n" +
"30 unemployed married primary 1,787")
);
paragraph1Result.getText().toString(), CoreMatchers.equalTo("age\njob\nmarital\neducation\nbalance\n30" +
" unemployed married primary 1,787\nage\njob\nmarital\neducation\nbalance"));
} catch (Exception e) {
handleException("Exception in SparkParagraphIT while testSqlSpark", e);
}
Expand Down
3 changes: 2 additions & 1 deletion zeppelin-web/.jshintrc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
"nv": false,
"ace": false,
"d3": false,
"BootstrapDialog": false
"BootstrapDialog": false,
"Handsontable": false
}
}
4 changes: 1 addition & 3 deletions zeppelin-web/bower.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,7 @@
"ngtoast": "~2.0.0",
"ng-focus-if": "~1.0.2",
"bootstrap3-dialog": "bootstrap-dialog#~1.34.7",
"floatThead": "~1.3.2",
"datatables.net-bs": "~1.10.11",
"datatables.net-buttons-bs": "~1.1.2"
"handsontable": "~0.24.2"
},
"devDependencies": {
"angular-mocks": "1.5.0"
Expand Down
134 changes: 33 additions & 101 deletions zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1218,110 +1218,42 @@ angular.module('zeppelinWebApp')
};

var setTable = function(type, data, refresh) {
var getTableContentFormat = function(d) {
if (isNaN(d)) {
if (d.length>'%html'.length && '%html ' === d.substring(0, '%html '.length)) {
return 'html';
} else {
return '';
}
} else {
return '';
}
};

var formatTableContent = function(d) {
if (isNaN(d)) {
var f = getTableContentFormat(d);
if (f !== '') {
return d.substring(f.length+2);
} else {
return d;
}
} else {
var dStr = d.toString();
var splitted = dStr.split('.');
var formatted = splitted[0].replace(/(\d)(?=(\d{3})+(?!\d))/g, '$1,');
if (splitted.length>1) {
formatted+= '.'+splitted[1];
}
return formatted;
}
};


var renderTable = function() {
var html = '';
html += '<table class="table table-hover table-condensed">';
html += ' <thead>';
html += ' <tr style="background-color: #F6F6F6; font-weight: bold;">';
for (var titleIndex in $scope.paragraph.result.columnNames) {
html += '<th>'+$scope.paragraph.result.columnNames[titleIndex].name+'</th>';
}
html += ' </tr>';
html += ' </thead>';
html += ' <tbody>';
for (var r in $scope.paragraph.result.msgTable) {
var row = $scope.paragraph.result.msgTable[r];
html += ' <tr>';
for (var index in row) {
var v = row[index].value;
if (getTableContentFormat(v) !== 'html') {
v = v.replace(/[\u00A0-\u9999<>\&]/gim, function(i) {
return '&#'+i.charCodeAt(0)+';';
});
}
html += ' <td>'+formatTableContent(v)+'</td>';
var height = $scope.paragraph.config.graph.height;
angular.element('#p' + $scope.paragraph.id + '_table').css('height', height);
var resultRows = $scope.paragraph.result.rows;
var columnNames = _.pluck($scope.paragraph.result.columnNames, 'name');
var container = document.getElementById('p' + $scope.paragraph.id + '_table');

var handsontable = new Handsontable(container, {
data: resultRows,
colHeaders: columnNames,
rowHeaders: false,
stretchH: 'all',
sortIndicator: true,
columnSorting: true,
contextMenu: false,
manualColumnResize: true,
manualRowResize: true,
editor: false,
fillHandle: false,
disableVisualSelection: true,
cells: function (row, col, prop) {
var cellProperties = {};
cellProperties.renderer = function(instance, td, row, col, prop, value, cellProperties) {
Handsontable.NumericCell.renderer.apply(this, arguments);
if (!isNaN(value)) {
cellProperties.type = 'numeric';
cellProperties.format = '0,0';
cellProperties.editor = false;
td.style.textAlign = 'left';
} else if (value.length > '%html'.length && '%html ' === value.substring(0, '%html '.length)) {
td.innerHTML = value.substring('%html'.length);
}
};
return cellProperties;
}
html += ' </tr>';
}
html += ' </tbody>';
html += '</table>';

var tableDomEl = angular.element('#p' + $scope.paragraph.id + '_table');
tableDomEl.html(html);
var oTable = tableDomEl.children(1).DataTable({
paging: false,
info: false,
autoWidth: false,
lengthChange: false,
searching: false,
dom: '<>'
});

if ($scope.paragraph.result.msgTable.length > 10000) {
tableDomEl.css({
'overflow': 'scroll',
'height': $scope.paragraph.config.graph.height
});
} else {

var dataTable = angular.element('#p' + $scope.paragraph.id + '_table .table');
dataTable.floatThead({
scrollContainer: function(dataTable) {
return tableDomEl;
}
});

dataTable.on('remove', function () {
dataTable.floatThead('destroy');
});

tableDomEl.css({
'position': 'relative',
'height': '100%'
});
tableDomEl.perfectScrollbar('destroy')
.perfectScrollbar({minScrollbarLength: 20});

angular.element('.ps-scrollbar-y-rail').css('z-index', '1002');

// set table height
var psHeight = $scope.paragraph.config.graph.height;
tableDomEl.css('height', psHeight);
tableDomEl.perfectScrollbar('update');
}

};

var retryRenderer = function() {
Expand Down
23 changes: 23 additions & 0 deletions zeppelin-web/src/app/notebook/paragraph/paragraph.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,29 @@ table.dataTable.table-condensed .sorting_desc:after {
background: none;
}

/*
Handsontable
*/

.handsontable th {
font-weight: bold;
}

.handsontable th, .handsontable td {
border-right: 0px;
border-left: 0px !important;
padding: 4px;
}

.handsontable tr:first-child th {
text-align: left;
border-top: 0px;
padding: 4px 0px 0px 0px;
border-left: 0px;
border-right: 0px;
border-bottom: 2px solid #CCC;
}

/*
Pivot CSS
*/
Expand Down
18 changes: 6 additions & 12 deletions zeppelin-web/src/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@
<link rel="stylesheet" href="bower_components/highlightjs/styles/github.css" />
<link rel="stylesheet" href="bower_components/ngtoast/dist/ngToast.css" />
<link rel="stylesheet" href="bower_components/bootstrap3-dialog/dist/css/bootstrap-dialog.min.css" />
<link rel="stylesheet" href="bower_components/datatables.net-bs/css/dataTables.bootstrap.css" />
<link rel="stylesheet" href="bower_components/datatables.net-buttons-bs/css/buttons.bootstrap.css" />
<link rel="stylesheet" href="bower_components/pikaday/css/pikaday.css" />
<link rel="stylesheet" href="bower_components/handsontable/dist/handsontable.css" />
<!-- endbower -->
<link rel="stylesheet" href="bower_components/jquery-ui/themes/base/all.css" />
<!-- endbuild -->
Expand Down Expand Up @@ -132,16 +132,10 @@
<script src="bower_components/ngtoast/dist/ngToast.js"></script>
<script src="bower_components/ng-focus-if/focusIf.js"></script>
<script src="bower_components/bootstrap3-dialog/dist/js/bootstrap-dialog.min.js"></script>
<script src="bower_components/floatThead/dist/jquery.floatThead.js"></script>
<script src="bower_components/floatThead/dist/jquery.floatThead.min.js"></script>
<script src="bower_components/datatables.net/js/jquery.dataTables.js"></script>
<script src="bower_components/datatables.net-bs/js/dataTables.bootstrap.js"></script>
<script src="bower_components/datatables.net-buttons/js/dataTables.buttons.js"></script>
<script src="bower_components/datatables.net-buttons/js/buttons.colVis.js"></script>
<script src="bower_components/datatables.net-buttons/js/buttons.flash.js"></script>
<script src="bower_components/datatables.net-buttons/js/buttons.html5.js"></script>
<script src="bower_components/datatables.net-buttons/js/buttons.print.js"></script>
<script src="bower_components/datatables.net-buttons-bs/js/buttons.bootstrap.js"></script>
<script src="bower_components/zeroclipboard/dist/ZeroClipboard.js"></script>
<script src="bower_components/moment/moment.js"></script>
<script src="bower_components/pikaday/pikaday.js"></script>
<script src="bower_components/handsontable/dist/handsontable.js"></script>
Copy link
Member

@Leemoonsoo Leemoonsoo Apr 28, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pickaday.js, moment.js, ZeroClipboard.js, pikaday.css are they required to use handsontable?

Copy link
Member Author

@r-kamath r-kamath Apr 28, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ZeroClipboard and pikaday.js are not required, need to see how to remove them from the build.
As of now moment.js is used only for number formatting (7bb508e)

Copy link
Member Author

@r-kamath r-kamath Apr 29, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wrong. All three deps are required. Use without pikaday #2841

<!-- endbower -->
<!-- endbuild -->
<!-- build:js({.tmp,src}) scripts/scripts.js -->
Expand Down
14 changes: 4 additions & 10 deletions zeppelin-web/test/karma.conf.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,16 +59,10 @@ module.exports = function(config) {
'bower_components/ngtoast/dist/ngToast.js',
'bower_components/ng-focus-if/focusIf.js',
'bower_components/bootstrap3-dialog/dist/js/bootstrap-dialog.min.js',
'bower_components/floatThead/dist/jquery.floatThead.js',
'bower_components/floatThead/dist/jquery.floatThead.min.js',
'bower_components/datatables.net/js/jquery.dataTables.js',
'bower_components/datatables.net-bs/js/dataTables.bootstrap.js',
'bower_components/datatables.net-buttons/js/dataTables.buttons.js',
'bower_components/datatables.net-buttons/js/buttons.colVis.js',
'bower_components/datatables.net-buttons/js/buttons.flash.js',
'bower_components/datatables.net-buttons/js/buttons.html5.js',
'bower_components/datatables.net-buttons/js/buttons.print.js',
'bower_components/datatables.net-buttons-bs/js/buttons.bootstrap.js',
'bower_components/zeroclipboard/dist/ZeroClipboard.js',
'bower_components/moment/moment.js',
'bower_components/pikaday/pikaday.js',
'bower_components/handsontable/dist/handsontable.js',
'bower_components/angular-mocks/angular-mocks.js',
// endbower
'src/app/app.js',
Expand Down