Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add: query runner for Cassandra and ScyllaDB #1236

Merged
merged 9 commits into from
Aug 29, 2016
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 137 additions & 0 deletions redash/query_runner/cass.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
import json
import sys
import logging

from redash.query_runner import *
from redash.utils import JSONEncoder

logger = logging.getLogger(__name__)

class Cassandra(BaseQueryRunner):
@classmethod
def configuration_schema(cls):
return {
'type': 'object',
'properties': {
'host': {
'type': 'string',
},
'port': {
'type': 'number',
'default': 9042,
},
'Keyspace': {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep all properties snake_cased (i.e. Keyspace -> keyspace).

'type': 'string',
'title': 'Keyspace name'
},
'username': {
'type': 'string',
'title': 'Username'
},
'password': {
'type': 'string',
'title': 'Password'
}
},
'required': ['Keyspace']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would assume host is required too, no?

}

@classmethod
def type(cls):
return "Cassandra"

@classmethod
def enabled(cls):
try:
from cassandra.cluster import Cluster
except ImportError:
return False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can move this whole block to the top of the file (to avoid duplicate import code), and set a variable (enabled) which you use here. You can see the BigQuery module as an example.


return True

def _get_tables(self, schema):
query = """
select columnfamily_name from system.schema_columnfamilies where keyspace_name = '{}';
""".format(self.configuration['Keyspace'])

results = self.run_query(query)
return results, error

def run_query(self, query):
from cassandra.cluster import Cluster
connection = None
try:
if self.configuration.get('username', '') and self.configuration.get('password', ''):
auth_provider = PlainTextAuthProvider(username='{}'.format(self.configuration.get('username', '')),
password='{}'.format(self.configuration.get('password', '')))
connection = Cluster([self.configuration.get('host', '')], auth_provider=auth_provider)
else:
connection = Cluster([self.configuration.get('host', '')])

session = connection.connect()
logger.info("Cassandra running query: %s", query)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this to debug level.

result = session.execute(query)

column_names = []
columns = []
for i in result.column_names:
column_names.append(i)

for column_name in column_names:

columns.append({
'name': column_name,
'friendly_name': column_name,
'type': 'string'
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can replace this code with the fetch_columns method from the BaseQueryRunner class.

Something like columns = self.fetch_columns(map(lambda c: (c, 'string'), column_names)).

rows = [dict(zip(column_names, row)) for row in result]

data = {'columns': columns, 'rows': rows}
json_data = json.dumps(data, cls=JSONEncoder)

error = None

except cassandra.cluster.Error, e:
error = e.args[1]
except KeyboardInterrupt:
error = "Query cancelled by user."
except Exception as e:
raise sys.exc_info()[1], None, sys.exc_info()[2]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which lines?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

99-100, you catch an exception and then raise it again. Might as well not catch it and let it bubble up on its own.


return json_data, error

class ScyllaDB(Cassandra):
@classmethod
def type(cls):
return "scylla"

@classmethod
def configuration_schema(cls):
return {
'type': 'object',
'properties': {
'host': {
'type': 'string',
},
'port': {
'type': 'number',
'default': 9042,
},
'Keyspace': {
'type': 'string',
'title': 'Keyspace name'
},
'username': {
'type': 'string',
'title': 'Username'
},
'password': {
'type': 'string',
'title': 'Password'
}
},
'required': ['Keyspace']
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it any different than Cassandra's configuration_schema?


register(Cassandra)
register(ScyllaDB)
1 change: 1 addition & 0 deletions requirements_all_ds.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ botocore==1.4.4
sasl>=0.1.3
thrift>=0.8.0
thrift_sasl>=0.1.0
cassandra-driver==3.1.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it pure Python or has some C/system package dependencies?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gcc python-dev

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍