Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add parser #292

Merged
merged 27 commits into from
Mar 2, 2017
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
78e2353
first pass at wiring in parser
Feb 20, 2017
39e2c1c
add fqn to model representation, rewiring some of the compiler
Feb 20, 2017
6658d87
Merge branch 'development' of github.com:analyst-collective/dbt into …
Feb 22, 2017
beedfd5
almost there
Feb 23, 2017
2473b12
down to 10 integration test failures
Feb 23, 2017
50d9896
schema and data tests running in parser
Feb 26, 2017
a125043
archive passing, hooks not so much
Feb 26, 2017
5a64113
integration tests passing!
Feb 26, 2017
f34892f
remove runners (they are unused now)
Feb 26, 2017
1a2ada7
remove get_compiled_models -- unused
Feb 26, 2017
c7dd776
ripping things out, part 1: compiled_model.py
Feb 26, 2017
8dc998b
ripping stuff out, part 2: archival and other unused model types
Feb 26, 2017
b3b17ee
pep8 compliance
Feb 27, 2017
7db4b1d
remove print() call from runner.py
Feb 27, 2017
7a0039d
remove print() calls from selector.py
Feb 27, 2017
6a3202e
remove schema_tester, dbt.archival import
Feb 27, 2017
2a2aec2
fix unit tests, compile cmd
Feb 27, 2017
305c80c
functional test improvements
Feb 27, 2017
77c480a
fix skipping, functional testing w/ revzilla
Feb 27, 2017
ac3e5ee
hooks work... finishing up?
Feb 27, 2017
102e8df
add compat module to deal with str/unicode/basestring diffs in 2 vs 3
Feb 27, 2017
f24c0b1
switch compilation import
Feb 27, 2017
99b9ea1
fun with string compatibility
Feb 28, 2017
d96913d
write_file is necessary
Feb 28, 2017
96b3d49
merged master
Mar 2, 2017
caf6105
re-add analyses
Mar 2, 2017
d3142cb
update changelog
Mar 2, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions dbt/adapters/postgres.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@ def rename(cls, profile, from_name, to_name, model_name=None):

@classmethod
def execute_model(cls, profile, model):
parts = re.split(r'-- (DBT_OPERATION .*)', model.compiled_contents)
parts = re.split(r'-- (DBT_OPERATION .*)', model.get('wrapped_sql'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll be able to replace this with separate nodes grouped into a transaction, right?

connection = cls.get_connection(profile)

if flags.STRICT_MODE:
Expand All @@ -317,7 +317,7 @@ def call_expand_target_column_types(kwargs):
func_map[function](kwargs)
else:
handle, cursor = cls.add_query_to_transaction(
part, connection, model.name)
part, connection, model.get('name'))

handle.commit()

Expand Down Expand Up @@ -504,6 +504,16 @@ def commit(cls, profile):
handle = connection.get('handle')
handle.commit()

@classmethod
def rollback(cls, profile):
connection = cls.get_connection(profile)

if flags.STRICT_MODE:
validate_connection(connection)

handle = connection.get('handle')
handle.rollback()

@classmethod
def get_status(cls, cursor):
return cursor.statusmessage
Expand Down
4 changes: 2 additions & 2 deletions dbt/adapters/snowflake.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ def rename(cls, profile, from_name, to_name, model_name=None):

@classmethod
def execute_model(cls, profile, model):
parts = re.split(r'-- (DBT_OPERATION .*)', model.compiled_contents)
parts = re.split(r'-- (DBT_OPERATION .*)', model.get('wrapped_sql'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll break these out into separate nodes in a future PR, yeah?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

absolutely

connection = cls.get_connection(profile)

if flags.STRICT_MODE:
Expand Down Expand Up @@ -216,7 +216,7 @@ def call_expand_target_column_types(kwargs):
func_map[function](kwargs)
else:
handle, cursor = cls.add_query_to_transaction(
part, connection, model.name)
part, connection, model.get('name'))

handle.commit()

Expand Down
70 changes: 0 additions & 70 deletions dbt/archival.py

This file was deleted.

45 changes: 45 additions & 0 deletions dbt/compat.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import codecs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is great


WHICH_PYTHON = None

try:
basestring
WHICH_PYTHON = 2
except NameError:
WHICH_PYTHON = 3

if WHICH_PYTHON == 2:
basestring = basestring
else:
basestring = str


def to_unicode(s):
if WHICH_PYTHON == 2:
return unicode(s)
else:
return str(s)


def to_string(s):
if WHICH_PYTHON == 2:
if isinstance(s, unicode):
return s
elif isinstance(s, basestring):
return to_unicode(s)
else:
return to_unicode(str(s))
else:
if isinstance(s, basestring):
return s
else:
return str(s)


def write_file(path, s):
if WHICH_PYTHON == 2:
with codecs.open(path, 'w', encoding='utf-8') as f:
return f.write(to_string(s))
else:
with open(path, 'w') as f:
return f.write(to_string(s))
Loading