get_tables_by_prefix, ideal for union_tables jobs

README.md

@@ -108,6 +108,34 @@ model_name:
 
 ---
 ### SQL helpers
+#### get_column_values ([source](macros/sql/get_column_values.sql))
+This macro returns the unique values for a column in a given table.
+
+Usage:
+```
+-- Returns a list of the top 50 states in the `users` table
+{% set states = fromjson(get_column_values(table=ref('users'), column='state', max_records=50)) %}
+
+{% for state in states %}
+    ...
+{% endfor %}
+
+...
+```
+
+#### get_tables_by_prefix ([source](macros/sql/get_tables_by_prefix.sql))
+This macro returns a list of tables that match a given prefix, with an optional
+exclusion pattern. It's particularly handy paired with `union_tables`.
+
+Usage:
+```
+-- Returns a list of tables that match schema.prefix%
+{{ set tables = get_tables_by_prefix('schema', 'prefix')}}
+
+-- Returns a list of tables as above, excluding any with underscores
+{{ set tables = get_tables_by_prefix('schema', 'prefix', '%_%')}}
+```
+
 #### group_by ([source](macros/sql/groupby.sql))
 This macro build a group by statement for fields 1...N
 
@@ -131,21 +159,6 @@ Usage:
 ```
 {{ union_tables(tables=[ref('table_1'), ref('table_2')], column_override={"some_field": "varchar(100)"}) }}
 ```
-
-#### get_column_values ([source](macros/sql/get_column_values.sql))
-This macro returns the unique values for a column in a given table.
-
-Usage:
-```
--- Returns a list of the top 50 states in the `users` table
-{% set states = fromjson(get_column_values(table=ref('users'), column='state', max_records=50)) %}
-
-{% for state in states %}
-    ...
-{% endfor %}
-
-...
-```
 ---
 ### Web
 #### get_url_parameter ([source](macros/web/get_url_parameter.sql))

macros/sql/get_tables_by_prefix.sql

@@ -0,0 +1,23 @@
+{% macro get_tables_by_prefix(schema, prefix, exclude='') %}
+
+    {%- call statement('tables', fetch_result=True) %}
+
+        select
+            distinct table_schema || '.' || table_name as ref
+        from information_schema.tables
+        where table_schema = '{{ schema }}'
+        and table_name ilike '{{ prefix }}%'
+        and table_name not ilike '{{ exclude }}'
+
+    {%- endcall -%}
+
+    {%- set table_list = load_result('tables') -%}
+
+    {%- if table_list and table_list['data'] -%}
+        {%- set tables = table_list['data'] | map(attribute=0) | list %}
+        {{ return(tables) }}
+    {%- else -%}
+        {{ return([]) }}
+    {%- endif -%}
+
+{% endmacro %}