feat: Make Jinja template applied in timestamp columns

[xingyc15]

SUMMARY

Fix the timestamp expression in sqla model.py, adding jinja template processor to it.

TESTING INSTRUCTIONS

diff --git a/docker/pythonpath_dev/superset_config.py b/docker/pythonpath_dev/superset_config.py
index 6c58bec79..83ebedca1 100644
--- a/docker/pythonpath_dev/superset_config.py
+++ b/docker/pythonpath_dev/superset_config.py
@@ -99,6 +99,15 @@ WEBDRIVER_BASEURL_USER_FRIENDLY = WEBDRIVER_BASEURL

 SQLLAB_CTAS_NO_LIMIT = True

+# Local env test
+FEATURE_FLAGS = {
+    "ENABLE_TEMPLATE_PROCESSING": True
+}
+
+JINJA_CONTEXT_ADDONS = {
+    'jinja_tmpl': 'date'
+}
+

Apply the above changes, run docker build -t apache/superset:latest-dev . and then use docker-compose to start the superset app. Go to localhost:8088 in browser and login to superset.

Go to dataset, find cleaned_sales_data and go to calculated columns, add a new item like this:

Then go to the chart Proportion of Revenue by Product Line, choose the jinja_date as time column and run. The chart should works fine without error. View the query and the order_{{ jinja_tmpl }} is already replaced by "order_date".

ADDITIONAL INFORMATION

• Has associated issue: JINJA Template does not work with calculated time column #16984
• Required feature flags: "ENABLE_TEMPLATE_PROCESSING": True
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[villebro]

Thanks for the contribution, really nice feature! I left a small proposal to make the feature support all query properties available.

[villebro]

To make sure the template processor has access to all query parameters which are defined early in the get_sqla_query method (see

superset/superset/connectors/sqla/models.py

Lines 970 to 982 in 4316fe6

	template_kwargs = {
	"columns": columns,
	"from_dttm": from_dttm.isoformat() if from_dttm else None,
	"groupby": groupby,
	"metrics": metrics,
	"row_limit": row_limit,
	"row_offset": row_offset,
	"time_column": granularity,
	"time_grain": time_grain,
	"to_dttm": to_dttm.isoformat() if to_dttm else None,
	"table_columns": [col.column_name for col in self.columns],
	"filter": filter,
	}

), it would be nice to pass the template processor (declared here:

superset/superset/connectors/sqla/models.py

Line 995 in 4316fe6

template_processor = self.get_template_processor(**template_kwargs)

) as an optional parameter to get_timesstamp_expression that can then be used if available.

[codecov]

Codecov Report

Merging #17237 (f0aec1a) into master (dd71035) will increase coverage by 0.00%.
The diff coverage is 75.00%.

@@           Coverage Diff           @@
##           master   #17237   +/-   ##
=======================================
  Coverage   76.95%   76.96%           
=======================================
  Files        1038     1038           
  Lines       55612    55614    +2     
  Branches     7590     7590           
=======================================
+ Hits        42798    42803    +5     
+ Misses      12564    12561    -3     
  Partials      250      250           

Flag	Coverage Δ
hive	81.52% <75.00%> (+0.01%)	⬆️
mysql	81.95% <75.00%> (+0.01%)	⬆️
postgres	81.96% <75.00%> (+0.01%)	⬆️
presto	81.82% <75.00%> (+0.01%)	⬆️
python	82.45% <75.00%> (+0.01%)	⬆️
sqlite	81.63% <75.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/connectors/sqla/models.py	87.39% <75.00%> (-0.09%)	⬇️
superset/annotation_layers/annotations/schemas.py	100.00% <0.00%> (+13.33%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd71035...f0aec1a. Read the comment docs.

[villebro]

LGTM, thanks for the change that was spot-on with what I was hoping for!