Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert SQL to JSON and back again, the space is missing in Window function #1739

Closed
RiccoYuan opened this issue Jun 7, 2023 · 1 comment

Comments

@RiccoYuan
Copy link

This is a very complex SQL. It is amazing that sqlglot can parse and restore it to this level. Awesome!

Here is a small problem I encountered, when converting from JSON back to SQL, there is a space missing between the alias of the Window and ORDER BY

image

import sqlglot

sql = """WITH t_attr_eas
AS (
   SELECT dp.attr_id
      ,dpo.parent
      ,dp.m_name
      ,dp.st
      ,fps.cs_date
      ,fps.actual_mgo_in_date
      ,fps.sd_date
      ,flb.eas_id
      ,flb.eas_type
      ,flb.eas_status
      ,flb.eas_start_date
      ,flb.eas_end_date
      ,flb.mgo_in_date
      ,flb.mgo_out_date
      ,flb.ct_start_date
      ,flb.ct_end_date
      ,flb.vd_date
      ,dlb.rtn
   FROM DLM.fact_attr_submission fps
   INNER JOIN DLM.dim_attr dp ON fps.attr_id = dp.attr_id
   LEFT JOIN DLM.dim_attr_os dpo ON dpo.attr_id = fps.attr_id
   LEFT JOIN DLM.fact_eas_base flb ON flb.attr_id = dp.attr_id
   LEFT JOIN DLM.dim_eas_base dlb ON flb.eas_id = dlb.eas_id
      AND flb.eas_status NOT IN ('Draft', 'Executed')
   WHERE fps.ts_complete IN ('CSA', 'SDA')
   )
   -- step 2, link the attr_eas to the date
   ,t_date_attr_eas
AS (
   SELECT dd.date_key
      ,dd.full_date_iso
      ,pl.*
      -- similar flags that will be used for aggregation
      ,CASE WHEN DATE (dd.full_date_iso) BETWEEN least(COALESCE(pl.mgo_in_date, pl.eas_start_date), pl.eas_start_date) AND COALESCE(pl.vd_date, pl.mgo_out_date, pl.eas_end_date) THEN 1 ELSE 0 END opd_attr
   FROM DLM.dim_date dd
   -- every attr for one date
   INNER JOIN t_attr_eas pl ON TRUE
   WHERE dd.date_key <= CAST(date_format(CURRENT_DATE, '%Y%m%d') AS INT)
      AND dd.date_key > 20100101
      AND DATE (dd.full_date_iso) BETWEEN pl.cs_date AND COALESCE(pl.sd_date, DATE ('2099-01-01'))
   )
   --SELECT * FROM t_date_attr_eas ORDER BY date_key DESC, attr_id
   ,t_daily_attr
AS (
   SELECT *
      ,count(attr_id) OVER (
         PARTITION BY date_key
         ,parent
         ,m_name
         ,st
         ) AS total_attr
      ,count(CASE WHEN opd_attr = 1 THEN attr_id END) OVER (
         PARTITION BY date_key
         ,parent
         ,m_name
         ,st
         ) AS total_eas
      ,ROW_NUMBER() OVER (
         PARTITION BY date_key
         ,parent
         ,m_name
         ,st
         ) AS row_num
   FROM t_date_attr_eas tdpl
   )
   --SELECT * FROM t_daily_attr
   ,t_tl_period
AS (
   SELECT *
   FROM (
      VALUES (30)
         ,(60)
         ,(90)
         ,(120)
         ,(365)
      ) AS t(tl_days)
   )
   -- SELECT * FROM t_daily_attr where parent ='CoJV1' and m_name= 'LAM' and stt = 'CSA' and row_num = 1 ,
   ,t_tl_attr
AS (
   SELECT date_key
      ,ttp.tl_days
      ,parent
      ,m_name
      ,st
      ,total_attr
      ,total_eas
      ,sum(total_attr) OVER (w ORDER BY date_key ROWS BETWEEN tl_days PRECEDING AND CURRENT ROW) AS total_tl_attr_day
      ,sum(total_eas) OVER (w ORDER BY date_key ROWS BETWEEN tl_days PRECEDING AND CURRENT ROW) AS total_tl_eas_day
   FROM t_daily_attr
   INNER JOIN t_tl_period ttp ON TRUE
   WHERE row_num = 1
   WINDOW w AS (
         PARTITION BY tl_days
         ,parent
         ,m_name
         ,st 
         )
   )
SELECT date_key
   ,tl_days
   ,parent
   ,m_name
   ,st
   ,total_attr
   ,total_eas
   ,total_tl_attr_day
   ,total_tl_eas_day
FROM t_tl_attr
ORDER BY date_key DESC"""

ast = sqlglot.parse_one(sql, read="presto")
json = sqlglot.Expression.dump(ast)
## print(json)

ast = sqlglot.Expression().load(json)
print(ast.sql(dialect="presto"))
@tobymao tobymao closed this as completed in be0de6e Jun 7, 2023
@tobymao
Copy link
Owner

tobymao commented Jun 7, 2023

thanks

adrianisk pushed a commit to adrianisk/sqlglot that referenced this issue Jun 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants