Make the Dashboard page compatible with the new data model #1493

roseeichelmann · 2024-07-25T22:03:02Z

Associated issues

The issue suggests using the crashes list view but the way this currently works is we are supposed to reflect the same values in this VZE that are in the socrata dataset, so i thought it made the most sense to use the socrata_export_crashes_view

Testing

URL to test:
Local

Steps to test:

Check out this branch, start up your data model db and apply hasura migrations/metadata and run the cris import script if necessary
Open up the hasura console and navigate to the socrata_export_crashes_view, sort by death count, years of life lost, and suspected serious injuries to see what we would expect the widgets on the dashboard to say. Keep in mind this is only for crashes that happened in 2024.
Start your VZE, go to the dashboard, and confirm you are seeing the numbers you would expect to see in the widgets.
Now go create some temp crash records with fatalities/serious injuries to further test the dashboard widget. Remember that the socrata dataset is only counting crashes that happened at least 14 days ago. So in your crash creation the date needs to be todays date - 14 days or farther back.
Also, you will need to update the private drive flag to NO in the crash details page of the temp record in order for it to factor into the socrata ytd totals. Sidenote - lets not let this requirement slip through the cracks so the VZ team isnt wondering why their temp crashes arent making it into the socrata totals
Update your computer to a different time zone. You should still see the same numbers in your widgets bc everything should be using UTC time and not expecting the user to be in CT.

Ship list

Check migrations for any conflicts with latest migrations in master branch
Confirm Hasura role permissions for necessary access
Code reviewed
Product manager approved

…a_model

roseeichelmann · 2024-07-29T16:50:38Z

atd-vze/src/views/Tools/CreateCrashRecord.js

@@ -347,13 +347,13 @@ const CreateCrashRecord = ({ client }) => {
                    fade={false}
                  >
                    {/*eslint-disable-next-line*/}
-                    <i className="fa fa-check-circle" /> Successfsully
-                    created crash{" "}
+                    <i className="fa fa-check-circle" /> Successfully created


lil typo 😄

roseeichelmann · 2024-07-29T16:54:39Z

atd-vze/src/views/VZDashboard/VZDashboard.js

-  const yearEnd = format(subDays(new Date(), 14), "yyyy-MM-dd");
+  const yearEnd = subDays(new Date(), 14)
+    .toISOString()
+    .substring(0, 19); // to match the ISO string format in the socrata view crash_timestamp column


we are creating a date in UTC time so that this is cross compatible with users in any time zone. cutting off the last milliseconds in the ISO string bc the crash_timestamp column in the socrata view doesnt have milliseconds

i think you can do away yearEnd completely, since we have the 14-day cutoff baked into the socrata view 🤔

hah you're so right wow what a silly oversight on my part 😅

roseeichelmann · 2024-07-29T17:29:22Z

atd-vze/src/views/Tools/CreateCrashRecordTable.js

-    crashes(where: { is_temp_record: { _eq: true } }) {
+    crashes(
+      where: { is_temp_record: { _eq: true } }
+      order_by: { record_locator: asc }


was annoying me that the temp records table wasnt being sorted so it was hard to find ur last record creation

do you want to sort this by desc instead, so the most recent records are at the top?

also, i could have sworn you had configured the query to refetch after form submit so that the temp records table gets refreshed. that's not happening anymore. maybe i broke that when i patched your branch?? 🙈

johnclary

i ran through all the tests—it works great. love that you went with the socrata export view for this!

johnclary · 2024-07-29T18:09:31Z

atd-vze/src/views/VZDashboard/VZDashboard.js

-  const yearEnd = format(subDays(new Date(), 14), "yyyy-MM-dd");
+  const yearEnd = subDays(new Date(), 14)
+    .toISOString()
+    .substring(0, 19); // to match the ISO string format in the socrata view crash_timestamp column


i think you can do away yearEnd completely, since we have the 14-day cutoff baked into the socrata view 🤔

johnclary · 2024-07-29T18:12:10Z

atd-vze/src/views/VZDashboard/VZDashboard.js


 import { GET_CRASHES_YTD } from "../../queries/dashboard";

 import bi_logo from "../../assets/img/brand/power_bi_icon_white_on_transparent.png";

 function VZDashboard() {
  const year = new Date().getFullYear();
-  const yearStart = `${year}-01-01`;
+  const yearStart = `${year}-01-01T00:00:00`;


little TZ bug here—to get the year value you want to use new Date().getUTCFullYear() since you're constructing a UTC timestamp

johnclary · 2024-07-29T18:15:27Z

atd-vze/src/views/Tools/CreateCrashRecordTable.js

-    crashes(where: { is_temp_record: { _eq: true } }) {
+    crashes(
+      where: { is_temp_record: { _eq: true } }
+      order_by: { record_locator: asc }


do you want to sort this by desc instead, so the most recent records are at the top?

also, i could have sworn you had configured the query to refetch after form submit so that the temp records table gets refreshed. that's not happening anymore. maybe i broke that when i patched your branch?? 🙈

chiaberry · 2024-07-29T18:30:55Z

fwiw I didnt see the temp records table refresh when i tested last week

roseeichelmann · 2024-07-29T22:09:42Z

@johnclary ready for re-review, i dont think there was ever refetch functionality but i just added that in!

roseeichelmann · 2024-07-29T22:17:55Z

atd-vze/src/views/Tools/CreateCrashRecordTable.js

  const [deleteRecord] = useMutation(DELETE_TEMP_RECORD);

-  useEffect(() => {


dont need this useEffect

roseeichelmann · 2024-07-29T22:18:26Z

atd-vze/src/views/Tools/CreateCrashRecordTable.js

-      });
-  };
+  // /**
+  //  * Deletes all the temporary records in the database


just commenting this out till we handle temp deletes in a following PR

johnclary

niceeee—thanks for adding in the refetch!

🚢 🚢 🚢 🚢 🚢 🚢 🚢

roseeichelmann added 4 commits July 25, 2024 17:55

add years of life lost as field to socrata export view

f21d573

add permissions to socrata view

4473149

query the socrata view instead for ytd aggregates

67bb56f

rework to use new query

f71e5c9

roseeichelmann added the WIP Work in progress label Jul 25, 2024

roseeichelmann added 5 commits July 25, 2024 18:29

update this comment

54919e3

linting

fab2dde

Merge branch '18149-data-model-socrata-etls' into 18328_dashboard_dat…

1fe972f

…a_model

fix this typo

9233a7c

use ISO format and get the time stuff all sorted out

23ab6cd

roseeichelmann commented Jul 29, 2024

View reviewed changes

roseeichelmann removed the WIP Work in progress label Jul 29, 2024

roseeichelmann requested review from johnclary, frankhereford, chiaberry, tillyw and mddilley July 29, 2024 16:59

sort by asc

a55f54e

roseeichelmann commented Jul 29, 2024

View reviewed changes

johnclary requested changes Jul 29, 2024

View reviewed changes

roseeichelmann added 5 commits July 29, 2024 17:06

remove year end variable from socrata query

ecc66eb

refetch temp record table on submit

bd92fe0

get data from parent component

636684e

remove year end var

73ce076

lets move this query into its own file in the query folder

0a13a19

roseeichelmann requested a review from johnclary July 29, 2024 22:10

roseeichelmann commented Jul 29, 2024

View reviewed changes

johnclary approved these changes Jul 29, 2024

View reviewed changes

roseeichelmann merged commit b97cb93 into 18149-data-model-socrata-etls Jul 30, 2024

roseeichelmann deleted the 18328_dashboard_data_model branch July 30, 2024 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make the Dashboard page compatible with the new data model #1493

Make the Dashboard page compatible with the new data model #1493

roseeichelmann commented Jul 25, 2024 •

edited

Loading

roseeichelmann Jul 29, 2024

roseeichelmann Jul 29, 2024

johnclary Jul 29, 2024

roseeichelmann Jul 29, 2024

roseeichelmann Jul 29, 2024

johnclary Jul 29, 2024

johnclary left a comment

johnclary Jul 29, 2024

johnclary Jul 29, 2024

johnclary Jul 29, 2024

chiaberry commented Jul 29, 2024

roseeichelmann commented Jul 29, 2024

roseeichelmann Jul 29, 2024

roseeichelmann Jul 29, 2024

johnclary left a comment

		const [deleteRecord] = useMutation(DELETE_TEMP_RECORD);

		useEffect(() => {

Make the Dashboard page compatible with the new data model #1493

Make the Dashboard page compatible with the new data model #1493

Conversation

roseeichelmann commented Jul 25, 2024 • edited Loading

Associated issues

Testing

Ship list

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johnclary left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chiaberry commented Jul 29, 2024

roseeichelmann commented Jul 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johnclary left a comment

Choose a reason for hiding this comment

roseeichelmann commented Jul 25, 2024 •

edited

Loading