Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the Dashboard page compatible with the new data model #1493

Merged
Merged
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -49,6 +49,7 @@ select
cimv.micromobility_sus_serious_injry_count as micromobility_serious_injury_count,
cimv.other_fatality_count as other_death_count,
cimv.other_sus_serious_injry_count as other_serious_injury_count,
cimv.years_of_life_lost,
to_char(
crashes.crash_timestamp, 'YYYY-MM-DD"T"HH24:MI:SS'
) as crash_timestamp,
Original file line number Diff line number Diff line change
@@ -1,3 +1,145 @@
table:
name: socrata_export_crashes_view
schema: public
select_permissions:
- role: editor
permission:
columns:
- bicycle_death_count
- bicycle_serious_injury_count
- death_cnt
- law_enf_fatality_count
- micromobility_death_count
- micromobility_serious_injury_count
- motorcycle_death_count
- motorcycle_serious_injury_count
- motor_vehicle_death_count
- motor_vehicle_serious_injury_count
- nonincap_injry_cnt
- non_injry_cnt
- other_death_count
- other_serious_injury_count
- pedestrian_death_count
- pedestrian_serious_injury_count
- poss_injry_cnt
- sus_serious_injry_cnt
- tot_injry_cnt
- unkn_injry_cnt
- years_of_life_lost
- crash_fatal_fl
- onsys_fl
- private_dr_fl
- road_constr_zone_fl
- crash_id
- crash_sev_id
- crash_speed_limit
- id
- latitude
- longitude
- address_primary
- address_secondary
- case_id
- crash_timestamp
- crash_timestamp_ct
- point
- rpt_block_num
- rpt_street_name
- rpt_street_sfx
- units_involved
filter: {}
allow_aggregations: true
comment: ""
- role: readonly
permission:
columns:
- bicycle_death_count
- bicycle_serious_injury_count
- death_cnt
- law_enf_fatality_count
- micromobility_death_count
- micromobility_serious_injury_count
- motorcycle_death_count
- motorcycle_serious_injury_count
- motor_vehicle_death_count
- motor_vehicle_serious_injury_count
- nonincap_injry_cnt
- non_injry_cnt
- other_death_count
- other_serious_injury_count
- pedestrian_death_count
- pedestrian_serious_injury_count
- poss_injry_cnt
- sus_serious_injry_cnt
- tot_injry_cnt
- unkn_injry_cnt
- years_of_life_lost
- crash_fatal_fl
- onsys_fl
- private_dr_fl
- road_constr_zone_fl
- crash_id
- crash_sev_id
- crash_speed_limit
- id
- latitude
- longitude
- address_primary
- address_secondary
- case_id
- crash_timestamp
- crash_timestamp_ct
- point
- rpt_block_num
- rpt_street_name
- rpt_street_sfx
- units_involved
filter: {}
allow_aggregations: true
comment: ""
- role: vz-admin
permission:
columns:
- bicycle_death_count
- bicycle_serious_injury_count
- death_cnt
- law_enf_fatality_count
- micromobility_death_count
- micromobility_serious_injury_count
- motorcycle_death_count
- motorcycle_serious_injury_count
- motor_vehicle_death_count
- motor_vehicle_serious_injury_count
- nonincap_injry_cnt
- non_injry_cnt
- other_death_count
- other_serious_injury_count
- pedestrian_death_count
- pedestrian_serious_injury_count
- poss_injry_cnt
- sus_serious_injry_cnt
- tot_injry_cnt
- unkn_injry_cnt
- years_of_life_lost
- crash_fatal_fl
- onsys_fl
- private_dr_fl
- road_constr_zone_fl
- crash_id
- crash_sev_id
- crash_speed_limit
- id
- latitude
- longitude
- address_primary
- address_secondary
- case_id
- crash_timestamp
- crash_timestamp_ct
- point
- rpt_block_num
- rpt_street_name
- rpt_street_sfx
- units_involved
filter: {}
allow_aggregations: true
comment: ""
Original file line number Diff line number Diff line change
@@ -49,6 +49,7 @@ select
cimv.micromobility_sus_serious_injry_count as micromobility_serious_injury_count,
cimv.other_fatality_count as other_death_count,
cimv.other_sus_serious_injry_count as other_serious_injury_count,
cimv.years_of_life_lost,
to_char(
crashes.crash_timestamp, 'YYYY-MM-DD"T"HH24:MI:SS'
) as crash_timestamp,
80 changes: 5 additions & 75 deletions atd-vze/src/queries/dashboard.js
Original file line number Diff line number Diff line change
@@ -1,92 +1,22 @@
import { gql } from "apollo-boost";

// These queries are derived from the ETL that populates our Socrata data sets
// This query is derived from the view that populates our Socrata dataset
// so that the VZV and VZE widget totals match up
// See https://github.com/cityofaustin/atd-vz-data/blob/master/atd-etl/app/process/socrata_queries.py
export const GET_CRASHES_YTD = gql`
query GetCrashesYTD($yearStart: date, $yearEnd: date) {
fatalities: atd_txdot_crashes_aggregate(
where: {
crash_date: { _lt: $yearEnd, _gte: $yearStart }
private_dr_fl: { _eq: "N" }
_and: [
{ crash_date: { _is_null: false } }
{ crash_time: { _is_null: false } }
]
_or: [
{ in_austin_full_purpose: { _eq: true } }
{ _and: [{ city_id: { _eq: 22 } }, { position: { _is_null: true } }] }
]
}
query GetCrashesYTD($yearStart: String, $yearEnd: String) {
socrata_export_crashes_view_aggregate(
where: { crash_timestamp: { _lt: $yearEnd, _gte: $yearStart } }
) {
aggregate {
sum {
atd_fatality_count
death_cnt
}
}
}
seriousInjuriesAndTotal: atd_txdot_crashes_aggregate(
where: {
crash_date: { _lt: $yearEnd, _gte: $yearStart }
private_dr_fl: { _eq: "N" }
_and: [
{ crash_date: { _is_null: false } }
{ crash_time: { _is_null: false } }
]
_or: [
{ in_austin_full_purpose: { _eq: true } }
{ _and: [{ city_id: { _eq: 22 } }, { position: { _is_null: true } }] }
]
}
) {
aggregate {
sum {
sus_serious_injry_cnt
}
}
}
primaryPersonFatalities: atd_txdot_primaryperson_aggregate(
where: {
crash: { private_dr_fl: { _eq: "N" } }
_and: {
prsn_injry_sev_id: { _eq: 4 }
crash: { crash_date: { _lt: $yearEnd, _gte: $yearStart } }
_or: [
{ crash: { in_austin_full_purpose: { _eq: true } } }
{
_and: [
{ crash: { city_id: { _eq: 22 } } }
{ crash: { position: { _is_null: true } } }
]
}
]
}
}
) {
aggregate {
sum {
years_of_life_lost
}
}
}
personFatalities: atd_txdot_person_aggregate(
where: {
crash: { private_dr_fl: { _eq: "N" } }
_and: {
prsn_injry_sev_id: { _eq: 4 }
crash: { crash_date: { _lt: $yearEnd, _gte: $yearStart } }
_or: [
{ crash: { in_austin_full_purpose: { _eq: true } } }
{
_and: [
{ crash: { city_id: { _eq: 22 } } }
{ crash: { position: { _is_null: true } } }
]
}
]
}
}
) {
aggregate {
sum {
years_of_life_lost
6 changes: 3 additions & 3 deletions atd-vze/src/views/Tools/CreateCrashRecord.js
Original file line number Diff line number Diff line change
@@ -347,13 +347,13 @@ const CreateCrashRecord = ({ client }) => {
fade={false}
>
{/*eslint-disable-next-line*/}
<i className="fa fa-check-circle" /> Successfsully
created crash{" "}
<i className="fa fa-check-circle" /> Successfully created
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lil typo 😄

crash{" "}
<Link
to={`/crashes/T${successfulNewRecordId}`}
className="alert-link"
>
#{successfulNewRecordId}
#T{successfulNewRecordId}
</Link>
</Alert>
<Alert
5 changes: 4 additions & 1 deletion atd-vze/src/views/Tools/CreateCrashRecordTable.js
Original file line number Diff line number Diff line change
@@ -23,7 +23,10 @@ import {

const GET_TEMP_RECORDS = gql`
query getTempRecords {
crashes(where: { is_temp_record: { _eq: true } }) {
crashes(
where: { is_temp_record: { _eq: true } }
order_by: { record_locator: asc }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was annoying me that the temp records table wasnt being sorted so it was hard to find ur last record creation

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you want to sort this by desc instead, so the most recent records are at the top?

also, i could have sworn you had configured the query to refetch after form submit so that the temp records table gets refreshed. that's not happening anymore. maybe i broke that when i patched your branch?? 🙈

) {
record_locator
case_id
crash_timestamp
30 changes: 10 additions & 20 deletions atd-vze/src/views/VZDashboard/VZDashboard.js
Original file line number Diff line number Diff line change
@@ -5,41 +5,31 @@ import { useQuery } from "@apollo/react-hooks";
import Widget02 from "../Widgets/Widget02";
import VZLinksWidget from "../Widgets/VZLinksWidget";
import VZNoticeWidget from "../Widgets/VZNoticeWidget";
import { format, subDays } from "date-fns";
import { subDays } from "date-fns";

import { GET_CRASHES_YTD } from "../../queries/dashboard";

import bi_logo from "../../assets/img/brand/power_bi_icon_white_on_transparent.png";

function VZDashboard() {
const year = new Date().getFullYear();
const yearStart = `${year}-01-01`;
const yearStart = `${year}-01-01T00:00:00`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

little TZ bug here—to get the year value you want to use new Date().getUTCFullYear() since you're constructing a UTC timestamp

// We use the same end date as VZV so VZE widget totals match VZV widgets
const yearEnd = format(subDays(new Date(), 14), "yyyy-MM-dd");
const yearEnd = subDays(new Date(), 14)
.toISOString()
.substring(0, 19); // to match the ISO string format in the socrata view crash_timestamp column
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are creating a date in UTC time so that this is cross compatible with users in any time zone. cutting off the last milliseconds in the ISO string bc the crash_timestamp column in the socrata view doesnt have milliseconds

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think you can do away yearEnd completely, since we have the 14-day cutoff baked into the socrata view 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hah you're so right wow what a silly oversight on my part 😅

const { loading, error, data } = useQuery(GET_CRASHES_YTD, {
variables: { yearStart, yearEnd },
});

if (loading) return "Loading...";
if (error) return `Error! ${error.message}`;

const {
years_of_life_lost: yearsOfLifeLostPrimaryPerson,
} = data.primaryPersonFatalities.aggregate.sum;

const {
years_of_life_lost: yearsOfLifeLostPerson,
} = data.personFatalities.aggregate.sum;

const {
sus_serious_injry_cnt: seriousInjuryCount,
} = data.seriousInjuriesAndTotal.aggregate.sum;
const { atd_fatality_count: deathCount } = data.fatalities.aggregate.sum;
const aggregateData =
data.socrata_export_crashes_view_aggregate.aggregate.sum;

const yearsOfLifeLostYTD =
yearsOfLifeLostPrimaryPerson + yearsOfLifeLostPerson;
const fatalitiesYTD = deathCount;
const seriousInjuriesYTD = seriousInjuryCount;
const yearsOfLifeLostYTD = aggregateData.years_of_life_lost;
const fatalitiesYTD = aggregateData.death_cnt;
const seriousInjuriesYTD = aggregateData.sus_serious_injry_cnt;

// Widget02 expects a string value, DB returns number or null
const commaSeparator = number =>