Proposal: Adding Flex rules to the validator

[mobilitydataio]

RULES PROPOSAL

MobilityData has put together a list of validation rules that need to be added or modified now that GTFS-Flex has been adopted into the official specification. (Thanks to @westontrillium for sharing some of Trillium's rules for validating Flex feeds pre-adoption - it was extremely helpful inspiration for this proposal!)

We're seeking feedback on this proposal from the community to see if:

• Rules are missing
• There should be special considerations for particular rules
• Logic should be modified
• There are feeds we can use for testing beyond what's already been documented
• There are suggestions for how to generically refer to stops.stop_id, locations.id, and location_groups.location_group_id when they all need to be checked in 1 rule

Issues will be made to tackle these specific sections at the beginning of May once feedback is gathered and the proposal is finalized.

SCOPING & TIMELINE

Since there are 24 new rules and 3 rule modifications identified within this document, MobilityData has split up the rules into the following categories, in order of priority:

• Pre-existing rules to modify (Phase 1)
• routes.txt rules and rules affecting multiple files
• stop_times.txt rules
• locations.geojson rules
• location_groups.txt rules
• stops.txt conditionally required rules (blocked)
• booking_rules.txt rules

MobilityData commits to completing the pre-existing rule changes before the next validator release, and starting to work through the the rest of the list in the order outlined above. However, given capacity, not all rules will be added in the next release if MobilityData works on this alone.

If you and/or your organization are interested in seeing this work accelerated, you are welcome to contribute! Please let us know here if you would like to support this development effort and we can discuss next steps.

RELATED ISSUES MOBILITYDATA WILL ADDRESS

• Investigating issues with parsing Flex feeds  #1767
• Flex: Add location group schema and update stop time schema #1749
• Flex: Add locations.geojson schema  #1750
• Flex: Update missing_trip_edge rule to create exception  #1755

MESSAGE

GTFS-Flex [Voting ver.] (#433)
* Add 3 "location" files and booking rules file
* Modify stops.stop_id
* Modify routes.continuous_pickup/drop_off
* Modify stop_times.arrival/departure_time
* Modify stop_times.stop_id
* Add stop_times.location_group_id & location_id
* Modify stop_times.stop_sequence
* Add stop_times.start/end_pickup_drop_off_window
* Modify pickup_type & drop_off_type
* Modify stop_times.continuous_pickup/drop_off
* Add pickup/drop_off_booking_rule_id
* Add "On-demand Service Routing Behavior"
* Modify conditions for start/end_pickup_drop_off_window
* Remove unnecessary table name & editorial changes

FILES CHANGED

google/transit@43d8bbc

https://github.com/google/transit/commit/43d8bbc5bcd9feb0a53744727eca03dd0f7d8eb0

---

[derhuerst]

Now that Flex v2 has been adopted, will the GTFS Validator support it?

[emmambd]

@derhuerst Yes! The next release will include Flex rules. There will be some discussion here in the next 2 months to solidify the new validation rules and requirements.

[emmambd]

This issue has been updated to include MobilityData's proposal for Flex validation rules. Please read it above and share feedback in the document.

[LeoFrachet]

@emmambd Hi Emma!

The link provided (https://share.mobilitydata.org/flex-validation-rules) doesn't seem to work. Is there an easy way to fix it, or maybe to provide the full URL until it's fixed? Thanks!

[emmambd]

@LeoFrachet Thanks for flagging this! I've updated it with the long form version. Let me know if you still have issues with it.

[LeoFrachet]

Perfect! Thanks, it works!

[emmambd]

Updated this issue with links to the rules MobilityData plans to focus on currently. You can take a look at the specific Github issues to provide more test data or feedback.

[LeoFrachet]

Thanks! While reviewing it I had a question: Is the stops.txt file still required? If they are no stops (only locations), should I output stops.txt empty or can I ignore it?

🤔 In theory it should become optional like every other empty file. That would break many existing GTFS parser, but anyway those parsers will have to be updated to accept stop_times.txt with no stop_id, so I guess it would make sense to make the file optional.

Should I post that kind of feedback here or directly in the document? Lemme know if I should follow another process.

Thanks!

[emmambd]

Hi @LeoFrachet - thanks for raising this! Adding it to the Github thread here was perfect. This consideration came up for us too in thinking through the Flex rules. Our team's created an issue on the spec side to discuss this with the community: google/transit#452.

Once clarified within the spec, we'll make the appropriate change in the validator. You're welcome to share your thoughts and recommendations there.

[emmambd]

As an update on the next GTFS Schedule Validator release, this has been delayed due to 1) some unexpected needed performance investigations on Flex rules 2) the MobilityData's team's current focus on releasing feed search for the Mobility Database this summer. As a result, the next validator release will happen later this summer rather than at the end of June.

The focus of the next validator release will be to remove false positive errors for Flex feeds. You can see the list of issues that will be covered in the next release here. Anyone interested in accelerating this work is welcome to contribute to the development effort.

[emmambd]

Our first phase of implementing Flex validation rules has been added to the 6.0 release!

The full proposal has been updated with a Status column to indicate which work is complete and which work is still in progress.

Please share any questions, comments or feedback in the discussion link.