Modes Architecture in MDS

[schnuerle]

Discussion space for discussing how MDS can support multiple modes on the horizon and into the future. This discussion kicked off in our WG Meeting July 1 2021. Please see the meeting minutes and action items there.

Resources

• Read the Extending MDS to New Modes document for review.

• Attend the Aug 19 Working Group meeting for a deeper discussion.

• See the feature branch where we will be doing our work.

Topics

• Architecture of overall MDS API schema for new modes and variants.
  • Potential future modes: carshare, passenger services/taxis, delivery robots, drones
  • How does it affect vehicles, trips, events?
  • What is the impact on the state machine? Are there new version or modifications only?
• Related to Issue #614
• Want to frame the right questions, discuss possible paths to take, and lay groundwork for mode inclusion.
• Will not be discussing specific modes, except in relation to the overall topic.

Questions

1. Should each mode have its own state machine and properties, in its own mode 'box', or does it all get crammed into one complicated data structure with lots of options?
2. Is there a 'core' definitions of MDS, with 'extensions' for each mode? How formal is the core and how formal are the extensions?
3. How do you compare and analyze across different modes if they are separated? How does this complicate future MaaS concepts?
4. What is impact on MDS of each mode being in its own box? Policy, events, trips, agency, etc.
5. How can we introduce the concept of modes into MDS initially, as a non-breaking change? Create a PR that shows how this is possible for 1.2.0.

[stevebrining]

It seems like the biggest divide between the current state machine and adjustments for car-share, taxi, drone, delivery bot is what I'll call driver-mode vs passenger-mode with the latter including human and non-human (packages/food) cargo. With the exception of carsharing, almost all of these new modes are in this second category, possibly having similar trip segments (pickup, delivery, roaming) and package/person count possibilities with different pickup and drop-off points. Drive-mode only has one trip segment > when a user is operating it, as it's not causing congestion on the streets otherwise like a delivery vehicle would be when it's moving to the next pickup, returning to base, etc.

[quicklywilliam]

One of the key question in my mind is whether there are really such things as "canonical modes" – ie modes that have a stable, universal definition independent from individual implementations (ie programs). For example what we now call "micromobility" started as a dock-based bikeshare mode and later added "dockless micromobility" (perhaps a new mode?) and now has many hybrid/blended versions (are these all new modes?). Is "micromobility" a vehicle or a mode? Another example is carshare. Can we say "carshare" is a mode when there are so many different versions – traditional, point-to-point, etc?

The reason I bring up all these questions is that it seems like it is problematic to say with any finality that such-and-such is a mode that has such-and-such states, transitions, rule-types, etc. Maybe eventually we start to see standardization, but that will only be enabled by an indeterminate period of trial-and-error. Even then, I believe we should expect (and hope for!) new "modes" to arise constantly. This process of innovation is driven both by providers and agencies.

One possibility here is that rather than trying to define "canonical modes", maybe we could attempt to come up with a set of building blocks from which any mode can be assembled, and a way of describing how those building blocks fit together for a given mode. This would ensure that the standard is flexible and future-proof while still using a shared language that enables things like multimodal control and analysis. To make things easier for cities and implementors, the OMF could still host definition files for commonly used modes – but these would be extensible and flexible by design.

The above questions and idea are intended to spark discussion here – I'm not yet convinced we need something abstracted or that I have come up with the simplest thing that would address the questions I raised.

[jfh01]

This is really interesting. At an epistemological level, I agree that we need a lot of fluidity in the concept of "mode" and we run the risk of always playing catchup if we're too prescriptive.

That said, I would argue that modes tend to get grouped into fairly well-defined types based on the regulatory system under which they fall. For example, micromobility tends to get regulated along a certain set of dimensions, which are different from those of for-hire vehicles. Even with a lot of variation within each modal category, the policies (and attendant data) are broadly similar from one jurisdiction to the next for a given "mode." Of course, it should be said that in many emerging cases (ex: delivery robots, urban freight) the regulatory space is still quite poorly defined.

Part of the value of MDS is to encourage that similarity within a given regulatory model. It makes it easier to build and sell data analysis tools, allows jurisdictions to share policy lessons with each other, and limits the complexity for operators who serve multiple jurisdictions. MDS supports many degrees of local flexibility about how to regulate a given mode, but it creates a clearly-defined, shared toolbox which cities can use to implement their preferred local approach.

Ultimately though, I agree with your point. As much as possible, we should be creating building blocks (see my comment below), and not treating each mode as a distinct, impermeable thing which is free to evolve divergently from every other mode. But somehow we also need to create legible "modal categories" for the widely-used regulatory models so that there is a "shared toolbox" for each.

To torture the metaphor, a plumber, a carpenter, and an electrician will likely all have a hammer, screwdriver, and pliers in their toolbox, but will also have tools unique to their profession and may use the common ones in slightly different ways.

[quicklywilliam]

Thanks. Item #1 in your comment below sounds a lot like what I had in mind. Essentially, we'd create a way to define a "mode", describing the subset of state/events that apply to it. Then the OMF could define (inside the spec) a set of ready-to-go, prebaked modes. Something like this:

{
   "id": "p2p-carshare"
   "Description": "Point to point carshare",
   "applicable_states": [
            {
              "state": "available",
              "applicable_transitions": ["trip", "unavailable",…]
            },
            {
              "state": "reserved",
              "applicable_transitions": ["available", "trip",…]
            },…
   ]
}

[jfh01]

That makes sense. Whether we do it in text or in code, having clearly defined state machines for delineated modes makes sense. A related question is if/when/how we want to allow someone to craft their own state machine for a new mode and start serving up conforming data (versus having to get any new modal state machine approved by OMF and formally added to MDS). It's a tension between fragmentation potential and adaptability, I think.

[quicklywilliam]

Right. Maybe one thing that would be helpful is to flesh out the lifecycle for a new mode – from conception, to a single city pilot, to wider adoption and iteration, to something fairly stable and widely understood. That lifecycle might be something we could circulate for feedback, and once we have some consensus we could figure out how MDS can best facilitate the entire lifecycle in a way that has flexibility/adaptability at the early stages and clarity/consistency at the later stages.

[sanjivnanda]

@jfh01 - as I was reading the discussion about trips, trip types and related trips in the document, and then read this analogy in your comment above - "To torture the metaphor, a plumber, a carpenter, and an electrician will likely all have a hammer, screwdriver, and pliers in their toolbox, but will also have tools unique to their profession and may use the common ones in slightly different ways."

Since we are at an early stage, I offer my thoughts for consideration. I think it's possible to define trips, trip types and related trips in such a way that the semantics are consistent and not abused across modes. Just because a field called related trips exists under trips, different modes should not be permitted to use this field in inconsistent ways.

For this it is useful to construct the most general trip type across delivery robots (the hierarchy, including deadheading, staging, and productive travel) and rideshare (which also has the hierarchy, and the productive travel has a further hierarchy of groups with overlapping trips). I wouldn't want to use the related trips field in unrelated ways...

Another thought to consider is whether it might be helpful to have distinct terms instead of using trip for both vehicles moving from A to B, and for cargo or people moving on a vehicle from A to B. In some ways, the former is the cost and the latter is the value.

My 2c.

[jfh01]

Documenting a few ideas from the WG meeting and some follow up thoughts:

1. We should attempt to create a universal list of states/event types. Individual modes may require subsets of that list, but the goal would be to reuse as many of the states/event types as makes sense to improve cross-modal data interoperability and to reduce the learning curve when implementing new modes. We may need to re-write some of the existing states/event type definitions to be more inclusive of other modes.
2. We should consider adding linking between trips in the data model. Potentially on a parent/child basis, or a sibling basis, or both. Example use cases: connecting the various phases of a delivery robot journey (travel to pick up, travel to drop off, return to base), a shared TNC that serves multiple passenger groups in a linked series of trips, a car share reservation containing multiple individual trips within it. This may require creating a "trip type" that would help explain the hierarchy (e.g. "a car share 'reservation' trip may contain multiple 'travel segment' trips). The idea here is to create a universal model for capturing complex, modally-specific trip structures. But maybe it's better/clearer to create modally-specific data elements?
   3.  Would be great to have a guide to adding new modes that expresses some design principles for things like state machines, linking trips, extending vehicle definitions, etc. The goal would be to put down a point-of-view that others can follow as the landscape evolves and people find new uses for MDS.

[schnuerle]

I thinking of how to define modes and keep them flexible, and future proof.

With all the vehicle types possible, this is something MobilityData is working on with their new vehicle definitions in regards to accessibility and adaptive vehicles. They are not trying to define every possibility accessibility feature, but rather describe the key properties of the vehicle that could be used in this determination. Properties like number of wheels, presence of a seat, and size of the baseboard. The approach is more about fitting the right type of device to a rider, rather than simply indicating whether it was adaptive or not. The same could apply to modes, where we define a set of properties for a use case, to determine policy/data needs and how MDS may need to change to accommodate that use case, and then align any number of modes to a set of 'properties'.

See the MobilityData GBFS VehicleTypes Needs Assessment for more details about this concept of modulatory.

So we could define the sorts of things that require some part of the state machine (events, states) to be changed consistently. For example, who is moving the vehicle, if there are passengers, and if there are multiple trips in a reservation all affect parts of the state machine independently.

Driver	Passengers	Multiple Trips	Verticality	Devices	Mode
Self	No	No	No	Bikes, Scooters	Shared Micromobility
Self	Yes	No	No	Mopeds	Shared Micromobility
Self	Yes	Yes	No	Cars	Carshare
Person	Yes	Yes	No	Cars	Taxis/Ridehail
AI/Remote	No	Yes	No	Cute Carts	Delivery Robots
AI	Yes	Yes	No	Cars	Autonomous Vehicles
AI/Remote	No	Yes	Yes	Drones	Air Delivery
Person	Yes	No	Yes	Hover Unit	Space Taxi

So the focus is on core things like the driver, passengers, trips, etc, which each affect the state machine in a pre-defined way. Note how in the example if the driver is self and there are not multiple trips needing to be tracked, then it's still under the "Shared Micromobility" mode. Every combination of these core properties can exist, with some being defined as modes now for convenience, and others being defined as they come into existence.

Then we only have to define each core property's impact on the state machine. For example, multiple trips affect the way on_trip and segments are defined. Having multiple overlapping passengers may affect how trips are organized and tracked for fees. Verticality might not affect the state machine, but is useful for categorizing modes. You just mix and match these defined changes for the mode you are supporting in your program.

I wanted to get this idea out there on paper for feedback. It's different than just making a new state machine for every mode, instead focusing on just the properties of modes that could apply to more than one mode. I welcome thoughts on this approach, even though it's not fully fleshed out yet.

[pomgod]

I think you have to add a couple of parameters to cover all modes (included PT):

• route-type: fixed (PT), flexible (micro-transit) or free (any mode where you can choose your route)
• entry-type: timetable (PT), on demand (taxi, AV,...); on the spot (shared mobility)
• exit-type (where can a trip be ended with this mode): open-end (anywhere), closed-end (within pre-defined zones), back-to-one.

[sanjivnanda]

I like the table, and the comment captured earlier by @jfh01 about constructing a "universal" list of states and permitted state transitions. This "analysis" has overhead now, but will pay off in terms of reuse of work to develop (and even more importantly) debug and eliminate bugs and errors once. This is a good time to leverage the expertise of the current batch of contributors to create this nice framework that each mode can adapt to its need.

Of course, we wont get it all correct, but having this core MDS state machine framework and a high bar to add/modify/correct this core requiring review by a core architecture group seems like a good way to create an MDS that will last.

Equally important is to publish a standard for each mode that an implementer is not required to review the core. You need to understand the core only if you want to contribute to MDS, not if you want to implement.

[schnuerle]

A model for extending MDS must be flexible enough to accommodate modes that are in some ways substantially different from micromobility. Currently planned extensions include car share (point-to-point and station-based), delivery robots (autonomous and remotely driven), and passenger transportation (taxis, TNCs, microtransit).

We have had 2 WG meetings which were focused on new modes in MDS (Jan 28, July 1), and others that touched on this topic and specific modes. As an action item after the last meeting, we created this discussion area, and the OMF Staff was tasked with creating an initial draft of how modes could be structured in MDS, for review by the working group.

The staff has worked on this over the last few weeks and created an Extending MDS to New Modes document for you to review.

Another action item was to have a meeting focused on new modes and to review this doc, and per the WGSC, this is scheduled for next week on Aug 19.

Right now the document is read only, though feel free to leave your initial comments here in this discussion area. After our Aug 19 meeting, we will open the document for inline comments and refinements.

Some time later the WGSC will present the draft to the OMF Technology Council for synthesized questions and themes, then finalize the document in WG meetings and online, before getting final review from the Tech Council in the coming months. This work will lay the foundation for adding new modes to the next MDS 2.0 major release.

Our goal is to make it easier for MDS community members to propose extensions by providing an architectural framework that can flex as new modes appear.

[e-lo]

Linking Trips: Explicit trips for vehicle vs payload/pax

The extension document discusses trips as being relatable across a related_trip_id field. In practice, it is likely useful to be more explicit about this relationship: particularly as it relates to a trip taken by a vehicle (and driver if applicable) and the payload/passengers. This could look like: vehicle_trip_id and [payload/pax]trip_id being explicitly stated, as shown in the marked up diagrams below.

I think this would also be simpler from an analytical join perspective.

[S-eb]

I agree a different object to represent the overall 'journey' is probably easier to handle

[e-lo]

Terminology Alignment

Linked trips

I suggest aligning the terminology of these multi-part trips with either the travel behavior field or other field which already has similar terminology, which would use the terms:

• tour (journey): journey starting and ending at a home-base which is comprised of one or more trips to accomplish one or more activities
• trip: journey from one activity to another, comprised of one or more legs connected by transfers
• leg: functional component of a trip which has a uniform vehicle (or lack thereof for walking)
• activity (stop): where something useful happens which fulfills (in part) the purpose of the tour

GOFS/GTFS

How do these terms align (not) with GTFS and GOFS?
Which fields have the potential for overlap/duplication with GOFS/GTFS?

At the very least, I think we need to align on definitions of common fields and data structures so that in the (probable?) event of some overlap in the functionality they can be easily relatable.

[S-eb]

One aspect to handle will be the overlapping segments in the case of ridehailing taking more that one Pax at the same time

[S-eb]

Some suggestion in preparation of the discussion this afternoon

To help clarify what the 'mode' concept is, it might be helpful to break it down in a couple of underlying concept.
The following proposal is taking this approach to help simplify the implementation, while maximizing flexibility.

• business type: this represents the business being operated by a given company, such as micromobility, car sharing etc...
• service type: represents the nature of the service: passenger service, freight, food ...
• journey type: represents the type of journey that can be operated, namely:
  • point to point journey
  • multi segments journey
  • overlapping multi segment journey

The matrix below reflects some of the situations we know already

Journey type	Pax	Freight	Food
point to point	Micromobility
multi segments journey	Taxi
overlapping segments journey	Ridehailing	Bots delivery

Journey type definition is not likely to evolve a lot as there's not a lot of other options about journey management.
Business type is probably where the greatest flexibility is required.

The state diagram can be attached to the journey type.
The required vehicle fields could be mandated at business type level.
While all the fields could be defined at the vehicle level.

Finally all these type could be attached @ provider level to limit the complexity.

We feel that splitting these concepts help simplifying the problem while reaching the objective to be more comprehensive without adding complexity to existing implementations.

[S-eb]

Another point: in the vehicle attributes, we should think about emission criteria. Some cities may enforce different policies depending on how much the vehicle is emitting.

[jfh01]

Do you have a sense of how these emissions criteria are defined? e.g. is it simply tied to propulsion type? Or, are there specific lists of vehicle make/model that are considered low/no-emission?

[schnuerle]

It would be good to try to figure this out in a way that makes sense, has meaning and value, and is implementable by providers. Many people have stated they would like to see data like this. We have an open issue about it with #643 and have discussed at WG meetings and have outreach action items for it, but no clear path forward.

[avatarneil]

I think @e-lo and @S-eb both have really interesting suggestions around this, and I definitely like them more than what's currently proposed!

I think that we're at risk of abusing the notion of what a 'trip' is in order to try to make a one-size-fits-all solution. Adding a potentially infinite tree-based hierarchy tingles my design spidey senses, and I think we really ought to avoid what's currently proposed in the google doc.

The fundamental question to ask, is what do we gain by splitting 'legs' of a trip up into separate 'trips' (particularly in the delivery bot scenario)? I think the answer is probably information such as distance, and in the context of the MDS Provider feed, having an idea of what's going on with the trip in near real-time, as opposed to waiting till the 'parent trip' has completed. I think that this could be achieved by keeping the notion of a Trip as a more top-level construct, and potentially enabling more real-time detail through exposing 'legs' prior to a trip being completed (where a leg could potentially be defined as what has changed between two state transitions, and would also include information such as distance of the leg).

See this other issue for some similar thoughts I've had on this.

What we (E&A/Lacuna) have done in our fork of MDS for Taxi & TNC (rideshare), is to add the notion of multiple concurrent trips without an explicit linkage. There is a notion of "Vehicle State", but also a notion of "Trip State". When you receive a trip-related event, both vehicle_state and trip_state are provided, and there is a precedence order for what vehicle_state should be when you have multiple trips ongoing with potentially different trip_states. It is not hard for us to join together these trips using overlapping segment analysis when need be. See https://github.com/lacuna-tech/mobility-data-specification/blob/add-taxi-mode/general-information.md#taxi-vehicle-states--events

[jfh01]

In the Lacuna implementation, do you get information about the overall driver session? Including movement data about the vehicle when it's not on_trip? (e.g. when it's available, driving around, but not yet "reserved")? This is definitely a use case from an emissions, congestion, and fair wage perspective. Overloading trip to represent driver session (in addition to using it to represent passenger journeys) would be one approach.

[avatarneil]

@jfh01 Yup, we absolutely do! This is do-able mostly because we use the Agency interface, where Telemetry data is not strictly tied to Trips. As part of the SLAs, if the vehicle is 'available', Telemetry is required to be sent with the same strictness as if the vehicle is on a trip. If we were using Provider, we would not be able to collect this information. This is one place where I could definitely imagine we need to improve the Provider interface; I wonder if we could have a Provider GET /telemetry endpoint where a device_id and time range could be supplied to satisfy this gap.

[jfh01]

That makes sense. This also relates to some of the ongoing provider/agency mismatches identified in #550.

[haydens30]

Question: Do you also want to create a mode for walking/running = "on foot" too?
E.g. cities may want to enforce just walkers in a pedestrianized area, or a specific zone where micro-mobility riders have to dismount

[avatarneil]

MDS for Pedestrians sounds like a headline to a scathing article to me 😅, I think we should probably steer clear of that. What we've done to-date for Micromobility (and I could imagine this extending to other modes), is to write Policy that expresses a given area is not allowed to have vehicles matching a particular status. If we were to extend this notion to other modes, I think that the expression would just be that for all defined modes a given area is considered off-limits; essentially defining the desired behavior exclusively instead of inclusively.

[haydens30]

OK, that's fine. But for a city/ authority is the policy "exclude all modes" the same as "just include pedestrians"?

[schnuerle]

I think for now this would be defined within a specific mode which are created based on a regulator purpose. Eg in 'Delivery' a delivery could be made by car, truck, bike or on foot. 'Micromobility' maybe there is an on foot way to move or rebalance a scooter or bike.

[haydens30]

As I explained on the the call last week, The Open Transport Initiative has a different purpose to OMF, as we are concerned with Transport & MaaS system customer account interoperability (rather than for city / authority operations). However some of our previous work on our Open Standard may be useful here... especially when looking to define the different modes across an entire transport & mobility ecosystem.

In the below diagram we have attempted to map different key modes and show some of their different characteristics & classifications across land, water & air.

Let me know if you think anything is missing or incorrect and what else we need to consider. Thanks

[jfh01]

We have synthesized the feedback on this thread and in our last WG discussion of new modes, along with some recommendations. We will be discussing these at our 10/7 WG meeting:

https://docs.google.com/document/d/1q6cHyBEsvv8PHJiE0Gv5-XprAogKGMweo2b9dNiaMqs/edit#

[schnuerle]

We will be doing the code changes needed for this in the feature-modes branch. A link to the PR to dev will be added here once it's created.

[tristan-charreton]

Hey OMF Team ! Hey @alexdemisch and @schnuerle . I see that this discussion has been quiet for a while but I think it might be the best place to ask those questions :

• Regarding the new modes, why have a mode specific to 'Delivery Robots' and not a mode 'Delivery' in which we precise if it is a robot in the vehicle type. The workflow, the states and the events should be the same (or almost the same) ? And we might want to include all types of deliveries in this MDS 2.
• It seems like you had a different definition of a trip (see Elizabeth's screenshot above from 14th Aug 2021) for a delivery robot (where a trip is a segment) and a rideshare (where a trip is an order from a customer). Although most delivery bots' operators only handle one order at a time :
  • it might still be interesting to have the same definition for every mode so that it helps the standardization (some operators do both deliveries and passenger services)
  • some operators might or will handle many deliveries at once (same idea as a pooled ridehail trip) so it's important to anticipate on that.

I came up with these questions as I am working with Kiwibot on the Delivery mode PR and I think these need to be addressed before we move on.