[WIP] Enhance metadata associated with dependencies in app-1.yml

[steveteahan]

The current implementation of dependencies in app-1.yml doesn't provide
much context about how a dependency is used. For example, is it a build
or runtime dependency? If it is a runtime dependency, knowing whether it
is upstream or downstream provides more information for troubleshooting
and also allows for creating directional graphs of the flow of traffic
across multiple services. A description field allows teams to provide
even more application-specific context which can also be helpful for
SREs that may not be very familiar with a given service.

dependencyFailureImpact was previously associated with a
dependency-1.yml, but multiple apps could be relying on the same
dependency, all with different impacts if the dependency is down. At some
point we may consider deprecating that field from the dependency itself?

A related change to dependency-1.yml also allows linking an app-1.yml
file to the dependency. This should make it easier to track down
information about a dependency when it is an internal app.

For more information, see APPSRE-4261.

This is a first pass at the change. An example usage in app-1.yml would look like:

dependencies:
  - dependency:
      $ref: /dependencies/example/example.yml
    dependencyType: runtime-upstream
    dependencyFailureImpact: partial-outage
    description: 'This service is used to gather data about X, Y, and Z'
  - dependency:
      $ref: /dependencies/api-gateway/example.yml
    dependencyType: runtime-downstream
    dependencyFailureImpact: complete-outage
    description: 'This is the API Gateway for example-service'

From this example, we could determine that:

/dependencies/api-gateway/example.yml --> (this) app.yml --> /dependencies/example/example.yml

Likewise, we can answer questions like: which services will have a complete outage if /dependencies/api-gateway/example.yml is down?

The open questions that I'm trying to answer and TODOs at this point are:

• Are we okay with making an API breaking change here given that dependencies don't appear to be widely used from my understanding?
• Given that we track upstream and downstream, do we expect both sides to declare the dependency? For instance, in the example above, /dependencies/api-gateway/example.yml is a downstream dependency of our example app. Do we also expect the app associated with /dependencies/api-gateway/example.yml?

[bhushanthakur93]

@steveteahan Would there a case where we could possibly end up with a circular dependency between multiple services i.e service A -> service B -> service C -> service A?

[steveteahan]

@steveteahan Would there a case where we could possibly end up with a circular dependency between multiple services i.e service A -> service B -> service C -> service A?

I'd expect that sort of an issue to be caught during an architecture review, but with this data it's something that we could reasonably map out and/or check for.

[bhushanthakur93]

@steveteahan Would there a case where we could possibly end up with a circular dependency between multiple services i.e service A -> service B -> service C -> service A?

I'd expect that sort of an issue to be caught during an architecture review, but with this data it's something that we could reasonably map out and/or check for.

Yes, that makes sense. Having a pre-emptive check with this data could also enforce tenants to perhaps re-architect their services or have some fallbacks in place. Do you think it makes sense to have that kind of a check when tenants open their PR against app-interface with the contract?

[steveteahan]

@steveteahan Would there a case where we could possibly end up with a circular dependency between multiple services i.e service A -> service B -> service C -> service A?

I'd expect that sort of an issue to be caught during an architecture review, but with this data it's something that we could reasonably map out and/or check for.

Yes, that makes sense. Having a pre-emptive check with this data could also enforce tenants to perhaps re-architect their services or have some fallbacks in place. Do you think it makes sense to have that kind of a check when tenants open their PR against app-interface with the contract?

It's a feature that we can consider. For the first pass, the primary goal is to enable the collection of dependency data, and to be able to expose that data via visual-qontract. Once we have the data, we could certainly explore other use cases.

[bhushanthakur93]

@steveteahan Would there a case where we could possibly end up with a circular dependency between multiple services i.e service A -> service B -> service C -> service A?

I'd expect that sort of an issue to be caught during an architecture review, but with this data it's something that we could reasonably map out and/or check for.

Yes, that makes sense. Having a pre-emptive check with this data could also enforce tenants to perhaps re-architect their services or have some fallbacks in place. Do you think it makes sense to have that kind of a check when tenants open their PR against app-interface with the contract?

It's a feature that we can consider. For the first pass, the primary goal is to enable the collection of dependency data, and to be able to expose that data via visual-qontract. Once we have the data, we could certainly explore other use cases.

Yes, 100% agree. That use case is out of scope for this story.

[steveteahan]

I'm rethinking this approach because I'd really like to see if we can avoid declaring these relationships in multiple app-1.yml files. In some ways, it's a feature because we could enforce that both sides need to remove the dependency before doing something like deprecating an app, etc. On the other hand, it can make things really inefficient. When I'm searching for all upstream/downstream dependencies for an app, that's simple if they're defined directly in the app-1.yml file. To find references to said app in other app files, you'd need to iterate through all apps, which isn't great.

I'll think about this a little bit more to see if I can think of a better way of doing this.

[bhushanthakur93]

@steveteahan We could perhaps look into utilizing adjacency matrix to represent these dependencies in memory. For e.g if service A is dependent on service B, we can have a matrix M such that M[0][0] = 0, M[0][1] = 1, M[1][0] = 0, M[1][1] = 0 ; where index 0 indicates service A , index 1 indicates service B and the value 1 indicates if there is a dependency from one index to another, 0 indicates no dependency.

We can then do a diagonal mirror of that matrix to get reverse edge. i.e M[0][0] = 0, M[0][1] = 0, M[1][0] = 1, M[1][1] = 0 which indicates reverse relationship based on how we define edges.

This will enable us to get upstream dependencies on fly based on the initial one way downstream dependency data within your proposal.

WDYT?

[steveteahan]

@steveteahan We could perhaps look into utilizing adjacency matrix to represent these dependencies in memory. For e.g if service A is dependent on service B, we can have a matrix M such that M[0][0] = 0, M[0][1] = 1, M[1][0] = 0, M[1][1] = 0 ; where index 0 indicates service A , index 1 indicates service B and the value 1 indicates if there is a dependency from one index to another, 0 indicates no dependency.

We can then do a diagonal mirror of that matrix to get reverse edge. i.e M[0][0] = 0, M[0][1] = 0, M[1][0] = 1, M[1][1] = 0 which indicates reverse relationship based on how we define edges.

This will enable us to get upstream dependencies on fly based on the initial one way downstream dependency data within your proposal.

WDYT?

I should have added more context to my last message, but it was essentially meant as "don't bother reviewing this yet because I'm rethinking the solution." An adjacency matrix can certainly be used to represent a graph. My concerns are really more concrete in terms of how we express these relationships in app-interface, and then how we expose the data with GraphQL, which is ultimately called by visual-qontract.

I haven't worked with visual-qontract at all yet, so I'm going to investigate what we're doing there and in qontract-server before I comment further. How to model this is much clearer to me with REST and RDBMS rather than GraphQL and yaml files.

If this particular ticket really interests you, then I'd be happy to pair with you on it once I have a chance to take a closer look at visual-qontract and qontract-server.

[bhushanthakur93]

@steveteahan We could perhaps look into utilizing adjacency matrix to represent these dependencies in memory. For e.g if service A is dependent on service B, we can have a matrix M such that M[0][0] = 0, M[0][1] = 1, M[1][0] = 0, M[1][1] = 0 ; where index 0 indicates service A , index 1 indicates service B and the value 1 indicates if there is a dependency from one index to another, 0 indicates no dependency.
We can then do a diagonal mirror of that matrix to get reverse edge. i.e M[0][0] = 0, M[0][1] = 0, M[1][0] = 1, M[1][1] = 0 which indicates reverse relationship based on how we define edges.
This will enable us to get upstream dependencies on fly based on the initial one way downstream dependency data within your proposal.
WDYT?

I should have added more context to my last message, but it was essentially meant as "don't bother reviewing this yet because I'm rethinking the solution." An adjacency matrix can certainly be used to represent a graph. My concerns are really more concrete in terms of how we express these relationships in app-interface, and then how we expose the data with GraphQL, which is ultimately called by visual-qontract.

I haven't worked with visual-qontract at all yet, so I'm going to investigate what we're doing there and in qontract-server before I comment further. How to model this is much clearer to me with REST and RDBMS rather than GraphQL and yaml files.

If this particular ticket really interests you, then I'd be happy to pair with you on it once I have a chance to take a closer look at visual-qontract and qontract-server.

Chatted with @steveteahan about this offline. Closing off this thread.