Enable Hateoas in project with servicenames (Kubernetes, Consul, Eureka, ...)

[pcornelissen]

Following up to #107
To be able to create links for a service based on it's servicename, which is used especially in Microservice environments that use consul, eureka or even kubernetes, you wouldn't want to create links pointing to the instance of the service, instead you need to create links pointing to the servicename, so any instance of the service can process further requests.

The current workaround is to create a filter that fakes X-Forwarding-host headers to trick another filter to create the right settings. This seems to be very hacky and hard to understand and maintain in the long run, as it's not very obvious.

It would be much better to have a dedicated setting that overrides the port/protocol and hostname. The most flexible solution would be a class that can be provided for that, but I'm not sure if that is overkill.

I'm also not completely sure if that mechanism shouldn't be in spring itself.

[gregturn]

Focusing on Eureka as a concrete instance of the "thing" in the room we're talking about,

I get your point. Forwarding is something you'd configure your load balancer to provide, given it's the part of your infrastructure that A) generates the need, B) has the right information, and C) should thus be responsible for curing the issue.

However, Eureka isn't a fixed device (physical or virtual), hence it doesn't offer out-of-the-box forwarding settings like that (right @spencergibb?)

At the same time, Spring HATEOAS itself won't know about Eureka either. Sorting out whose responsibility this rests with sounds key before implementing a solution. And it appears that either Eureka itself should put forth the information, or whomever is setting it up.

I'd be thrilled of Eureka/Consul/whomever ALSO served RFC-compliant headers. Because that carries us all the way back to A) they are generating the need, B) they have the right information, and C) they should thus be responsible for providing the solution.

[pcornelissen]

No, the requests are not routed through some systems.
Consul+Eureka are just service registries, where services register their instances in a dynamic way.
Clients usually download a list of active services and instances and do client side loadbalancing based on the servicenames.
So I have 3 instances of MyService running, then I just make a GET call to http://MyService/whatever, then the client library (eureka/consul/...) fetches the list of instances and performs the request for example round-robin.
And it wouldn't make sense to request that callers provide artificial forwarded headers when they make calls to a service, as this is a new requirement that the "world" has to obey, because the service is not capable of creating correct links. Also you might have calls that are triggered without a request by JMS, timers, ...

[odrotbohm]

I don't think this is stuff that should live inside Spring HATEOAS. We're not in the business of reasoning about logical names for services and any kind of system's topology at all. Spring HATEOAS is in the business of easing the use of hypermedia in application code and we're currently using the information we have accessible and adapt based on standardized means (HTTp Headers). If there's anything in place performing a logical translation of a service name to a real service instance, that thing either has to make sure it can adapt the responses or tweak the requests in a way that the server produces responses in a way that's appropriate for the client context.

The point is: you're not making a request to http://MyService/whatever simply because that thing doesn't even exist. You're making a request to whatever is backed by that logical name and the instance translating that name into a real service instance needs to make sure it sets up the request created in a way that the server either provides a response matching that special name resolution in the first place or simply post-process the response accordingly.

So what you argue being "very hacky" is essentially one of the building blocks of REST called Layered System.

I'd love to hear @spencergibb's input on this and whether the Spring Cloud team thinks they can do something about this.

[spencergibb]

There is some work already done in spring cloud commons IIRC. We can certainly entertain more requests. Not sure about X-Forwarded as it's not a proxy.

[pcornelissen]

@odrotbohm I think you misunderstood me. It's not spring hateoas that should do any kind of mapping, it's just that the protocoll, hostname and port for creating URLs in Linkbuilders etc. should be configurable, so you as a service can create links that also work when the service instance that created them is down. For this you should create a link based on servicenames instead of real hostnames.
As I said, it may be a topic for lower layers in the spring framework, but that's your decision, I just state something that we as microservice app developers need to avoid to create the same hacks over and over again.

Remembering your arguments prior spring hateoas 1.0 that the getId override of the former wrapper classes shadowed, you argued that the "id" of a restful resource is it's URL and not a numeric ID or something like that. And this is only possible when such an ID is stable and not targets a single instance of a service, because that may be gone in the next minute and your URI to http://instance-0815.my.net/hdjsdghjsgjd is no longer valid. This ain't the case when you create links on the service name, which should be resolvable as long as instances of it exist.

[spencergibb]

Existing integration in spring cloud here https://github.com/spring-cloud/spring-cloud-commons/tree/master/spring-cloud-commons/src/main/java/org/springframework/cloud/client/hypermedia

[odrotbohm]

How does that problem get solved with a property configurable in Spring HATEOAS? The logical name a service is available is unavailable to the target service. It might even be exposed under multiple logical names. How would the possibility to configure one name to be used for the link creation help if that name changes? Would you change the property and restart the service? What about multiple logical service names for a service, how's the target service supposed to decide which logical name to use for a request?

[pcornelissen]

Well, in all cases I have seen until now is that you have a single pretty stable servicename. We had a service that registered multiple serviceIDs in consul, but even that would require to have a way to modify the hostname part of the link to select the right one. But I consider that to be an outlier, which may be solvable by providing an easy way to modify the hostname for the links.
The smaller the services are, th eless likely are multiple service-IDs from my experience.

I have not seen that the same service is called differently in other services without doing something strange. (Actually we have aliases for the service aliases in our project but that is die to historic tec-debt)
If you think that the links you create need to be client specific, you could instead of having a fixed property, introduce a "LinkBasePathDiscoverer" or something like that and in more complicated cases you could write your own in a project, when this is necessary.

@spencergibb I had a quick glance and could not find something hostname related in these classes in cloud commons

[spencergibb]

@pcornelissen just to follow up, you are actually looking for the following?

it's just that the protocoll, hostname and port for creating URLs in Linkbuilders etc. should be configurable, so you as a service can create links that also work when the service instance that created them is down.

[pcornelissen]

Right. You can just create all links by hand, but then you can't use the nice Spring Linkbuilders, so there needs to be a way to influence that on a lower level