Enable external Authentication (authn) and Authorization (authz) via Extensible Auth Provider.

[petemiron]

Requirements

1. An administrator must be able to configure an HTTP-based external auth provider.
   1.1. The external auth provider must support TLS, including specifying certificate authority.
   1.2. The external auth provider should accept and check credentials (username and secret) for the gnatsd server.
   1.3. The external auth provider must have a configurable timeout for DNS, TCP connect, and response. These timeouts may be tracked in a single timeout or separated.
   1.4. Metrics for requests, successful, failed and at least average response time of queries to external auth provider must be available through monitoring endpoints.
   1.5. If no external auth provider is configured, the must be no additional impact on CONNECT performance.
2. For an external auth, the gnatsd server must pass connect user credential information (username and password) to the external endpoint
   2.1 the external auth provider must check authn and return a 200 with authz data (similar to example in Create Endpoint API option for client authorization #428):{ user: 'optional', permissions: { publish: ['foo.*'], subscribe: ['foo.*', 'bar.*'] } }
   2.2. The credentials must be checked during client CONNECT.
   2.3. The external auth provider may return a Time-to-Live (TTL) for authz data.
   2.4. If a TTL is returned, the server should respect the TTL and re-request authn for the user on any new message sent to or received from that user after TTL expiration.
   2.5. The external auth provider must provide a means for failover (eg. DNS round-robin, or multiple addresses in the configuration).

Plugin Interface Mockup

For discussion, here is a mockup of a plugin interface that passes a context around. This pushes locking responsibility into the plugin itself. It is not complete by any stretch of the imagination.

// Simple Mock up of a plugin.  In practice, uid will be some sort of
// principal struct, logger may be passed to the auth plugin, etc.

// AuthPlugin interface to for users to implement.
type AuthPlugin interface {
	Startup()
	Shutdown()

	// GetContext is invoked every time a new client connection is established
	// The plugin can choose to return a singleton, a context from a pool, or
	// a new context.  The plugin has the responsibility of locking accordingly.
	GetContext() interface{}

	CheckPublishPermissions(context interface{}, uid, subject string) bool
	CheckSubscribePermissions(context interface{}, uid, subject string) bool
	CheckConnectPermissions(context interface{}, uid, pass string) bool
}

// MyLittleAuthPlugin is a user plugin
type MyLittleAuthPlugin struct {
	// general stuff, configuration, plugin wide stuff.
	// this could optionally be the context as well, if the
	// context was a singleton.
	singleContext bool
	context       interface{}
}

// MyLittleAuthPluginContext is a context to be passed around to the
// APIs.  If the context is a singleton, the plugin itself can be used.
type MyLittleAuthPluginContext struct {
	Username string
	Password string
	Subject  string
}

func (mlap *MyLittleAuthPlugin) Startup() {
	fmt.Printf("Starting up.")
}

func (mlap *MyLittleAuthPlugin) Shutdown() {
	fmt.Printf("Starting up.")
}

// invoked every time a new client connection is established
func (mlap *MyLittleAuthPlugin) GetContext() interface{} {
	if mlap.singleContext {
		return mlap.context
	}

	return &MyLittleAuthPluginContext{
		Username: "colin",
		Password: "password",
		Subject:  "foo",
	}

}

func (mlap *MyLittleAuthPlugin) CheckPublishPermissions(context interface{}, uid, subject string) bool {
	ma := context.(*MyLittleAuthPluginContext)
	return strings.Compare(ma.Subject, subject) == 0
}
func (mlap *MyLittleAuthPlugin) CheckSubscribePermissions(context interface{}, uid, subject string) bool {
	ma := context.(*MyLittleAuthPluginContext)
	return strings.Compare(ma.Subject, subject) == 0
}

func (mlap *MyLittleAuthPlugin) CheckConnectPermissions(context interface{}, uid, pass string) bool {
	ma := context.(*MyLittleAuthPluginContext)
	if strings.Compare(ma.Username, uid) == 0 && strings.Compare(ma.Password, pass) == 0 {
		return true
	}
	return false
}

func TestAuthPluginAsServer(t *testing.T) {
	// Server's responsibilities
	var plugin AuthPlugin

	// On startup server sets the plugin
	plugin = &MyLittleAuthPlugin{}

	plugin.Startup()

	// At connection time, get the user context.
	uc := plugin.GetContext()
	// context is stored with the connection, and passed to relevant APIs
	if !plugin.CheckConnectPermissions(uc, "colin", "password") {
		t.Fatalf("credential check failed")
	}

	if plugin.CheckConnectPermissions(uc, "colin", "garbage") {
		t.Fatalf("credential check failed")
	}

	if !plugin.CheckPublishPermissions(uc, "colin", "foo") {
		t.Fatalf("publish check failed")
	}

	if plugin.CheckSubscribePermissions(uc, "colin", "bar") {
		t.Fatalf("subscribe check failed")
	}

	plugin.Shutdown()
}

Related Issues

#428
#429
#369

[wenzheng]

Hi @petemiron, do you have any plan to implement this requirement?

[petemiron]

Hi @wenzheng, do you have feedback on this? Many of the ideas stem from your Pull Request, but we've tried to balance performance with the flexibility in your suggestion in these requirements. If you agree and would like to modify your PR to suit these requirements, we'd happily review it. We do think this is a great idea, but our core team doesn't have the bandwidth to implement at this point. We just wanted to make sure to capture the suggestions as a set of requirements.

[wenzheng]

Hi @petemiron

I see our colleague @firebook had commented in the previous PR#429, but yes I think it would be possible for us to modify the PR to suit the requirements, I will talk to our team and see when can we make this happen

[x6j8x]

👍 for this... This would enable us to provide NATS as a brokered service to our apps running on Cloudfoundry. In fact I could live with a very minimal implementation as in #428.

[valichek]

Hi, We would like to know what is the current status of this issue? We are going to use nats to connect >50k devices from "outer space" and not happy with updating/reloading config files.

@petemiron @derekcollison It's clear that the performance and reliability are top priorities when reviewing external auth feature. But what about having user auth service internally connected to nats with system subscriptions. Have you discussed this possibility already? Are there any stop factors to implement it? The idea is to have have nats client(s) to serve as auth provider. I see it now like having additional subscriptions and/or maybe protocol message, so auth provider(s) is able to announce when ready and be registered by gnatsd.

[derekcollison]

I believe reloading configuration files, a WIP, will solve the majority of needs here.

[x6j8x]

@derekcollison At least for our use-case (automatic provisioning of nats subject subtrees to apps on Cloudfoundry & Kubernetes via a service-broker) it feels awkward and error prone to generate nats configs on multiple nodes and then to rely on config reloads.

[derekcollison]

I would imagine that the process would be automated, where config files are properly generated, updated, distributed securely, and the server's signalled properly to reload the configuration.

[x6j8x]

It's certainly doable, but something like #428 would be so much simpler and less error prone (and without timing issues - the state of the authn/authz can always authoritatively be answered by the external entity and is not "in flux" during the regeneration/reload of the config).

[derekcollison]

I think the idea has merit, however it is not complete. For instance, when a user is removed, or permissions updated, these cases are covered by configuration reload, but are currently not accounted for in #428. The synchronization issues are the same in both cases IMO as well as error handling and exceptions.

[valichek]

Removed user or updated permission could be tracked with auth TTL (should have some configured period) if not available/changed - close connection to force client to re-connect. If one don't want TTL defined because of additional traffic, there should be a possibility to receive the message from auth provider and kill connection. It could be webhook or subscription if auth provider is nats client

[eljefedelrodeodeljefe]

Can you clarify on the timeline? I can only chime in, that this is much sought after.

[VladimirAkopyan]

something like this would be great, and I imagine quite simple:
https://github.com/rabbitmq/rabbitmq-auth-backend-http

[vtolstov]

any progress?

[derekcollison]

We are moving forward with this through our Nkey and JWT work. Will keep everyone posted as best we can. Look for something before end of year.

[vtolstov]

may be you have some testing code in branch? i'm realy want to check this

[derekcollison]

You can customize permissions with NATS in operator mode using claim JWTs and private nkeys (or designate the JWT to be a bearer token).

Do you own the Oauth2 domain?

[bbdb68]

thanks for your quick answer. This may be not the best place to discuss this, but I do not own the domain. I receive on the client side an authorization token I am supposed to validate on the server side, at login time, using custom rules (validate against public key retrieved by http, and check a few fields of the token). I wonder how it could be done in the nats ecosystem.

[ripienaar]

As far as I know the only way we support today is if you emedded the nats broker into your own code, you can then supply your own authentication logic.

[derekcollison]

You could have a bearer token delivered to the client. Or a normal creds file as well. Will that now work?

[bbdb68]

I saw #1149 and it is really close. IMHO there may be corner cases where a server side script for token validation may be usefull. But this PR is very close to offer Oauth2 compliant authenticaction, yes.

[bbdb68]

as an example there is a rfc that specifies how to validate the token (RFC 7662 I think), but the people I work with (automotive industry) ask for specific way to validate the token, so the parameters exposed in the PR may not be enough for all cases. IMHO it woud be nice to have some kind of plugin or external resolver pattern for custom token validation, as not every identity provider or deployment are not exactly following the spec. I agree that in strict oauth2 cases, it should be possible to implement validation through configuration file.

[YoSev]

We wanted to go for nats back in 2017 but we missed this feature.
Now, five and a half years later, we are working on a new platform and see this ticket is still pending, so again we will not be able to use nats.
Are there any hot news/plans you could share regarding this request @derekcollison?
Thanks in advance.

[ripienaar]

Are there any hot news/plans you could share regarding this request @derekcollison? Thanks in advance.

Its on the near term roadmap. As it's such a varied topic maybe you can expand a bit on exactly how you would want to see this work?

[YoSev]

Thanks for that update @ripienaar

We use the message bus not only for internal microservice communication, but we expose it and connect every user that is using our platform's webinterface + our thick app to the message bus, to use its full potential. This can be up to 500k users in our case.

Pushing messages to a webinterface using fine graded subscribe patterns like app.message.<username>.new enables us to reach out to a user (or a group of users like app.message.<groupId>.new) just by allowing them to subscribe for a topic others are not allowed to subscribe for.
Each user sends his username + JWT in form of a password after establishing a connection which will be forwarded and validated by our own identity provider using HTTPs.

Why recreating the complete exchange logic across a horizontally scaled infrastructure to reach certain users being connected to different endpoints, when nats does exactly this?

[ripienaar]

@JOHN-DEV thanks, for the input sure it will be handy when we work on this.

Building something like this using nats server as a library is fairly trivial - have done a NATS server that calls a external auth in under a hour, but sure its not to everyones liking to build custom server binaries.

[YoSev]

Exactly, using custom build binaries is not an option for us, unfortunately.
I guess we wait then - hopefully not for another 5 years ;-)

[ripienaar]

The auth system is an already supported extension point, when calling the server from Go.

[YoSev]

Can you outline what that exactly means?

[ripienaar]

Can you outline what that exactly means?

Ignoring whats involved in starting a nats server in go - we have many examples in tests - you can replace the auth system entirely with your own.

type MyAuth struct {
  user string
  pass string
}

func (a *MyAuth) Check(Check(c server.ClientAuthentication) bool {
  opts := c.GetOpts()

  return opts.Username == user && opts.Password == pass
}

You can now set this into the run time options CustomClientAuthentication and the server will use that auth.

So this is a super simple example, but there in opts you have access to the supplied JWT and much more, ip addresses, the TLS state is there and all sorts of things and can enable all you need.

To note this entirely replaces the built in auth system.

So the work to support external auth would be to work on this area, extend it to perhaps call external binaries or perhaps to call out over NATS in specially designated account for a service that can handle auth and so forth based on what users say they need, we're likely to start with something basic and expand over time.

[YoSev]

Interesting.
I managed to implement out own backend-auth logic, is there also an option for a custom permission source?

Yup, you can:

user := &server.User{
		Username:    "bob",
		Account:     "some_account_that_exists",
		Permissions: &server.Permissions{
				// set permissions  
		},
}

c.RegisterUser(user)

[ripienaar]

Sorry, Account is a handle to the server account that you can get with LookupOrRegisterAccount()

[YoSev]

Thanks for that, i got authentication and permissions working.
I will do some tests within our platform next week and check if thats a route we would like to go.

It is not ideal to have a custom build but well, nothing is perfect.
Having this implemented as a module would still be appreciated for a lot of people i assume.

[ripienaar]

Indeed, not ideal, but maybe this gets you going and moving to whatever the final natively supported system we decide on will not be onerous.

[ripienaar]

Wow, I note I edited your earlier comment instead of commenting new - sorry about that, anyway we got there in the end!

[derekcollison]

Auth callouts and mapping external Authn to NATS Authz is top priority for 2.10 release. Along with the ability to reuse an existing connection.

[drodriguez-wave]

Hi, is there a temporary branch where we can test this before release in its current state?

[jnmoyne]

Yes, the dev branch

[gedw99]

NATS Authorization Callout I also need

[jnmoyne]

@bruth I think this can now be closed as 2.10 has been released

[bruth]

See https://docs.nats.io/running-a-nats-service/configuration/securing_nats/auth_callout