-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
HBASE-23347 Allow custom authentication methods for RPCs
Decouple the HBase internals such that someone can implement their own SASL-based authentication mechanism and plug it into HBase RegionServers/Masters. Comes with a design doc in dev-support/design-docs and an example in hbase-examples known as "Shade" which uses a flat-password file for authenticating users. Closes #884 Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Andrew Purtell <apurtell@apache.org> Signed-off-by: Reid Chan <reidchan@apache.org>
- Loading branch information
Showing
53 changed files
with
4,102 additions
and
537 deletions.
There are no files selected for viewing
179 changes: 179 additions & 0 deletions
179
dev-support/design-docs/HBASE-23347-pluggable-authentication.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| <!-- | ||
| Licensed to the Apache Software Foundation (ASF) under one | ||
| or more contributor license agreements. See the NOTICE file | ||
| distributed with this work for additional information | ||
| regarding copyright ownership. The ASF licenses this file | ||
| to you under the Apache License, Version 2.0 (the | ||
| "License"); you may not use this file except in compliance | ||
| with the License. You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| --> | ||
|
|
||
| # Pluggable Authentication for HBase RPCs | ||
|
|
||
| ## Background | ||
|
|
||
| As a distributed database, HBase must be able to authenticate users and HBase | ||
| services across an untrusted network. Clients and HBase services are treated | ||
| equivalently in terms of authentication (and this is the only time we will | ||
| draw such a distinction). | ||
|
|
||
| There are currently three modes of authentication which are supported by HBase | ||
| today via the configuration property `hbase.security.authentication` | ||
|
|
||
| 1. `SIMPLE` | ||
| 2. `KERBEROS` | ||
| 3. `TOKEN` | ||
|
|
||
| `SIMPLE` authentication is effectively no authentication; HBase assumes the user | ||
| is who they claim to be. `KERBEROS` authenticates clients via the KerberosV5 | ||
| protocol using the GSSAPI mechanism of the Java Simple Authentication and Security | ||
| Layer (SASL) protocol. `TOKEN` is a username-password based authentication protocol | ||
| which uses short-lived passwords that can only be obtained via a `KERBEROS` authenticated | ||
| request. `TOKEN` authentication is synonymous with Hadoop-style [Delegation Tokens](https://steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/hadoop_tokens.html#delegation-tokens). `TOKEN` authentication uses the `DIGEST-MD5` | ||
| SASL mechanism. | ||
|
|
||
| [SASL](https://docs.oracle.com/javase/8/docs/technotes/guides/security/sasl/sasl-refguide.html) | ||
| is a library which specifies a network protocol that can authenticate a client | ||
| and a server using an arbitrary mechanism. SASL ships with a [number of mechanisms](https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml) | ||
| out of the box and it is possible to implement custom mechanisms. SASL is effectively | ||
| decoupling an RPC client-server model from the mechanism used to authenticate those | ||
| requests (e.g. the RPC code is identical whether username-password, Kerberos, or any | ||
| other method is used to authenticate the request). | ||
|
|
||
| RFC's define what [SASL mechanisms exist](https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xml), | ||
| but what RFC's define are a superset of the mechanisms that are | ||
| [implemented in Java](https://docs.oracle.com/javase/8/docs/technotes/guides/security/sasl/sasl-refguide.html#SUN). | ||
| This document limits discussion to SASL mechanisms in the abstract, focusing on those which are well-defined and | ||
| implemented in Java today by the JDK itself. However, it is completely possible that a developer can implement | ||
| and register their own SASL mechanism. Writing a custom mechanism is outside of the scope of this document, but | ||
| not outside of the realm of possibility. | ||
|
|
||
| The `SIMPLE` implementation does not use SASL, but instead has its own RPC logic | ||
| built into the HBase RPC protocol. `KERBEROS` and `TOKEN` both use SASL to authenticate, | ||
| relying on the `Token` interface that is intertwined with the Hadoop `UserGroupInformation` | ||
| class. SASL decouples an RPC from the mechanism used to authenticate that request. | ||
|
|
||
| ## Problem statement | ||
|
|
||
| Despite HBase already shipping authentication implementations which leverage SASL, | ||
| it is (effectively) impossible to add a new authentication implementation to HBase. The | ||
| use of the `org.apache.hadoop.hbase.security.AuthMethod` enum makes it impossible | ||
| to define a new method of authentication. Also, the RPC implementation is written | ||
| to only use the methods that are expressly shipped in HBase. Adding a new authentication | ||
| method would require copying and modifying the RpcClient implementation, in addition | ||
| to modifying the RpcServer to invoke the correct authentication check. | ||
|
|
||
| While it is possible to add a new authentication method to HBase, it cannot be done | ||
| cleanly or sustainably. This is what is meant by "impossible". | ||
|
|
||
| ## Proposal | ||
|
|
||
| HBase should expose interfaces which allow for pluggable authentication mechanisms | ||
| such that HBase can authenticate against external systems. Because the RPC implementation | ||
| can already support SASL, HBase can standardize on SASL, allowing any authentication method | ||
| which is capable of using SASL to negotiate authentication. `KERBEROS` and `TOKEN` methods | ||
| will naturally fit into these new interfaces, but `SIMPLE` authentication will not (see the following | ||
| chapter for a tangent on SIMPLE authentication today) | ||
|
|
||
| ### Tangent: on SIMPLE authentication | ||
|
|
||
| `SIMPLE` authentication in HBase today is treated as a special case. My impression is that | ||
| this stems from HBase not originally shipping an RPC solution that had any authentication. | ||
|
|
||
| Re-implementing `SIMPLE` authentication such that it also flows through SASL (e.g. via | ||
| the `PLAIN` SASL mechanism) would simplify the HBase codebase such that all authentication | ||
| occurs via SASL. This was not done for the initial implementation to reduce the scope | ||
| of the changeset. Changing `SIMPLE` authentication to use SASL may result in some | ||
| performance impact in setting up a new RPC. The same conditional logic to determine | ||
| `if (sasl) ... else SIMPLE` logic is propagated in this implementation. | ||
|
|
||
| ## Implementation Overview | ||
|
|
||
| HBASE-23347 includes a refactoring of HBase RPC authentication where all current methods | ||
| are ported to a new set of interfaces, and all RPC implementations are updated to use | ||
| the new interfaces. In the spirit of SASL, the expectation is that users can provide | ||
| their own authentication methods at runtime, and HBase should be capable of negotiating | ||
| a client who tries to authenticate via that custom authentication method. The implementation | ||
| refers to this "bundle" of client and server logic as an "authentication provider". | ||
|
|
||
| ### Providers | ||
|
|
||
| One authentication provider includes the following pieces: | ||
|
|
||
| 1. Client-side logic (providing a credential) | ||
| 2. Server-side logic (validating a credential from a client) | ||
| 3. Client selection logic to choose a provider (from many that may be available) | ||
|
|
||
| A provider's client and server side logic are considered to be one-to-one. A `Foo` client-side provider | ||
| should never be used to authenticate against a `Bar` server-side provider. | ||
|
|
||
| We do expect that both clients and servers will have access to multiple providers. A server may | ||
| be capable of authenticating via methods which a client is unaware of. A client may attempt to authenticate | ||
| against a server which the server does not know how to process. In both cases, the RPC | ||
| should fail when a client and server do not have matching providers. The server identifies | ||
| client authentication mechanisms via a `byte authCode` (which is already sent today with HBase RPCs). | ||
|
|
||
| A client may also have multiple providers available for it to use in authenticating against | ||
| HBase. The client must have some logic to select which provider to use. Because we are | ||
| allowing custom providers, we must also allow a custom selection logic such that the | ||
| correct provider can be chosen. This is a formalization of the logic already present | ||
| in `org.apache.hadoop.hbase.security.token.AuthenticationTokenSelector`. | ||
|
|
||
| To enable the above, we have some new interfaces to support the user extensibility: | ||
|
|
||
| 1. `interface SaslAuthenticationProvider` | ||
| 2. `interface SaslClientAuthenticationProvider extends SaslAuthenticationProvider` | ||
| 3. `interface SaslServerAuthenticationProvider extends SaslAuthenticationProvider` | ||
| 4. `interface AuthenticationProviderSelector` | ||
|
|
||
| The `SaslAuthenticationProvider` shares logic which is common to the client and the | ||
| server (though, this is up to the developer to guarantee this). The client and server | ||
| interfaces each have logic specific to the HBase RPC client and HBase RPC server | ||
| codebase, as their name implies. As described above, an implementation | ||
| of one `SaslClientAuthenticationProvider` must match exactly one implementation of | ||
| `SaslServerAuthenticationProvider`. Each Authentication Provider implementation is | ||
| a singleton and is intended to be shared across all RPCs. A provider selector is | ||
| chosen per client based on that client's configuration. | ||
|
|
||
| A client authentication provider is uniquely identified among other providers | ||
| by the following characteristics: | ||
|
|
||
| 1. A name, e.g. "KERBEROS", "TOKEN" | ||
| 2. A byte (a value between 0 and 255) | ||
|
|
||
| In addition to these attributes, a provider also must define the following attributes: | ||
|
|
||
| 3. The SASL mechanism being used. | ||
| 4. The Hadoop AuthenticationMethod, e.g. "TOKEN", "KERBEROS", "CERTIFICATE" | ||
| 5. The Token "kind", the name used to identify a TokenIdentifier, e.g. `HBASE_AUTH_TOKEN` | ||
|
|
||
| It is allowed (even expected) that there may be multiple providers that use `TOKEN` authentication. | ||
|
|
||
| N.b. Hadoop requires all `TokenIdentifier` implements to have a no-args constructor and a `ServiceLoader` | ||
| entry in their packaging JAR file (e.g. `META-INF/services/org.apache.hadoop.security.token.TokenIdentifier`). | ||
| Otherwise, parsing the `TokenIdentifier` on the server-side end of an RPC from a Hadoop `Token` will return | ||
| `null` to the caller (often, in the `CallbackHandler` implementation). | ||
|
|
||
| ### Factories | ||
|
|
||
| To ease development with these unknown set of providers, there are two classes which | ||
| find, instantiate, and cache the provider singletons. | ||
|
|
||
| 1. Client side: `class SaslClientAuthenticationProviders` | ||
| 2. Server side: `class SaslServerAuthenticationProviders` | ||
|
|
||
| These classes use [Java ServiceLoader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) | ||
| to find implementations available on the classpath. The provided HBase implementations | ||
| for the three out-of-the-box implementations all register themselves via the `ServiceLoader`. | ||
|
|
||
| Each class also enables providers to be added via explicit configuration in hbase-site.xml. | ||
| This enables unit tests to define custom implementations that may be toy/naive/unsafe without | ||
| any worry that these may be inadvertently deployed onto a production HBase cluster. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.