Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OIDC k8s provider #1576

Merged
merged 5 commits into from
Feb 26, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 82 additions & 8 deletions .evergreen/.evg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -798,6 +798,30 @@ functions:
${PREPARE_SHELL}
MONGODB_URI="${MONGODB_URI}" JAVA_VERSION="${JAVA_VERSION}" .evergreen/run-graalvm-native-image-app.sh

"oidc-auth-test-k8s-func":
- command: shell.exec
type: test
params:
shell: bash
include_expansions_in_env: ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_SESSION_TOKEN"]
script: |-
set -o errexit
${PREPARE_SHELL}
export K8S_VARIANT=${VARIANT}
cd src
git add .
git commit --allow-empty -m "add files"
# uncompressed tar used to allow appending .git folder
export K8S_DRIVERS_TAR_FILE=/tmp/mongo-java-driver.tar
git archive -o $K8S_DRIVERS_TAR_FILE HEAD
tar -rf $K8S_DRIVERS_TAR_FILE .git
export K8S_TEST_CMD="OIDC_ENV=k8s VARIANT=${VARIANT} ./.evergreen/run-mongodb-oidc-test.sh"
bash $DRIVERS_TOOLS/.evergreen/auth_oidc/k8s/setup-pod.sh
bash $DRIVERS_TOOLS/.evergreen/auth_oidc/k8s/run-self-test.sh
source $DRIVERS_TOOLS/.evergreen/auth_oidc/k8s/secrets-export.sh
bash $DRIVERS_TOOLS/.evergreen/auth_oidc/k8s/run-driver-test.sh
bash $DRIVERS_TOOLS/.evergreen/auth_oidc/k8s/teardown-pod.sh

# Anchors

pre:
Expand Down Expand Up @@ -921,6 +945,22 @@ tasks:
export GCPOIDC_TEST_CMD="OIDC_ENV=gcp ./.evergreen/run-mongodb-oidc-test.sh"
bash $DRIVERS_TOOLS/.evergreen/auth_oidc/gcp/run-driver-test.sh

- name: "oidc-auth-test-k8s"
commands:
- command: ec2.assume_role
params:
role_arn: ${aws_test_secrets_role}
duration_seconds: 1800
- func: "oidc-auth-test-k8s-func"
vars:
VARIANT: eks
- func: "oidc-auth-test-k8s-func"
vars:
VARIANT: aks
- func: "oidc-auth-test-k8s-func"
vars:
VARIANT: gke
Comment on lines +948 to +962
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we break this into separate tasks and list them in the test-oidc-k8s-task-group task group. This would allow us to:

  • Run specific tests for EKS, AKS, or GCE individually if needed (currently, all three run together).
  • Reduce test runtime by leveraging Evergreen’s parallel execution of tasks.


- name: serverless-test
commands:
- func: "run serverless"
Expand Down Expand Up @@ -2011,7 +2051,7 @@ task_groups:
tasks:
- testazurekms-task

- name: testoidc_task_group
- name: test-oidc-task-group
setup_group:
- func: fetch source
- func: prepare resources
Expand All @@ -2036,7 +2076,7 @@ task_groups:
tasks:
- oidc-auth-test

- name: testazureoidc_task_group
- name: test-oidc-azure-task-group
setup_group:
- func: fetch source
- func: prepare resources
Expand All @@ -2059,7 +2099,7 @@ task_groups:
tasks:
- oidc-auth-test-azure

- name: testgcpoidc_task_group
- name: test-oidc-gcp-task-group
setup_group:
- func: fetch source
- func: prepare resources
Expand All @@ -2083,6 +2123,33 @@ task_groups:
tasks:
- oidc-auth-test-gcp

- name: test-oidc-k8s-task-group
setup_group_can_fail_task: true
setup_group_timeout_secs: 1800
teardown_task_can_fail_task: true
teardown_group_timeout_secs: 180
setup_group:
- func: fetch source
- func: prepare resources
- func: fix absolute paths
- command: ec2.assume_role
params:
role_arn: ${aws_test_secrets_role}
- command: subprocess.exec
params:
binary: bash
include_expansions_in_env: ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_SESSION_TOKEN"]
args:
- ${DRIVERS_TOOLS}/.evergreen/auth_oidc/k8s/setup.sh
teardown_group:
- command: subprocess.exec
params:
binary: bash
args:
- ${DRIVERS_TOOLS}/.evergreen/auth_oidc/k8s/teardown.sh
tasks:
- oidc-auth-test-k8s

buildvariants:

# Test packaging and other release related routines
Expand Down Expand Up @@ -2254,21 +2321,28 @@ buildvariants:
display_name: "OIDC Auth"
run_on: ubuntu2204-small
tasks:
- name: testoidc_task_group
- name: test-oidc-task-group
batchtime: 20160 # 14 days

- name: testazureoidc-variant
- name: test-oidc-azure-variant
display_name: "OIDC Auth Azure"
run_on: ubuntu2204-small
tasks:
- name: testazureoidc_task_group
- name: test-oidc-azure-task-group
batchtime: 20160 # 14 days

- name: testgcpoidc-variant
- name: test-oidc-gcp-variant
display_name: "OIDC Auth GCP"
run_on: ubuntu2204-small
tasks:
- name: testgcpoidc_task_group
- name: test-oidc-gcp-task-group
batchtime: 20160 # 14 days

- name: test-oidc-k8s-variant
display_name: "OIDC Auth K8S"
run_on: ubuntu2204-small
tasks:
- name: test-oidc-k8s-task-group
batchtime: 20160 # 14 days

- matrix_name: "aws-auth-test"
Expand Down
10 changes: 10 additions & 0 deletions .evergreen/run-mongodb-oidc-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ elif [ $OIDC_ENV == "azure" ]; then
source ./env.sh
elif [ $OIDC_ENV == "gcp" ]; then
source ./secrets-export.sh
elif [ $OIDC_ENV == "k8s" ]; then
# Make sure K8S_VARIANT is set.
if [ -z "$K8S_VARIANT" ]; then
echo "Must specify K8S_VARIANT"
popd
exit 1
fi

# fix for git permissions issue:
git config --global --add safe.directory /tmp/test
else
echo "Unrecognized OIDC_ENV $OIDC_ENV"
exit 1
Expand Down
9 changes: 7 additions & 2 deletions driver-core/src/main/com/mongodb/MongoCredential.java
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ public final class MongoCredential {
/**
* Mechanism property key for specifying the environment for OIDC, which is
* the name of a built-in OIDC application environment integration to use
* to obtain credentials. The value must be either "gcp" or "azure".
* to obtain credentials. The value must be either "k8s", "gcp", or "azure".
* This is an alternative to supplying a callback.
* <p>
* The "gcp" and "azure" environments require
Expand All @@ -199,6 +199,11 @@ public final class MongoCredential {
* {@link MongoCredential#OIDC_CALLBACK_KEY} and
* {@link MongoCredential#OIDC_HUMAN_CALLBACK_KEY}
* must not be provided.
* <p>
* The "k8s" environment will check the env vars
* {@code AZURE_FEDERATED_TOKEN_FILE}, and then {@code AWS_WEB_IDENTITY_TOKEN_FILE},
* for the token file path, and if neither is set will then use the path
* {@code /var/run/secrets/kubernetes.io/serviceaccount/token}.
*
* @see #createOidcCredential(String)
* @see MongoCredential#TOKEN_RESOURCE_KEY
Expand Down Expand Up @@ -265,7 +270,7 @@ public final class MongoCredential {
"*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"));

/**
* Mechanism property key for specifying he URI of the target resource (sometimes called the audience),
* Mechanism property key for specifying the URI of the target resource (sometimes called the audience),
* used in some OIDC environments.
*
* <p>A TOKEN_RESOURCE with a comma character must be given as a `MongoClient` configuration and not as
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,11 @@ public final class OidcAuthenticator extends SaslAuthenticator {
private static final String TEST_ENVIRONMENT = "test";
private static final String AZURE_ENVIRONMENT = "azure";
private static final String GCP_ENVIRONMENT = "gcp";
private static final String K8S_ENVIRONMENT = "k8s";
private static final List<String> IMPLEMENTED_ENVIRONMENTS = Arrays.asList(
AZURE_ENVIRONMENT, GCP_ENVIRONMENT, TEST_ENVIRONMENT);
AZURE_ENVIRONMENT, GCP_ENVIRONMENT, K8S_ENVIRONMENT, TEST_ENVIRONMENT);
private static final List<String> USER_SUPPORTED_ENVIRONMENTS = Arrays.asList(
AZURE_ENVIRONMENT, GCP_ENVIRONMENT);
AZURE_ENVIRONMENT, GCP_ENVIRONMENT, K8S_ENVIRONMENT);
private static final List<String> REQUIRES_TOKEN_RESOURCE = Arrays.asList(
AZURE_ENVIRONMENT, GCP_ENVIRONMENT);
private static final List<String> ALLOWS_USERNAME = Arrays.asList(
Expand All @@ -90,6 +91,10 @@ public final class OidcAuthenticator extends SaslAuthenticator {

public static final String OIDC_TOKEN_FILE = "OIDC_TOKEN_FILE";

private static final String K8S_FALLBACK_FILE = "/var/run/secrets/kubernetes.io/serviceaccount/token";
private static final String K8S_AZURE_FILE = "AZURE_FEDERATED_TOKEN_FILE";
private static final String K8S_AWS_FILE = "AWS_WEB_IDENTITY_TOKEN_FILE";

private static final int CALLBACK_API_VERSION_NUMBER = 1;

@Nullable
Expand Down Expand Up @@ -192,6 +197,8 @@ private OidcCallback getRequestCallback() {
machine = getAzureCallback(getMongoCredential());
} else if (GCP_ENVIRONMENT.equals(environment)) {
machine = getGcpCallback(getMongoCredential());
} else if (K8S_ENVIRONMENT.equals(environment)) {
machine = getK8sCallback();
} else {
machine = getOidcCallbackMechanismProperty(OIDC_CALLBACK_KEY);
}
Expand All @@ -206,6 +213,24 @@ private static OidcCallback getTestCallback() {
};
}

@VisibleForTesting(otherwise = VisibleForTesting.AccessModifier.PRIVATE)
static OidcCallback getK8sCallback() {
return (context) -> {
String azure = System.getenv(K8S_AZURE_FILE);
String aws = System.getenv(K8S_AWS_FILE);
String path;
if (azure != null) {
path = azure;
} else if (aws != null) {
path = aws;
} else {
path = K8S_FALLBACK_FILE;
}
String accessToken = readTokenFromFile(path);
return new OidcCallbackResult(accessToken);
};
}

@VisibleForTesting(otherwise = VisibleForTesting.AccessModifier.PRIVATE)
static OidcCallback getAzureCallback(final MongoCredential credential) {
return (context) -> {
Expand Down Expand Up @@ -499,6 +524,10 @@ private static String readTokenFromFile() {
throw new MongoClientException(
format("Environment variable must be specified: %s", OIDC_TOKEN_FILE));
}
return readTokenFromFile(path);
}

private static String readTokenFromFile(final String path) {
try {
return new String(Files.readAllBytes(Paths.get(path)), StandardCharsets.UTF_8);
} catch (IOException e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,14 @@ static class ShutdownHook extends Thread {
@Override
public void run() {
if (cluster != null) {
new DropDatabaseOperation(getDefaultDatabaseName(), WriteConcern.ACKNOWLEDGED).execute(getBinding());
try {
new DropDatabaseOperation(getDefaultDatabaseName(), WriteConcern.ACKNOWLEDGED).execute(getBinding());
} catch (MongoCommandException e) {
// if we do not have permission to drop the database, assume it is cleaned up in some other way
if (!e.getMessage().contains("Command dropDatabase requires authentication")) {
throw e;
}
}
cluster.close();
Comment on lines -264 to 272
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As of this PR, we will sometimes lack permission to drop the database.

}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@
},
{
"description": "should throw an exception if username is specified for test (MONGODB-OIDC)",
"uri": "mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC&ENVIRONMENT:test",
"uri": "mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=ENVIRONMENT:test",
"valid": false,
"credential": null
},
Expand Down Expand Up @@ -631,6 +631,26 @@
"uri": "mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=ENVIRONMENT:gcp",
"valid": false,
"credential": null
},
{
"description": "should recognise the mechanism with k8s provider (MONGODB-OIDC)",
"uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=ENVIRONMENT:k8s",
"valid": true,
"credential": {
"username": null,
"password": null,
"source": "$external",
"mechanism": "MONGODB-OIDC",
"mechanism_properties": {
"ENVIRONMENT": "k8s"
}
}
},
{
"description": "should throw an error for a username and password with k8s provider (MONGODB-OIDC)",
"uri": "mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=ENVIRONMENT:k8s",
"valid": false,
"credential": null
}
]
}
Original file line number Diff line number Diff line change
Expand Up @@ -826,6 +826,7 @@ private MongoClientSettings createSettings(
String cleanedConnectionString = callback == null ? connectionString : connectionString
.replace("ENVIRONMENT:azure,", "")
.replace("ENVIRONMENT:gcp,", "")
.replace("&authMechanismProperties=ENVIRONMENT:k8s", "")
.replace("ENVIRONMENT:test,", "");
return createSettings(cleanedConnectionString, callback, commandListener, OIDC_CALLBACK_KEY);
}
Expand Down Expand Up @@ -1042,6 +1043,8 @@ private OidcCallbackResult callback(final OidcCallbackContext context) {
c = OidcAuthenticator.getAzureCallback(credential);
} else if (oidcEnv.contains("gcp")) {
c = OidcAuthenticator.getGcpCallback(credential);
} else if (oidcEnv.contains("k8s")) {
c = OidcAuthenticator.getK8sCallback();
} else {
c = getProseTestCallback();
}
Expand Down