Basic enterprise search support #309
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,29 +14,35 @@ import ( | |
|
|
||
| appsv1 "k8s.io/api/apps/v1" | ||
| corev1 "k8s.io/api/core/v1" | ||
| apierrors "k8s.io/apimachinery/pkg/api/errors" | ||
| ctrl "sigs.k8s.io/controller-runtime" | ||
|
|
||
| mdbv1 "github.com/mongodb/mongodb-kubernetes/api/v1/mdb" | ||
| searchv1 "github.com/mongodb/mongodb-kubernetes/api/v1/search" | ||
| "github.com/mongodb/mongodb-kubernetes/controllers/search_controller" | ||
| mdbcv1 "github.com/mongodb/mongodb-kubernetes/mongodb-community-operator/api/v1" | ||
| "github.com/mongodb/mongodb-kubernetes/mongodb-community-operator/controllers/watch" | ||
| kubernetesClient "github.com/mongodb/mongodb-kubernetes/mongodb-community-operator/pkg/kube/client" | ||
| "github.com/mongodb/mongodb-kubernetes/pkg/kube" | ||
| "github.com/mongodb/mongodb-kubernetes/pkg/kube/commoncontroller" | ||
| "github.com/mongodb/mongodb-kubernetes/pkg/util" | ||
| "github.com/mongodb/mongodb-kubernetes/pkg/util/env" | ||
| ) | ||
|
|
||
| type MongoDBSearchReconciler struct { | ||
| kubeClient kubernetesClient.Client | ||
| mdbcWatcher watch.ResourceWatcher | ||
| mdbWatcher watch.ResourceWatcher | ||
| secretWatcher watch.ResourceWatcher | ||
| operatorSearchConfig search_controller.OperatorSearchConfig | ||
| } | ||
|
|
||
| func newMongoDBSearchReconciler(client client.Client, operatorSearchConfig search_controller.OperatorSearchConfig) *MongoDBSearchReconciler { | ||
| return &MongoDBSearchReconciler{ | ||
| kubeClient: kubernetesClient.NewClient(client), | ||
| mdbcWatcher: watch.New(), | ||
|
There was a problem hiding this comment. blocking: we cannot have these watchers in the controller instance. It must be placed in MongoDBSearchReconcilerHelper as the controller is a singleton executed concurrently by the runtime in case we have more than one reconcile at a time configured. Also the resource watcher here is not made to be thread safe and we should migrate to the one from MEKO packages, which have locks (controllers/operator/watch/resource_watcher.go). Also There was a problem hiding this comment. So, I tried repurposing field indexes for watches as described in Kubebuilder's last-version docs and it works pretty neatly for resource we have a direct relationship against. We can generate reconcile requests for changes in However, where this breaks down for me is watching related resources like the CA or keyfile secrets since we don't build indexes against them. I was looking into transitive events, as in "if the CA or keyfile secrets change, shouldn't that cause at least a temporary change in the MongoDBCommunity status that will trigger the search controller's mdbc field index watch?" but I didn't have the time to verify that assumption. That's why I fell back to using watches. Field indexes aside, I'll switch to the thread-safe watcher used in |
||
| mdbWatcher: watch.New(), | ||
| secretWatcher: watch.New(), | ||
| operatorSearchConfig: operatorSearchConfig, | ||
| } | ||
| } | ||
|
|
@@ -51,36 +57,52 @@ func (r *MongoDBSearchReconciler) Reconcile(ctx context.Context, request reconci | |
| return result, err | ||
| } | ||
|
|
||
| searchSource, err := r.getSourceMongoDBForSearch(ctx, r.kubeClient, mdbSearch, log) | ||
| if err != nil { | ||
| return reconcile.Result{RequeueAfter: time.Second * util.RetryTimeSec}, err | ||
| } | ||
|
|
||
| r.secretWatcher.Watch(ctx, kube.ObjectKey(mdbSearch.GetNamespace(), searchSource.KeyfileSecretName()), mdbSearch.NamespacedName()) | ||
|
|
||
| reconcileHelper := search_controller.NewMongoDBSearchReconcileHelper(kubernetesClient.NewClient(r.kubeClient), mdbSearch, searchSource, r.operatorSearchConfig) | ||
|
|
||
| return reconcileHelper.Reconcile(ctx, log).ReconcileResult() | ||
| } | ||
|
|
||
| func (r *MongoDBSearchReconciler) getSourceMongoDBForSearch(ctx context.Context, kubeClient client.Client, search *searchv1.MongoDBSearch, log *zap.SugaredLogger) (search_controller.SearchSourceDBResource, error) { | ||
| if search.IsExternalMongoDBSource() { | ||
| return search_controller.NewExternalSearchSource(search.Namespace, search.Spec.Source.ExternalMongoDBSource), nil | ||
| } | ||
|
|
||
| sourceMongoDBResourceRef := search.GetMongoDBResourceRef() | ||
| if sourceMongoDBResourceRef == nil { | ||
| return nil, xerrors.New("MongoDBSearch source MongoDB resource reference is not set") | ||
| } | ||
|
|
||
| sourceName := types.NamespacedName{Namespace: search.GetNamespace(), Name: sourceMongoDBResourceRef.Name} | ||
| log.Infof("Looking up Search source %s", sourceName) | ||
|
|
||
| mdb := &mdbv1.MongoDB{} | ||
| if err := kubeClient.Get(ctx, sourceName, mdb); err != nil { | ||
| if !apierrors.IsNotFound(err) { | ||
| return nil, xerrors.Errorf("error getting MongoDB %s: %w", sourceName, err) | ||
| } | ||
| } else { | ||
| r.mdbWatcher.Watch(ctx, sourceName, search.NamespacedName()) | ||
| return search_controller.NewEnterpriseResourceSearchSource(mdb), nil | ||
| } | ||
|
|
||
| mdbc := &mdbcv1.MongoDBCommunity{} | ||
| if err := kubeClient.Get(ctx, sourceName, mdbc); err != nil { | ||
| if !apierrors.IsNotFound(err) { | ||
| return nil, xerrors.Errorf("error getting MongoDBCommunity %s: %w", sourceName, err) | ||
| } | ||
| } else { | ||
| r.mdbcWatcher.Watch(ctx, sourceName, search.NamespacedName()) | ||
| return search_controller.NewCommunityResourceSearchSource(mdbc), nil | ||
| } | ||
|
|
||
| return nil, xerrors.Errorf("No database resource named %s found", sourceName) | ||
| } | ||
|
|
||
| func mdbcSearchIndexBuilder(rawObj client.Object) []string { | ||
|
|
@@ -103,7 +125,9 @@ func AddMongoDBSearchController(ctx context.Context, mgr manager.Manager, operat | |
| return ctrl.NewControllerManagedBy(mgr). | ||
| WithOptions(controller.Options{MaxConcurrentReconciles: env.ReadIntOrDefault(util.MaxConcurrentReconcilesEnv, 1)}). // nolint:forbidigo | ||
| For(&searchv1.MongoDBSearch{}). | ||
| Watches(&mdbv1.MongoDB{}, r.mdbWatcher). | ||
| Watches(&mdbcv1.MongoDBCommunity{}, r.mdbcWatcher). | ||
| Watches(&corev1.Secret{}, r.secretWatcher). | ||
| Owns(&appsv1.StatefulSet{}). | ||
| Owns(&corev1.Secret{}). | ||
| Complete(r) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't we return false here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure. If we return false then we won't mirror the keyfile from Ops Manager into the secret that mongot will be waiting for. If customers are using a customized MongoDB version it might either not need the
searchCoordinatorrole polyfill, or they might apply it themselves in theMongoDBspec, so not returning false if we can't parse the MongoDB version doesn't prevent that very convoluted scenario from actually working with Search.On the other hand, I can't realistically imagine when we might fail parsing the MongoDB version, so I can't decide if this case is actually worth guarding against.