Make account registration whitelists local (#3043)

 Make account registration whitelists local #3043 

https://wearezeta.atlassian.net/browse/SQPIT-405 (a related wire infrastructure PR is linked in the ticket)

This is changing a feature wire has been using on our staging environment, and (probably?) not anywhere else. See the changelog if you think you may be affected.

Since the service is both outdated and almost unused, this PR moves the data from that service into the local server config yaml.

Migration should be painless, since the new settings are in a different place than the old ones. Just make sure the new fields are added to the config before the upgrade, and then you can remove the old ones at any time after.

changelog.d/0-release-notes/SQPIT-405-make-whitelist-local

@@ -0,0 +1 @@
+In (the unlikely) case your server config file contains `setWhitelist:`, you need to change this before the upgrade!  It used to refer to a whitelisting service, which is now replaced with a local list of allowed domains and phone numbers.  See [docs](https://docs.wire.com/developer/reference/user/activation.html?highlight=whitelist#phone-email-whitelist) for details.  Migration path: add new config fields; upgrade, remove old config fields.

charts/brig/templates/configmap.yaml

@@ -270,8 +270,11 @@ data:
       {{- if .setSftListAllServers }}
       setSftListAllServers: {{ .setSftListAllServers }}
       {{- end }}
-      {{- if .setWhitelist }}
-      setWhitelist: {{ toYaml .setWhitelist | nindent 8 }}
+      {{- if .setAllowlistEmailDomains }}
+      setAllowlistEmailDomains: {{ toYaml .setAllowlistEmailDomains | nindent 8 }}
+      {{- end }}
+      {{- if .setAllowlistPhonePrefixes }}
+      setAllowlistPhonePrefixes: {{ toYaml .setAllowlistPhonePrefixes | nindent 8 }}
       {{- end }}
       {{- if .setFeatureFlags }}
       setFeatureFlags: {{ toYaml .setFeatureFlags | nindent 8 }}

docs/src/developer/developer/pr-guidelines.md

@@ -18,7 +18,7 @@ See `docs/legacy/developer/changelog.md` for more information.
 
 ## Schema migrations
 
-Don't delete columns that are still used by versions that are deployed. If you delete columns then the old version will fail in the deployment process. Rather than deleting keep the unused columns around and comment them as being discontinued in the schema migration code. 
+Don't delete columns that are still used by versions that are deployed. If you delete columns then the old version will fail in the deployment process. Rather than deleting keep the unused columns around and comment them as being discontinued in the schema migration code.
 
 If a cassandra schema migration has been added then add this to the checklist:
 
@@ -84,7 +84,7 @@ If a PR adds new configuration options for say brig, the following files need to
 * [ ] The integration test config: `services/brig/brig.integration.yaml`
 * [ ] The charts: `charts/brig/templates/configmap.yaml`
 * [ ] The default values: `charts/brig/values.yaml`
-* [ ] The values files for CI: `hack/helm_vars/wire-server/values.yaml`
+* [ ] The values files for CI: `hack/helm_vars/wire-server/values.yaml.gotmpl`
 * [ ] The configuration docs: `docs/src/developer/reference/config-options.md`
 
 If any new configuration value is required and has no default, then:
@@ -100,8 +100,9 @@ Remove them with the PR from wire-server `./charts` folder, as charts are linked
 
 ### Renaming configuration flags
 
-Avoid doing this. If you must, see Removing/adding sections above. But please note that all people who have an installation of wire also may have overridden any of the configuration option you may wish to change the name of. As this is not type checked, it's very error prone and people may find themselves with default configuration values being used instead of their intended configuration settings. Guideline: only rename for good reasons, not for aesthetics; or be prepared to spend a significant
-amount on documenting and communication about this change.
+Avoid doing this, it's usually viable to introduce an at-least-equally-good name and remove the old one, that admins can first add the new options, then uprade the software, then remove the old ones.
+
+If you must, see Removing/adding sections above. But please note that all people who have an installation of wire also may have overridden any of the configuration option you may wish to change the name of. As this is not type checked, it's very error prone and people may find themselves with default configuration values being used instead of their intended configuration settings. Guideline: only rename for good reasons, not for aesthetics; or be prepared to spend a significant amount on documenting and communication about this change.
 
 ## Changes to developer workflow
 

docs/src/developer/reference/user/activation.md

@@ -10,7 +10,7 @@ A user is called _activated_ they have a verified identity -- e.g. a phone numbe
 
 A user that has been provisioned via single sign-on is always considered to be activated.
 
-## Activated vs. non-activated users 
+## Activated vs. non-activated users
 (RefActivationBenefits)=
 
 Non-activated users can not [connect](connection.md) to others, nor can connection requests be made to anonymous accounts from verified accounts. As a result:
@@ -19,10 +19,10 @@ Non-activated users can not [connect](connection.md) to others, nor can connecti
 
 The only flow where it makes sense for non-activated users to exist is the [wireless flow](RefRegistrationWireless) used for [guest rooms](https://wire.com/en/features/encrypted-guest-rooms/)
 
-## API 
+## API
 (RefActivationApi)=
 
-### Requesting an activation code 
+### Requesting an activation code
 (RefActivationRequest)=
 
 During the [standard registration flow](RefRegistrationStandard), the user submits an email address or phone number by making a request to `POST /activate/send`. A six-digit activation code will be sent to that email address / phone number. Sample request and response:
@@ -44,7 +44,7 @@ The user can submit the activation code during registration to prove that they o
 
 The same `POST /activate/send` endpoint can be used to re-request an activation code. Please use this ability sparingly! To avoid unnecessary activation code requests, users should be warned that it might take up to a few minutes for an email or text message to arrive.
 
-### Activating an existing account 
+### Activating an existing account
 (RefActivationSubmit)=
 
 If the account [has not been activated during verification](RefRegistrationNoPreverification), it can be activated afterwards by submitting an activation code to `POST /activate`. Sample request and response:
@@ -80,7 +80,7 @@ If the email or phone has been verified already, `POST /activate` will return st
 
 There is a maximum of 3 activation attempts per activation code. On the third failed attempt the code is invalidated and a new one must be requested.
 
-### Activation event 
+### Activation event
 (RefActivationEvent)=
 
 When the user becomes activated, they receive an event:
@@ -92,7 +92,7 @@ When the user becomes activated, they receive an event:
 }
 ```
 
-### Detecting activation in the self profile 
+### Detecting activation in the self profile
 (RefActivationProfile)=
 
 In addition to the [activation event](RefActivationEvent), activation can be detected by polling the self profile:
@@ -114,7 +114,7 @@ GET /self
 
 If the profile includes `"email"` or `"phone"`, the account is activated.
 
-## Automating activation via email 
+## Automating activation via email
 (RefActivationEmailHeaders)=
 
 Our email verification messages contain headers that can be used to automate the activation process.
@@ -134,20 +134,23 @@ X-Zeta-Key: ...
 X-Zeta-Code: 123456
 ```
 
-## Phone/email whitelist 
-(RefActivationWhitelist)=
+## Phone/email whitelist
+(RefActivationAllowlist)=
 
-The backend can be configured to only allow specific phone numbers or email addresses to register. The following options have to be set in `brig.yaml`:
+The backend can be configured to only allow specific phone number prefixes and email address domains to register. The following options have to be set in `brig.yaml`:
 
 ```yaml
 optSettings:
-  setWhitelist:
-    whitelistUrl: ...     # Checker URL
-    whitelistUser: ...    # Basic auth username
-    whitelistPass: ...    # Basic auth password
+  setAllowlistEmailDomains:
+    - wire.com
+    - example.com
+    - notagoodexample.com
+  setAllowlistPhonePrefixes:
+    - +49
+    - +1555555
 ```
 
-When those options are present, the backend will do a GET request at `<whitelistUrl>?email=...` or `<whitelistUrl>?mobile=...` for every activation request it receives. It will expect either status code 200 ("everything good") or 404 ("provided email/phone is not on the whitelist").
+When those options are present, the backend will match every activation request against these lists.
 
 If an email address or phone number are rejected by the whitelist, `POST /activate/send` or `POST /register` will return `403 Forbidden`:
 
@@ -158,5 +161,3 @@ If an email address or phone number are rejected by the whitelist, `POST /activa
     "message": "Unauthorized e-mail address or phone number."
 }
 ```
-
-Currently emails at `@wire.com` are always considered whitelisted, regardless of the whitelist service's response.

libs/wire-api/src/Wire/API/Error/Brig.hs

@@ -43,7 +43,7 @@ data BrigError
   | HandleNotFound
   | UserCreationRestricted
   | BlacklistedPhone
-  | WhitelistError
+  | AllowlistError
   | InvalidInvitationCode
   | MissingIdentity
   | BlacklistedEmail
@@ -135,7 +135,7 @@ type instance MapError 'MLSDuplicatePublicKey = 'StaticError 400 "mls-duplicate-
 
 type instance MapError 'BlacklistedPhone = 'StaticError 403 "blacklisted-phone" "The given phone number has been blacklisted due to suspected abuse or a complaint"
 
-type instance MapError 'WhitelistError = 'StaticError 403 "unauthorized" "Unauthorized e-mail address or phone number."
+type instance MapError 'AllowlistError = 'StaticError 403 "unauthorized" "Unauthorized e-mail address or phone number."
 
 type instance MapError 'InvalidInvitationCode = 'StaticError 400 "invalid-invitation-code" "Invalid invitation code."
 

libs/wire-api/src/Wire/API/User.hs

@@ -519,7 +519,7 @@ instance Arbitrary NewUserPublic where
   arbitrary = arbitrary `QC.suchThatMap` (rightMay . validateNewUserPublic)
 
 data RegisterError
-  = RegisterErrorWhitelistError
+  = RegisterErrorAllowlistError
   | RegisterErrorInvalidInvitationCode
   | RegisterErrorMissingIdentity
   | RegisterErrorUserKeyExists
@@ -537,7 +537,7 @@ data RegisterError
 instance GSOP.Generic RegisterError
 
 type RegisterErrorResponses =
-  '[ ErrorResponse 'WhitelistError,
+  '[ ErrorResponse 'AllowlistError,
      ErrorResponse 'InvalidInvitationCode,
      ErrorResponse 'MissingIdentity,
      ErrorResponse 'UserKeyExists,

services/brig/brig.cabal

@@ -17,6 +17,7 @@ extra-source-files:
 library
   -- cabal-fmt: expand src
   exposed-modules:
+    Brig.Allowlists
     Brig.API
     Brig.API.Auth
     Brig.API.Client
@@ -130,7 +131,6 @@ library
     Brig.User.Search.TeamUserSearch
     Brig.User.Template
     Brig.Version
-    Brig.Whitelist
     Brig.ZAuth
 
   other-modules:      Paths_brig

services/brig/brig.integration.yaml

@@ -193,6 +193,10 @@ optSettings:
   setDpopTokenExpirationTimeSecs: 300 # 5 minutes
   setPublicKeyBundle: test/resources/jwt/ed25519_bundle.pem
   setEnableMLS: true
+  setAllowlistEmailDomains:
+    - wire.com
+  setAllowlistPhonePrefixes:
+    - +1555555
 
 logLevel: Warn
 logNetStrings: false

services/brig/src/Brig/API/Auth.hs

@@ -72,7 +72,7 @@ access mcid t mt =
 
 sendLoginCode :: SendLoginCode -> Handler r LoginCodeTimeout
 sendLoginCode (SendLoginCode phone call force) = do
-  checkWhitelist (Right phone)
+  checkAllowlist (Right phone)
   c <- wrapClientE (Auth.sendLoginCode phone call force) !>> sendLoginCodeError
   pure $ LoginCodeTimeout (pendingLoginTimeout c)
 

services/brig/src/Brig/API/Error.hs

@@ -295,8 +295,8 @@ activationCodeNotFound = invalidActivationCode "Activation key/code not found or
 deletionCodePending :: Wai.Error
 deletionCodePending = Wai.mkError status403 "pending-delete" "A verification code for account deletion is still pending."
 
-whitelistError :: Wai.Error
-whitelistError = Wai.mkError status403 "unauthorized" "Unauthorized e-mail address or phone number."
+allowlistError :: Wai.Error
+allowlistError = Wai.mkError status403 "unauthorized" "Unauthorized e-mail address or phone number."
 
 blacklistedEmail :: Wai.Error
 blacklistedEmail =

services/brig/src/Brig/API/Handler.hs

@@ -24,22 +24,22 @@ module Brig.API.Handler
     -- * Utilities
     JSON,
     parseJsonBody,
-    checkWhitelist,
-    checkWhitelistWithError,
-    isWhiteListed,
+    checkAllowlist,
+    checkAllowlistWithError,
+    isAllowlisted,
     UserNotAllowedToJoinTeam (..),
   )
 where
 
-import Bilge (MonadHttp, RequestId (..))
+import Bilge (RequestId (..))
 import Brig.API.Error
 import qualified Brig.AWS as AWS
+import qualified Brig.Allowlists as Allowlists
 import Brig.App
 import Brig.CanonicalInterpreter (BrigCanonicalEffects, runBrigToIO)
 import Brig.Email (Email)
-import Brig.Options (setWhitelist)
+import Brig.Options (setAllowlistEmailDomains, setAllowlistPhonePrefixes)
 import Brig.Phone (Phone, PhoneException (..))
-import qualified Brig.Whitelist as Whitelist
 import Control.Error
 import Control.Exception (throwIO)
 import Control.Lens (set, view)
@@ -167,18 +167,26 @@ type JSON = Media "application" "json"
 parseJsonBody :: (FromJSON a, MonadIO m) => JsonRequest a -> ExceptT Error m a
 parseJsonBody req = parseBody req !>> StdError . badRequest
 
--- | If a whitelist is configured, consult it, otherwise a no-op. {#RefActivationWhitelist}
-checkWhitelist :: Either Email Phone -> (Handler r) ()
-checkWhitelist = wrapHttpClientE . checkWhitelistWithError (StdError whitelistError)
+-- | If an Allowlist is configured, consult it, otherwise a no-op. {#RefActivationAllowlist}
+checkAllowlist :: Either Email Phone -> (Handler r) ()
+checkAllowlist = wrapHttpClientE . checkAllowlistWithError (StdError allowlistError)
 
-checkWhitelistWithError :: (MonadReader Env m, MonadIO m, Catch.MonadMask m, MonadHttp m, MonadError e m) => e -> Either Email Phone -> m ()
-checkWhitelistWithError e key = do
-  ok <- isWhiteListed key
+-- checkAllowlistWithError :: (MonadReader Env m, MonadIO m, Catch.MonadMask m, MonadHttp m, MonadError e m) => e -> Either Email Phone -> m ()
+checkAllowlistWithError :: (MonadReader Env m, MonadError e m) => e -> Either Email Phone -> m ()
+checkAllowlistWithError e key = do
+  ok <- isAllowlisted key
   unless ok (throwError e)
 
+isAllowlisted :: (MonadReader Env m) => Either Email Phone -> m Bool
+isAllowlisted key = do
+  env <- view settings
+  pure $ Allowlists.verify (setAllowlistEmailDomains env) (setAllowlistPhonePrefixes env) key
+
+{-
 isWhiteListed :: (MonadReader Env m, MonadIO m, Catch.MonadMask m, MonadHttp m) => Either Email Phone -> m Bool
 isWhiteListed key = do
-  eb <- setWhitelist <$> view settings
+  eb <- setAllowlist <$> view settings
   case eb of
     Nothing -> pure True
-    Just b -> Whitelist.verify b key
+    Just b -> Allowlist.verify b key
+-}

services/brig/src/Brig/API/Public.hs

@@ -129,7 +129,7 @@ import Wire.API.SwaggerHelper (cleanupSwagger)
 import Wire.API.SystemSettings
 import qualified Wire.API.Team as Public
 import Wire.API.Team.LegalHold (LegalholdProtectee (..))
-import Wire.API.User (RegisterError (RegisterErrorWhitelistError))
+import Wire.API.User (RegisterError (RegisterErrorAllowlistError))
 import qualified Wire.API.User as Public
 import qualified Wire.API.User.Activation as Public
 import qualified Wire.API.User.Auth as Public
@@ -597,8 +597,8 @@ createUser ::
   (Handler r) (Either Public.RegisterError Public.RegisterSuccess)
 createUser (Public.NewUserPublic new) = lift . runExceptT $ do
   API.checkRestrictedUserCreation new
-  for_ (Public.newUserEmail new) $ mapExceptT wrapHttp . checkWhitelistWithError RegisterErrorWhitelistError . Left
-  for_ (Public.newUserPhone new) $ mapExceptT wrapHttp . checkWhitelistWithError RegisterErrorWhitelistError . Right
+  for_ (Public.newUserEmail new) $ mapExceptT wrapHttp . checkAllowlistWithError RegisterErrorAllowlistError . Left
+  for_ (Public.newUserPhone new) $ mapExceptT wrapHttp . checkAllowlistWithError RegisterErrorAllowlistError . Right
   result <- API.createUser new
   let acc = createdAccount result
 
@@ -822,7 +822,7 @@ beginPasswordReset ::
   Public.NewPasswordReset ->
   (Handler r) ()
 beginPasswordReset (Public.NewPasswordReset target) = do
-  checkWhitelist target
+  checkAllowlist target
   (u, pair) <- API.beginPasswordReset target !>> pwResetError
   loc <- lift $ wrapClient $ API.lookupLocale u
   lift $ case target of
@@ -849,7 +849,7 @@ sendActivationCode ::
   (Handler r) ()
 sendActivationCode Public.SendActivationCode {..} = do
   either customerExtensionCheckBlockedDomains (const $ pure ()) saUserKey
-  checkWhitelist saUserKey
+  checkAllowlist saUserKey
   API.sendActivationCode saUserKey saLocale saCall !>> sendActCodeError
 
 -- | If the user presents an email address from a blocked domain, throw an error.

services/brig/src/Brig/Allowlists.hs

@@ -0,0 +1,50 @@
+-- This file is part of the Wire Server implementation.
+--
+-- Copyright (C) 2022 Wire Swiss GmbH <opensource@wire.com>
+--
+-- This program is free software: you can redistribute it and/or modify it under
+-- the terms of the GNU Affero General Public License as published by the Free
+-- Software Foundation, either version 3 of the License, or (at your option) any
+-- later version.
+--
+-- This program is distributed in the hope that it will be useful, but WITHOUT
+-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+-- FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+-- details.
+--
+-- You should have received a copy of the GNU Affero General Public License along
+-- with this program. If not, see <https://www.gnu.org/licenses/>.
+
+-- | > docs/reference/user/activation.md {#RefActivationAllowlist}
+--
+-- Email/phone whitelist.
+module Brig.Allowlists
+  ( AllowlistEmailDomains (..),
+    AllowlistPhonePrefixes (..),
+    verify,
+  )
+where
+
+import Data.Aeson
+import qualified Data.Text as Text
+import Imports
+import Wire.API.User.Identity
+
+-- | A service providing a whitelist of allowed email addresses and phone numbers
+data AllowlistEmailDomains = AllowlistEmailDomains [Text]
+  deriving (Show, Generic)
+
+instance FromJSON AllowlistEmailDomains
+
+data AllowlistPhonePrefixes = AllowlistPhonePrefixes [Text]
+  deriving (Show, Generic)
+
+instance FromJSON AllowlistPhonePrefixes
+
+-- | Consult the whitelist settings in brig's config file and verify that the provided
+-- email/phone address is whitelisted.
+verify :: Maybe AllowlistEmailDomains -> Maybe AllowlistPhonePrefixes -> Either Email Phone -> Bool
+verify (Just (AllowlistEmailDomains allowed)) _ (Left email) = emailDomain email `elem` allowed
+verify _ (Just (AllowlistPhonePrefixes allowed)) (Right phone) = any (`Text.isPrefixOf` fromPhone phone) allowed
+verify Nothing _ (Left _) = True
+verify _ Nothing (Right _) = True