Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discover and expose SFT servers in /calls/config/v2 #1177

Merged
merged 48 commits into from
Jul 31, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
14b3692
Add comment, minor hlint
jschaul Jul 21, 2020
189043f
Add FUTUREWORK to ensure we watch the threads we spawn
akshaymankar Jul 23, 2020
27fc2e8
Copy code for SRV lookup from federation-util to dedicated package
akshaymankar Jul 24, 2020
f9de188
Read SFT options and start service discovery thread
akshaymankar Jul 24, 2020
07f848a
[WIP] Return discovered SFT servers in /calls/config/v2
akshaymankar Jul 27, 2020
a345016
Fix typo
akshaymankar Jul 28, 2020
8a5f363
Add unit tests for sftDiscoveryLoop
akshaymankar Jul 28, 2020
a0d7fe5
rename Wire.API.TURN -> Wire.API.Calling
jschaul Jul 28, 2020
eaaa4f3
second rename TURN -> Calling
jschaul Jul 28, 2020
7101bd9
merge TURN into Calling
jschaul Jul 28, 2020
fb42bc3
another TURN -> Calling rename
jschaul Jul 28, 2020
1b5804d
rename dns-srv to dns-util
jschaul Jul 28, 2020
b6c81ad
Rename testGroup "turn" -> "calling"
akshaymankar Jul 29, 2020
c00e688
Ormolu for module rename
akshaymankar Jul 29, 2020
a7820b5
Remove dependency on polysemy-plugin so we can compile on alpine
akshaymankar Jul 29, 2020
6fcb23f
Add unit test for SrvTarget -> SFTServer conversion
akshaymankar Jul 29, 2020
28d9971
Deflake brig calling unit tests
akshaymankar Jul 29, 2020
05145d1
Make SFT discovery interval configurable
akshaymankar Jul 29, 2020
46df70e
Fix brig's API.Calling integration test
akshaymankar Jul 29, 2020
1b1a85f
Give up on avoiding Embed IO for threadDelay
akshaymankar Jul 29, 2020
b51205f
Add comment to explain integration-test dns names
akshaymankar Jul 29, 2020
a4e51da
WIP: Move orderSrvEntry to dns-util
akshaymankar Jul 29, 2020
f4d5282
finish copying to dns-util
jschaul Jul 29, 2020
3327d9f
remove federation-util
jschaul Jul 29, 2020
c7a8150
dns-util: remove unnecessary comment
akshaymankar Jul 29, 2020
bbe9458
Abandon QQ because stack does something bad
akshaymankar Jul 29, 2020
096a8e5
add LICENSE file that was forgotten when moving
jschaul Jul 29, 2020
271f648
remove unnecessary comment
jschaul Jul 29, 2020
42d6584
Log whenever sft service discovery fails or returns empty
akshaymankar Jul 29, 2020
fde6771
move defaults to Options.hs
jschaul Jul 29, 2020
009961d
randomize sft URLs
jschaul Jul 29, 2020
b554390
Commit the forgotten Brig.PolyLog module
akshaymankar Jul 29, 2020
7525503
Fix compile errors
akshaymankar Jul 30, 2020
7edee6a
Simplify DNSLookup interpretation
akshaymankar Jul 30, 2020
959adfa
Cleanup async thread creation for SFT discovery
akshaymankar Jul 30, 2020
a48689c
Remove leftover TODO comment
akshaymankar Jul 30, 2020
7ec72b1
Add a safe way to create `HttpsUrl`
akshaymankar Jul 30, 2020
c4a6a9a
Replace maybe with another type to clarify semantics of discovery
akshaymankar Jul 30, 2020
2f7c0fb
Group TURN tests so they can be skipped by CI as they were
akshaymankar Jul 30, 2020
b5e26ae
Put SFT test in a testGroup for symmetry
akshaymankar Jul 30, 2020
939c802
brig/integration API.Calling: Slightly better formatting
akshaymankar Jul 30, 2020
2c3ef08
Add FUTUREWORK to extract a function
akshaymankar Jul 30, 2020
f54e611
Add roundtrip test for Call.Config.SFTServer
akshaymankar Jul 30, 2020
4f0d7d2
More accurate doc for SFTServer
akshaymankar Jul 30, 2020
b7b92d8
Add link to issue for FUTUREWORK
akshaymankar Jul 30, 2020
3929ea6
Pointfree sft discovery async call
akshaymankar Jul 30, 2020
1f176f4
Add deprecation notice to /calls/config swagger docs
akshaymankar Jul 30, 2020
55cb7b0
Brig.Calling.Internal: Declutter sftServerFromSrvTarget
akshaymankar Jul 30, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions libs/brig-types/brig-types.cabal
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
cabal-version: 1.12

-- This file has been generated from package.yaml by hpack version 0.31.2.
-- This file has been generated from package.yaml by hpack version 0.33.0.
--
-- see: https://github.com/sol/hpack
--
-- hash: 7497d04521f12339e2a8f5537dacf242839dea9034f69e67f0a254b1548cadd9
-- hash: fe16e52e870cb548573366fe9192319004e52c3e4d2ece172df69e408cbe391b

name: brig-types
version: 1.35.0
Expand All @@ -21,6 +21,7 @@ library
exposed-modules:
Brig.Types
Brig.Types.Activation
Brig.Types.Calling
Brig.Types.Client
Brig.Types.Client.Prekey
Brig.Types.Code
Expand All @@ -37,7 +38,6 @@ library
Brig.Types.Team.Invitation
Brig.Types.Team.LegalHold
Brig.Types.Test.Arbitrary
Brig.Types.TURN
Brig.Types.User
Brig.Types.User.Auth
other-modules:
Expand Down
2 changes: 1 addition & 1 deletion libs/brig-types/src/Brig/Types.hs
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ module Brig.Types
where

import Brig.Types.Activation as M
import Brig.Types.Calling as M
import Brig.Types.Client as M
import Brig.Types.Connection as M
import Brig.Types.Properties as M
import Brig.Types.Search as M
import Brig.Types.TURN as M
import Brig.Types.Team as M
import Brig.Types.User as M
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
-- You should have received a copy of the GNU Affero General Public License along
-- with this program. If not, see <https://www.gnu.org/licenses/>.

module Brig.Types.TURN
module Brig.Types.Calling
( -- * re-exports
RTCConfiguration,
rtcConfiguration,
Expand Down Expand Up @@ -50,4 +50,4 @@ module Brig.Types.TURN
)
where

import Wire.API.Call.TURN
import Wire.API.Call.Config
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
cabal-version: 1.12

-- This file has been generated from package.yaml by hpack version 0.31.2.
-- This file has been generated from package.yaml by hpack version 0.33.0.
--
-- see: https://github.com/sol/hpack
--
-- hash: d327ef72460d5f79332d80fa4a70516b7351472f50c17a8c491d28c65ec0f024
-- hash: 82f20a0525faea5f899c0a68fdc8a82623913d8b063f34a6cd4c6e32aa6acf54

name: federation-util
name: dns-util
version: 0.1.0
synopsis: Various helpers for federation
description: Small helper functions useful when federating.
category: Web
synopsis: Library to deal with DNS SRV records
description: Library to deal with DNS SRV records
category: Network
author: Wire Swiss GmbH
maintainer: Wire Swiss GmbH <backend@wire.com>
copyright: (c) 2020 Wire Swiss GmbH
Expand All @@ -20,66 +20,44 @@ build-type: Simple

library
exposed-modules:
Network.Federation.Util
Network.Federation.Util.DNS
Network.Federation.Util.Internal
Wire.Network.DNS.Effect
Wire.Network.DNS.SRV
other-modules:
Paths_federation_util
Paths_dns_util
hs-source-dirs:
src
default-extensions: AllowAmbiguousTypes BangPatterns ConstraintKinds DataKinds DefaultSignatures DerivingStrategies DeriveFunctor DeriveGeneric DeriveLift DeriveTraversable EmptyCase FlexibleContexts FlexibleInstances FunctionalDependencies GADTs InstanceSigs KindSignatures LambdaCase MultiParamTypeClasses MultiWayIf NamedFieldPuns NoImplicitPrelude OverloadedStrings PackageImports PatternSynonyms PolyKinds QuasiQuotes RankNTypes ScopedTypeVariables StandaloneDeriving TemplateHaskell TupleSections TypeApplications TypeFamilies TypeFamilyDependencies TypeOperators UndecidableInstances ViewPatterns
ghc-options: -O2 -Wall -Wincomplete-uni-patterns -Wincomplete-record-updates -Wpartial-fields -fwarn-tabs -optP-Wno-nonportable-include-path
build-depends:
async >=2.0
, base >=4.6 && <5.0
, bytestring >=0.10
, bytestring-conversion >=0.3
, containers >=0.5
base >=4.6 && <5.0
, dns
, errors >=2.0
, exceptions >=0.6
, http-types >=0.8
, imports
, polysemy
, random
, stm
, streaming-commons >=0.1
, string-conversions
, text >=0.11
, tinylog >=0.8
, transformers >=0.3
default-language: Haskell2010

test-suite spec
type: exitcode-stdio-1.0
main-is: Spec.hs
other-modules:
Test.DNSSpec
Paths_federation_util
Test.Wire.Network.DNS.SRVSpec
Paths_dns_util
hs-source-dirs:
test
default-extensions: AllowAmbiguousTypes BangPatterns ConstraintKinds DataKinds DefaultSignatures DerivingStrategies DeriveFunctor DeriveGeneric DeriveLift DeriveTraversable EmptyCase FlexibleContexts FlexibleInstances FunctionalDependencies GADTs InstanceSigs KindSignatures LambdaCase MultiParamTypeClasses MultiWayIf NamedFieldPuns NoImplicitPrelude OverloadedStrings PackageImports PatternSynonyms PolyKinds QuasiQuotes RankNTypes ScopedTypeVariables StandaloneDeriving TemplateHaskell TupleSections TypeApplications TypeFamilies TypeFamilyDependencies TypeOperators UndecidableInstances ViewPatterns
ghc-options: -O2 -Wall -Wincomplete-uni-patterns -Wincomplete-record-updates -Wpartial-fields -fwarn-tabs -optP-Wno-nonportable-include-path -threaded -rtsopts -with-rtsopts=-N
build-tool-depends:
hspec-discover:hspec-discover
build-depends:
QuickCheck
, async >=2.0
, base >=4.6 && <5.0
, bytestring >=0.10
, bytestring-conversion >=0.3
, containers >=0.5
, dns
, errors >=2.0
, exceptions >=0.6
, federation-util
, dns-util
, hspec
, hspec-discover
, http-types >=0.8
, imports
, polysemy
, random
, stm
, streaming-commons >=0.1
, string-conversions
, text >=0.11
, tinylog >=0.8
, transformers >=0.3
, uri-bytestring
default-language: Haskell2010
29 changes: 9 additions & 20 deletions libs/federation-util/package.yaml → libs/dns-util/package.yaml
Original file line number Diff line number Diff line change
@@ -1,32 +1,21 @@
defaults:
local: ../../package-defaults.yaml
name: federation-util
name: dns-util
version: '0.1.0'
synopsis: Various helpers for federation
description: Small helper functions useful when federating.
category: Web
synopsis: Library to deal with DNS SRV records
description: Library to deal with DNS SRV records
category: Network
author: Wire Swiss GmbH
maintainer: Wire Swiss GmbH <backend@wire.com>
copyright: (c) 2020 Wire Swiss GmbH
license: AGPL-3
dependencies:
- async >=2.0
- base >=4.6 && <5.0
- bytestring >=0.10
- bytestring-conversion >=0.3
- containers >=0.5
- errors >=2.0
- exceptions >=0.6
- http-types >=0.8
- imports
- dns
- random
- streaming-commons >=0.1
- string-conversions
- stm
- text >=0.11
- transformers >=0.3
- tinylog >=0.8
- imports
- polysemy
library:
source-dirs: src

Expand All @@ -36,10 +25,10 @@ tests:
source-dirs:
- test
ghc-options: -threaded -rtsopts -with-rtsopts=-N
build-tools:
- hspec-discover:hspec-discover
dependencies:
- hspec
- hspec-discover
- QuickCheck
- federation-util
- uri-bytestring

- dns-util
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,22 @@
-- You should have received a copy of the GNU Affero General Public License along
-- with this program. If not, see <https://www.gnu.org/licenses/>.

module Brig.TURN where
module Wire.Network.DNS.Effect where

import Brig.Types (TurnURI)
import Control.Lens
import Data.List1
import Imports
import OpenSSL.EVP.Digest (Digest)
import System.Random.MWC (GenIO, createSystemRandom)
import Network.DNS (Domain)
import qualified Network.DNS as DNS
import Polysemy
import Wire.Network.DNS.SRV

data Env = Env
{ _turnServers :: List1 TurnURI,
_turnTokenTTL :: Word32,
_turnConfigTTL :: Word32,
_turnSecret :: ByteString,
_turnSHA512 :: Digest,
_turnPrng :: GenIO
}
data DNSLookup m a where
LookupSRV :: Domain -> DNSLookup m SrvResponse

makeLenses ''Env
makeSem ''DNSLookup

newEnv :: Digest -> List1 TurnURI -> Word32 -> Word32 -> ByteString -> IO Env
newEnv sha512 srvs tTTL cTTL secret = Env srvs tTTL cTTL secret sha512 <$> createSystemRandom
runDNSLookupDefault :: Member (Embed IO) r => Sem (DNSLookup ': r) a -> Sem r a
runDNSLookupDefault =
interpret $ \(LookupSRV domain) -> embed $ do
rs <- DNS.makeResolvSeed DNS.defaultResolvConf
DNS.withResolver rs $ \resolver ->
interpretResponse <$> DNS.lookupSRV resolver domain
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
-- You should have received a copy of the GNU Affero General Public License along
-- with this program. If not, see <https://www.gnu.org/licenses/>.

-- Parts of this code, namely functions srvLookup'' and orderSrvResult,
-- Parts of this code, namely functions interpretResponse and orderSrvResult,
-- which were taken from http://hackage.haskell.org/package/pontarius-xmpp
-- are also licensed under the three-clause BSD license:
--
Expand Down Expand Up @@ -55,12 +55,12 @@
-- LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
-- OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

module Network.Federation.Util.Internal where
module Wire.Network.DNS.SRV where

import Control.Category ((>>>))
import Data.Text.Encoding (encodeUtf8)
import Data.List.NonEmpty (NonEmpty (..))
import Imports
import Network.DNS (DNSError, Domain, ResolvSeed, Resolver, lookupSRV, withResolver)
import Network.DNS (DNSError, Domain)
import System.Random (randomRIO)

data SrvEntry = SrvEntry
Expand All @@ -78,37 +78,21 @@ data SrvTarget = SrvTarget
}
deriving (Eq, Show)

toSrvEntry :: (Word16, Word16, Word16, Domain) -> SrvEntry
toSrvEntry (prio, weight, port, domain) = SrvEntry prio weight (SrvTarget domain port)
data SrvResponse
= SrvNotAvailable
| SrvAvailable (NonEmpty SrvEntry)
| SrvResponseError DNSError
deriving (Eq, Show)

-- Given a prefix (e.g. _wire-server) and a domain (e.g. wire.com),
-- provides a list of A(AAA) names and port numbers upon a successful
-- DNS-SRV request, or `Nothing' if the DNS-SRV request failed.
-- Modified version inspired from http://hackage.haskell.org/package/pontarius-xmpp
srvLookup' :: Text -> Text -> ResolvSeed -> IO (Maybe [SrvTarget])
srvLookup' = srvLookup'' lookupSRV
interpretResponse :: Either DNSError [(Word16, Word16, Word16, Domain)] -> SrvResponse
interpretResponse = \case
Left err -> SrvResponseError err
Right [] -> SrvNotAvailable
Right [(_, _, _, ".")] -> SrvNotAvailable -- According to RFC2782
Right (r : rs) -> SrvAvailable $ fmap toSrvEntry (r :| rs)

-- internal version for testing
--
-- FUTUREWORK: return more precise errors than 'Nothing'?
srvLookup'' ::
(Resolver -> Domain -> IO (Either DNSError [(Word16, Word16, Word16, Domain)])) ->
Text ->
Text ->
ResolvSeed ->
IO (Maybe [SrvTarget])
srvLookup'' lookupF prefix realm resolvSeed = withResolver resolvSeed $ \resolver -> do
srvResult <- lookupF resolver $ encodeUtf8 $ prefix <> "._tcp." <> realm <> "."
case srvResult of
-- The service is not available at this domain.
Left _ -> return Nothing
Right [] -> return Nothing
Right [(_, _, _, ".")] -> return Nothing -- "not available" as in RFC2782
Right srvResult' -> do
let srvEntries = toSrvEntry <$> srvResult'
-- Get [(Domain, PortNumber)] of SRV request, if any.
-- Sorts the records based on the priority value.
Just . fmap srvTarget <$> orderSrvResult srvEntries
toSrvEntry :: (Word16, Word16, Word16, Domain) -> SrvEntry
toSrvEntry (prio, weight, port, domain) = SrvEntry prio weight (SrvTarget domain port)

-- FUTUREWORK: maybe improve sorting algorithm here? (with respect to performance and code style)
--
Expand All @@ -121,7 +105,7 @@ srvLookup'' lookupF prefix realm resolvSeed = withResolver resolvSeed $ \resolve
orderSrvResult :: [SrvEntry] -> IO [SrvEntry]
orderSrvResult =
-- Order the result set by priority.
sortBy (comparing srvPriority)
sortOn srvPriority
-- Group elements in sublists based on their priority.
-- The result type is `[[(Word16, Word16, Word16, Domain)]]' (nested list).
>>> groupBy ((==) `on` srvPriority)
Expand All @@ -148,7 +132,7 @@ orderSrvResult =
(b, (c : e)) -> (b, c, e)
_ -> error "orderSrvResult: no record with running sum greater than random number"
-- Remove the running total number from the remaining elements.
let remainingSrvs = map (\(srv, _) -> srv) (concat [beginning, end])
let remainingSrvs = map fst (concat [beginning, end])
-- Repeat the ordering procedure on the remaining elements.
rest <- orderSublist remainingSrvs
return $ firstSrv : rest
File renamed without changes.
Loading