Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discover and expose SFT servers in /calls/config/v2 #1177

Merged
merged 48 commits into from
Jul 31, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
14b3692
Add comment, minor hlint
jschaul Jul 21, 2020
189043f
Add FUTUREWORK to ensure we watch the threads we spawn
akshaymankar Jul 23, 2020
27fc2e8
Copy code for SRV lookup from federation-util to dedicated package
akshaymankar Jul 24, 2020
f9de188
Read SFT options and start service discovery thread
akshaymankar Jul 24, 2020
07f848a
[WIP] Return discovered SFT servers in /calls/config/v2
akshaymankar Jul 27, 2020
a345016
Fix typo
akshaymankar Jul 28, 2020
8a5f363
Add unit tests for sftDiscoveryLoop
akshaymankar Jul 28, 2020
a0d7fe5
rename Wire.API.TURN -> Wire.API.Calling
jschaul Jul 28, 2020
eaaa4f3
second rename TURN -> Calling
jschaul Jul 28, 2020
7101bd9
merge TURN into Calling
jschaul Jul 28, 2020
fb42bc3
another TURN -> Calling rename
jschaul Jul 28, 2020
1b5804d
rename dns-srv to dns-util
jschaul Jul 28, 2020
b6c81ad
Rename testGroup "turn" -> "calling"
akshaymankar Jul 29, 2020
c00e688
Ormolu for module rename
akshaymankar Jul 29, 2020
a7820b5
Remove dependency on polysemy-plugin so we can compile on alpine
akshaymankar Jul 29, 2020
6fcb23f
Add unit test for SrvTarget -> SFTServer conversion
akshaymankar Jul 29, 2020
28d9971
Deflake brig calling unit tests
akshaymankar Jul 29, 2020
05145d1
Make SFT discovery interval configurable
akshaymankar Jul 29, 2020
46df70e
Fix brig's API.Calling integration test
akshaymankar Jul 29, 2020
1b1a85f
Give up on avoiding Embed IO for threadDelay
akshaymankar Jul 29, 2020
b51205f
Add comment to explain integration-test dns names
akshaymankar Jul 29, 2020
a4e51da
WIP: Move orderSrvEntry to dns-util
akshaymankar Jul 29, 2020
f4d5282
finish copying to dns-util
jschaul Jul 29, 2020
3327d9f
remove federation-util
jschaul Jul 29, 2020
c7a8150
dns-util: remove unnecessary comment
akshaymankar Jul 29, 2020
bbe9458
Abandon QQ because stack does something bad
akshaymankar Jul 29, 2020
096a8e5
add LICENSE file that was forgotten when moving
jschaul Jul 29, 2020
271f648
remove unnecessary comment
jschaul Jul 29, 2020
42d6584
Log whenever sft service discovery fails or returns empty
akshaymankar Jul 29, 2020
fde6771
move defaults to Options.hs
jschaul Jul 29, 2020
009961d
randomize sft URLs
jschaul Jul 29, 2020
b554390
Commit the forgotten Brig.PolyLog module
akshaymankar Jul 29, 2020
7525503
Fix compile errors
akshaymankar Jul 30, 2020
7edee6a
Simplify DNSLookup interpretation
akshaymankar Jul 30, 2020
959adfa
Cleanup async thread creation for SFT discovery
akshaymankar Jul 30, 2020
a48689c
Remove leftover TODO comment
akshaymankar Jul 30, 2020
7ec72b1
Add a safe way to create `HttpsUrl`
akshaymankar Jul 30, 2020
c4a6a9a
Replace maybe with another type to clarify semantics of discovery
akshaymankar Jul 30, 2020
2f7c0fb
Group TURN tests so they can be skipped by CI as they were
akshaymankar Jul 30, 2020
b5e26ae
Put SFT test in a testGroup for symmetry
akshaymankar Jul 30, 2020
939c802
brig/integration API.Calling: Slightly better formatting
akshaymankar Jul 30, 2020
2c3ef08
Add FUTUREWORK to extract a function
akshaymankar Jul 30, 2020
f54e611
Add roundtrip test for Call.Config.SFTServer
akshaymankar Jul 30, 2020
4f0d7d2
More accurate doc for SFTServer
akshaymankar Jul 30, 2020
b7b92d8
Add link to issue for FUTUREWORK
akshaymankar Jul 30, 2020
3929ea6
Pointfree sft discovery async call
akshaymankar Jul 30, 2020
1f176f4
Add deprecation notice to /calls/config swagger docs
akshaymankar Jul 30, 2020
55cb7b0
Brig.Calling.Internal: Declutter sftServerFromSrvTarget
akshaymankar Jul 30, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions libs/dns-srv/dns-srv.cabal
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
cabal-version: 1.12

-- This file has been generated from package.yaml by hpack version 0.33.0.
--
-- see: https://github.com/sol/hpack
--
-- hash: 2b22f84e0da77b424651866179a468047b326b1b1d904a05a1a9fb9b06e26bfe

name: dns-srv
version: 0.1.0
synopsis: Library to deal with DNS SRV records
description: Library to deal with DNS SRV records
category: Network
author: Wire Swiss GmbH
maintainer: Wire Swiss GmbH <backend@wire.com>
copyright: (c) 2020 Wire Swiss GmbH
license: AGPL-3
build-type: Simple

library
exposed-modules:
Wire.Network.DNS.Effect
Wire.Network.DNS.SRV
other-modules:
Paths_dns_srv
hs-source-dirs:
src
default-extensions: AllowAmbiguousTypes BangPatterns ConstraintKinds DataKinds DefaultSignatures DerivingStrategies DeriveFunctor DeriveGeneric DeriveLift DeriveTraversable EmptyCase FlexibleContexts FlexibleInstances FunctionalDependencies GADTs InstanceSigs KindSignatures LambdaCase MultiParamTypeClasses MultiWayIf NamedFieldPuns NoImplicitPrelude OverloadedStrings PackageImports PatternSynonyms PolyKinds QuasiQuotes RankNTypes ScopedTypeVariables StandaloneDeriving TemplateHaskell TupleSections TypeApplications TypeFamilies TypeFamilyDependencies TypeOperators UndecidableInstances ViewPatterns
ghc-options: -O2 -Wall -Wincomplete-uni-patterns -Wincomplete-record-updates -Wpartial-fields -fwarn-tabs -optP-Wno-nonportable-include-path
build-depends:
base >=4.6 && <5.0
, dns
, imports
, polysemy
, random
, text >=0.11
default-language: Haskell2010

test-suite spec
type: exitcode-stdio-1.0
main-is: Spec.hs
other-modules:
Test.Wire.Network.DNS.SRVSpec
Paths_dns_srv
hs-source-dirs:
test
default-extensions: AllowAmbiguousTypes BangPatterns ConstraintKinds DataKinds DefaultSignatures DerivingStrategies DeriveFunctor DeriveGeneric DeriveLift DeriveTraversable EmptyCase FlexibleContexts FlexibleInstances FunctionalDependencies GADTs InstanceSigs KindSignatures LambdaCase MultiParamTypeClasses MultiWayIf NamedFieldPuns NoImplicitPrelude OverloadedStrings PackageImports PatternSynonyms PolyKinds QuasiQuotes RankNTypes ScopedTypeVariables StandaloneDeriving TemplateHaskell TupleSections TypeApplications TypeFamilies TypeFamilyDependencies TypeOperators UndecidableInstances ViewPatterns
ghc-options: -O2 -Wall -Wincomplete-uni-patterns -Wincomplete-record-updates -Wpartial-fields -fwarn-tabs -optP-Wno-nonportable-include-path -threaded -rtsopts -with-rtsopts=-N
build-tool-depends:
hspec-discover:hspec-discover
build-depends:
QuickCheck
, base >=4.6 && <5.0
, dns
, dns-srv
, hspec
, hspec-discover
, imports
, polysemy
, random
, text >=0.11
default-language: Haskell2010
34 changes: 34 additions & 0 deletions libs/dns-srv/package.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
defaults:
local: ../../package-defaults.yaml
name: dns-srv
version: '0.1.0'
synopsis: Library to deal with DNS SRV records
description: Library to deal with DNS SRV records
category: Network
author: Wire Swiss GmbH
maintainer: Wire Swiss GmbH <backend@wire.com>
copyright: (c) 2020 Wire Swiss GmbH
license: AGPL-3
dependencies:
- base >=4.6 && <5.0
- dns
- random
- text >=0.11
- imports
- polysemy
library:
source-dirs: src

tests:
spec:
main: Spec.hs
source-dirs:
- test
ghc-options: -threaded -rtsopts -with-rtsopts=-N
build-tools:
- hspec-discover:hspec-discover
dependencies:
- hspec
- hspec-discover
- QuickCheck
- dns-srv
41 changes: 41 additions & 0 deletions libs/dns-srv/src/Wire/Network/DNS/Effect.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
-- This file is part of the Wire Server implementation.
--
-- Copyright (C) 2020 Wire Swiss GmbH <opensource@wire.com>
--
-- This program is free software: you can redistribute it and/or modify it under
-- the terms of the GNU Affero General Public License as published by the Free
-- Software Foundation, either version 3 of the License, or (at your option) any
-- later version.
--
-- This program is distributed in the hope that it will be useful, but WITHOUT
-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
-- FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
-- details.
--
-- You should have received a copy of the GNU Affero General Public License along
-- with this program. If not, see <https://www.gnu.org/licenses/>.

module Wire.Network.DNS.Effect where
akshaymankar marked this conversation as resolved.
Show resolved Hide resolved
akshaymankar marked this conversation as resolved.
Show resolved Hide resolved

import Imports
import Network.DNS (Domain)
import qualified Network.DNS as DNS
import Polysemy
import Wire.Network.DNS.SRV

data DNSLookup m a where
LookupSRV :: Domain -> DNSLookup m SrvResponse

makeSem ''DNSLookup

runDNSLookupDefault :: Member (Embed IO) r => Sem (DNSLookup ': r) a -> Sem r a
runDNSLookupDefault =
interpret $ \l -> do
rs <- embed $ DNS.makeResolvSeed DNS.defaultResolvConf
embed $ DNS.withResolver rs $ \resolver ->
case l of
LookupSRV domain -> interpretResponse <$> DNS.lookupSRV resolver domain
akshaymankar marked this conversation as resolved.
Show resolved Hide resolved

-- class Monad m => MonadDNSLookup m where
-- monadLookupSRV :: Domain -> m SrvResponse
-- I don't know how to make this choice at runtime
akshaymankar marked this conversation as resolved.
Show resolved Hide resolved
53 changes: 53 additions & 0 deletions libs/dns-srv/src/Wire/Network/DNS/SRV.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
-- This file is part of the Wire Server implementation.
--
-- Copyright (C) 2020 Wire Swiss GmbH <opensource@wire.com>
--
-- This program is free software: you can redistribute it and/or modify it under
-- the terms of the GNU Affero General Public License as published by the Free
-- Software Foundation, either version 3 of the License, or (at your option) any
-- later version.
--
-- This program is distributed in the hope that it will be useful, but WITHOUT
-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
-- FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
-- details.
--
-- You should have received a copy of the GNU Affero General Public License along
-- with this program. If not, see <https://www.gnu.org/licenses/>.

module Wire.Network.DNS.SRV where
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, the types here follow the RFC that describes SFT server lookup, or maybe something slightly more general, but more specific than DNS. Could we link that RFC here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RFC defines what SRV records are. The dns package interprets an SRV record into a tuple of 4 things, we named them here for it to be easier to comprehend across the code. Not sure if the mention is worth it. We mention the RFC in the ordering function because it might not be as obvious.


import Data.List.NonEmpty
import Imports
import Network.DNS (DNSError, Domain)

data SrvEntry = SrvEntry
{ srvPriority :: !Word16,
srvWeight :: !Word16,
srvTarget :: !SrvTarget
}
deriving (Eq, Show)

data SrvTarget = SrvTarget
{ -- | the hostname on which the service is offered
srvTargetDomain :: !Domain,
-- | the port on which the service is offered
srvTargetPort :: !Word16
}
deriving (Eq, Show)

data SrvResponse
= SrvNotAvailable
| SrvAvailable (NonEmpty SrvEntry)
| SrvResponseError DNSError
deriving (Eq, Show)

interpretResponse :: Either DNSError [(Word16, Word16, Word16, Domain)] -> SrvResponse
interpretResponse = \case
Left err -> SrvResponseError err
Right [] -> SrvNotAvailable
Right [(_, _, _, ".")] -> SrvNotAvailable -- According to RFC2782
Right (r : rs) -> SrvAvailable $ fmap toSrvEntry (r :| rs)
akshaymankar marked this conversation as resolved.
Show resolved Hide resolved
where
toSrvEntry :: (Word16, Word16, Word16, Domain) -> SrvEntry
toSrvEntry (prio, weight, port, domain) = SrvEntry prio weight (SrvTarget domain port)
18 changes: 18 additions & 0 deletions libs/dns-srv/test/Spec.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{-# OPTIONS_GHC -F -pgmF hspec-discover #-}

-- This file is part of the Wire Server implementation.
--
-- Copyright (C) 2020 Wire Swiss GmbH <opensource@wire.com>
--
-- This program is free software: you can redistribute it and/or modify it under
-- the terms of the GNU Affero General Public License as published by the Free
-- Software Foundation, either version 3 of the License, or (at your option) any
-- later version.
--
-- This program is distributed in the hope that it will be useful, but WITHOUT
-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
-- FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
-- details.
--
-- You should have received a copy of the GNU Affero General Public License along
-- with this program. If not, see <https://www.gnu.org/licenses/>.
47 changes: 47 additions & 0 deletions libs/dns-srv/test/Test/Wire/Network/DNS/SRVSpec.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
-- This file is part of the Wire Server implementation.
--
-- Copyright (C) 2020 Wire Swiss GmbH <opensource@wire.com>
--
-- This program is free software: you can redistribute it and/or modify it under
-- the terms of the GNU Affero General Public License as published by the Free
-- Software Foundation, either version 3 of the License, or (at your option) any
-- later version.
--
-- This program is distributed in the hope that it will be useful, but WITHOUT
-- ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
-- FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
-- details.
--
-- You should have received a copy of the GNU Affero General Public License along
-- with this program. If not, see <https://www.gnu.org/licenses/>.

module Test.Wire.Network.DNS.SRVSpec where

import Data.List.NonEmpty
import Imports
import qualified Network.DNS as DNS
import Test.Hspec
import Wire.Network.DNS.SRV

spec :: Spec
spec = describe "interpretResponse" $ do
it "should interpret error correctly" $
interpretResponse (Left DNS.UnknownDNSError) `shouldBe` SrvResponseError DNS.UnknownDNSError

it "should interpret empty response as SrvNotAvailable" $
interpretResponse (Right []) `shouldBe` SrvNotAvailable

it "should interpret explicitly not available response as SrvNotAvailable" $
interpretResponse (Right [(0, 0, 0, ".")]) `shouldBe` SrvNotAvailable

it "should interpret an available service correctly" $ do
let input =
[ (0, 1, 443, "service01.example.com."),
(10, 20, 8443, "service02.example.com.")
]
let expectedOutput =
SrvAvailable
( SrvEntry 0 1 (SrvTarget "service01.example.com." 443)
:| [SrvEntry 10 20 (SrvTarget "service02.example.com." 8443)]
)
interpretResponse (Right input) `shouldBe` expectedOutput
5 changes: 5 additions & 0 deletions stack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ packages:
- libs/cassandra-util
- libs/extended
- libs/federation-util
- libs/dns-srv
- libs/galley-types
- libs/gundeck-types
- libs/hscim
Expand Down Expand Up @@ -170,6 +171,10 @@ extra-deps:
- QuickCheck-2.14
- splitmix-0.0.4 # needed for QuickCheck

# Newer than the one one stackage
- polysemy-1.3.0.0
- polysemy-plugin-0.2.5.0

############################################################
# Development tools
############################################################
Expand Down
Loading