Integrate mycelium in zos #2342

Eslam-Nawara · 2024-05-23T14:21:26Z

Description

Integrate mycelium in zos and allow flists to be served over mycelium

Changes

Add mycelium to networkd

Related Issues

Add mycelium support to zos #2338

Checklist

Tests included
Build pass
Documentation
Code format and docstring

xmonader · 2024-05-28T20:02:41Z

don't forget to add an ADR please

cmds/modules/networkd/main.go

xmonader · 2024-05-30T14:02:58Z

cmds/modules/networkd/main.go

@@ -148,9 +161,34 @@ func action(cli *cli.Context) error {
 		if err := dmzYgg.SetYggIP(ip, gw.IP); err != nil {
 			return errors.Wrap(err, "failed to set yggdrasil ip for dmz")
 		}
+
+		// set up mycelium in ndmz
+		dmzMy, err := mycelium.NewMyNamespace(dmz.Namespace())


can be extracted into another function

xmonader · 2024-05-30T14:03:40Z

docs/architecture/decisions/0010-mycelium.md

+
+## Decision
+
+Integrate mycelium in zos and allow zos host to have mycelium IPs, serve flists over mycelium, and support mycelium on zdbs


is it serve or mount flists over mycelium?

xmonader · 2024-05-30T14:04:06Z

pkg/network/types/constant.go

 	YggBridge = "br-ygg"
+	// MyBridge mycelium bridge
+	MyBridge = "br-my"


MyCeliumBridge

xmonader · 2024-05-30T14:05:05Z

pkg/network/networker.go

+	}
+
+	ip, err := myc.SubnetFor(hw)
+	log.Info().Msgf("myc subnet is %s", ip)


myc -> mycelium

xmonader · 2024-05-30T14:05:16Z

pkg/network/networker.go

@@ -114,13 +117,14 @@ func NewNetworker(identity *stubs.IdentityManagerStub, ndmz ndmz.DMZ, ygg *yggdr
 		portSet:        set.NewInt(),

 		ygg:  ygg,
+		myc:  myc,


xmonader · 2024-05-30T14:05:57Z

pkg/network/mycelium/utils.go

+	"github.com/threefoldtech/zos/pkg/zinit"
+)
+
+func EnsureMycelium(ctx context.Context, privateKey ed25519.PrivateKey, ns MyceliumNamespace) (*MyServer, error) {


move this into mycelium.go and remove utils file

muhamadazmy · 2024-05-30T13:58:29Z

pkg/flist/flist.go

 	// this should return always the flist mountpoint. which is used
 	// as a base for all RW mounts.
 	sublog := log.With().Str("url", url).Str("storage", storage).Logger()
 	sublog.Info().Msg("request to mount flist")

-	hash, flistPath, err := f.downloadFlist(url)
+	hash, flistPath, err := f.downloadFlist(url, namespace)


You probably need to use this same namespace at line 262 below

muhamadazmy · 2024-05-30T14:05:34Z

pkg/network/mycelium/mycelium.go

+
+	key := x25519.EdPrivateKeyToX25519([]byte(privateKey))
+
+	return json.NewEncoder(f).Encode(key)


Why are u using json here. the priv key of mycelium is not json encoded. It's a binary file. i don't believe this file is loaded by mycelium and is either ignored is overridden.

If this ever worked, then i assume it's because mycelium just read key length bytes from the file and used that as seed for the key!

This is definitely an issue, because it means the key u generated here is not secure. because it's not the full "Random" key but only part of the key json encoded bytes.

Do you get what I mean ?

muhamadazmy · 2024-05-30T14:07:21Z

pkg/network/mycelium/config.go

+	TunName    string
+	Peers      []string
+	PrivateKey ed25519.PrivateKey
+	PublicKey  string


Why is this needed?

muhamadazmy · 2024-05-30T14:08:42Z

pkg/network/mycelium/config.go

+	KeyFile    string
+	TunName    string
+	Peers      []string
+	PrivateKey ed25519.PrivateKey


Why do you keep the private key as ed25519 while we need x25519. I think it's better to convert the key before storing it on the NodeConfig from the start instead of changing it on start

muhamadazmy · 2024-05-30T14:10:36Z

pkg/network/mycelium/config.go

+	Peers      []string
+	PrivateKey ed25519.PrivateKey
+	PublicKey  string
+	NodeInfo   map[string]interface{}


What is this and what is it used for?

We don't need to copy everything over from ygg. Ygg has a completely different configuration and even requires a config file with way more config than mycelium.

Please drop all attributes that are not needed.

muhamadazmy · 2024-05-30T14:19:56Z

pkg/network/mycelium/mycelium.go

+	myBin        = "mycelium"
+	zinitService = "mycelium"
+	confPath     = "/tmp/mycelium_priv_key.bin"
+	MyListenTCP  = 9651


you will have to add this to the list of reserved ports of the system

it's added to reserved ports here

muhamadazmy · 2024-05-30T14:20:29Z

pkg/network/mycelium/mycelium.go

+func (s *MyServer) NodeID() (ed25519.PublicKey, error) {
+	if s.cfg.PublicKey == "" {
+		panic("EncryptionPublicKey empty")
+	}
+
+	return hex.DecodeString(s.cfg.PublicKey)
+}


muhamadazmy · 2024-05-30T14:24:03Z

pkg/network/mycelium/mycelium.go

+		return inspection, err
+	}
+
+	cmd := fmt.Sprintf(`exec ip netns exec %s %s inspect --json --key-file %s `, s.ns, bin, confPath)


Any idea why your command is a sh then u do exec and not the command directly!
I mean why not do

os.Command("ip", "netns", "exec", s.ns, bin, "inspect", "--json", "--key-file", confPath)

?!

muhamadazmy · 2024-05-30T14:25:11Z

pkg/network/mycelium/mycelium.go

+	Address   net.IP `json:"address"`
+}
+
+type Subnet [8]byte


this is not used

muhamadazmy · 2024-05-30T14:28:08Z

pkg/network/mycelium/mycelium.go

+
+// SubnetFor return an IP address out of the node allocated subnet by hasing b and using it
+// to generate the last 64 bits of the IPV6 address
+func (m *MyceliumInspection) SubnetFor(b []byte) (net.IPNet, error) {


I really think this should be called IPFor not SubnetFor even if it's called like that in ygg it's still wrong.

Eslam-Nawara added 7 commits May 23, 2024 10:56

set up mycelium in a namespace

d033324

add mycelium to networkd

02f6e5a

run mycelium in networkd

6c363f6

add mycelium to networkder

fdafe9c

add subnet function

0fc85ce

Merge branch 'main' into integrate-mycelium-in-zos

eff0cf2

rename dummy device

7525a0a

Eslam-Nawara force-pushed the integrate-mycelium-in-zos branch 2 times, most recently from b260121 to 52be829 Compare May 28, 2024 12:45

download flists in namespace

70b73ee

Eslam-Nawara force-pushed the integrate-mycelium-in-zos branch from 52be829 to 70b73ee Compare May 28, 2024 13:17

fix flist tests

6868cf7

Eslam-Nawara added 5 commits May 29, 2024 10:59

add mycelium adr

c1ed169

fix flist failure

4fe25cf

add mycelium support for zdbs

d5b8e6e

Merge branch 'main' into integrate-mycelium-in-zos

f574a14

fix mycelium ip returns as publicIp

2e128a2

Eslam-Nawara force-pushed the integrate-mycelium-in-zos branch from 37da9be to f6fddb9 Compare May 30, 2024 13:03

log failure in ensuring mycelium for zdb

29253a8

Eslam-Nawara force-pushed the integrate-mycelium-in-zos branch from f6fddb9 to 29253a8 Compare May 30, 2024 13:35

Eslam-Nawara marked this pull request as ready for review May 30, 2024 13:47

Eslam-Nawara requested review from muhamadazmy, delandtj, mariobassem, ashraffouda, rawdaGastan and xmonader as code owners May 30, 2024 13:47

xmonader reviewed May 30, 2024

View reviewed changes

cmds/modules/networkd/main.go Show resolved Hide resolved

xmonader reviewed May 30, 2024

View reviewed changes

pkg/network/types/constant.go Outdated

YggBridge = "br-ygg"

// MyBridge mycelium bridge

MyBridge = "br-my"

Copy link

Collaborator

xmonader May 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyCeliumBridge

xmonader reviewed May 30, 2024

View reviewed changes

pkg/network/networker.go Outdated

}

ip, err := myc.SubnetFor(hw)

log.Info().Msgf("myc subnet is %s", ip)

Copy link

Collaborator

xmonader May 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

myc -> mycelium

xmonader reviewed May 30, 2024

View reviewed changes

muhamadazmy requested changes May 30, 2024

View reviewed changes

rename mycelium constants

be8bc71

Eslam-Nawara force-pushed the integrate-mycelium-in-zos branch from 1fde494 to be8bc71 Compare June 3, 2024 07:21

Eslam-Nawara added 2 commits June 3, 2024 16:15

Merge branch 'main' into integrate-mycelium-in-zos

81c6c34

remove zdb ensure mycelium

8a16016

rawdaGastan mentioned this pull request Jun 4, 2024

Convert docker image to a flist and serve it using Mycelium on TFGrid threefoldtech/rfs#59

Closed

Eslam-Nawara force-pushed the integrate-mycelium-in-zos branch 4 times, most recently from 775b12f to f45a0e1 Compare June 10, 2024 09:02

run ensure container setup even if container is running

14ebe83

Eslam-Nawara force-pushed the integrate-mycelium-in-zos branch from f45a0e1 to 14ebe83 Compare June 10, 2024 09:06

muhamadazmy approved these changes Jun 10, 2024

View reviewed changes

muhamadazmy merged commit fd7d15f into main Jun 10, 2024
24 checks passed

muhamadazmy deleted the integrate-mycelium-in-zos branch June 10, 2024 11:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate mycelium in zos #2342

Integrate mycelium in zos #2342

Eslam-Nawara commented May 23, 2024 •

edited

Loading

xmonader commented May 28, 2024 •

edited

Loading

xmonader May 30, 2024

xmonader May 30, 2024

xmonader May 30, 2024

xmonader May 30, 2024

xmonader May 30, 2024

xmonader May 30, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024

Eslam-Nawara Jun 2, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024

muhamadazmy May 30, 2024


		## Decision

		Integrate mycelium in zos and allow zos host to have mycelium IPs, serve flists over mycelium, and support mycelium on zdbs


		key := x25519.EdPrivateKeyToX25519([]byte(privateKey))

		return json.NewEncoder(f).Encode(key)

Integrate mycelium in zos #2342

Integrate mycelium in zos #2342

Conversation

Eslam-Nawara commented May 23, 2024 • edited Loading

Description

Changes

Related Issues

Checklist

xmonader commented May 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Eslam-Nawara commented May 23, 2024 •

edited

Loading

xmonader commented May 28, 2024 •

edited

Loading