Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSS should consume "pool of addresses for Nexus" #1530

Open
smklein opened this issue Aug 1, 2022 · 3 comments
Open

RSS should consume "pool of addresses for Nexus" #1530

smklein opened this issue Aug 1, 2022 · 3 comments
Labels
bootstrap services For those occasions where you want the rack to turn on

Comments

@smklein
Copy link
Collaborator

smklein commented Aug 1, 2022

In a handful of spots, this is being hardcoded as a single address. We will want to support multiple external IPs for Nexus, and should instead consume a pool of IPs.

@rmustacc
Copy link

rmustacc commented Aug 1, 2022

In addition to allowing us to specify a specific IP pool that isn't being used for customers, we should assume that operators will want to explicitly set and control these external IPs and select addresses from the pool.

@smklein
Copy link
Collaborator Author

smklein commented Aug 1, 2022

In addition to allowing us to specify a specific IP pool that isn't being used for customers, we should assume that operators will want to explicitly set and control these external IPs and select addresses from the pool.

This tracks - I figured there would need to be:

  • A setup / transfer mechanism to get a minimal set of this info from RSS -> Nexus, for bootstrapping
  • A general purpose API within Nexus for operators to use - ideally, similar to the customer-visible IP pool APIs - for managing these "used-for-Oxide-service" IPs.

smklein added a commit that referenced this issue Aug 3, 2022
Necessary for #1530

- Adds endpoints for "IP Pools used by Oxide services". These use the same "IP Pool" database implementation internally, but expose a distinct endpoint through the HTTP API.
- Adds a step to the "populate" process for creating the Oxide-owned IP pool. This is where IPs provisioned for usage by Nexus will be stored.
smklein added a commit that referenced this issue Sep 6, 2022
#1611)

Extends the allocation of external IPs to also support "service IPs", which are not associated with instances.

- Modifies the `instance_external_ip` table to let the project be optional. A "service IP" is one where the project and instance IDs are both "NULL".
- Modifies the external IP allocation CTE to support this new type of provision.
  - While I was there, fixed the idempotence of provisioniong when out of IP pool space for all external IP allocations.

Part of #1530
Fixes #1610
@smklein smklein added the bootstrap services For those occasions where you want the rack to turn on label Dec 6, 2022
@bnaecker
Copy link
Collaborator

bnaecker commented Apr 4, 2023

Just clarifying another point from the networking sync today. We will need to add a way to refer to one or more IP Pools when configuring each individual service. As an example, customers may create more than one IP Pool during RSS, on totally separate networks, and want to have Nexus listen on an address from each pool. So the service configuration data needs to include a list of IP Pools.

luqmana added a commit that referenced this issue Apr 12, 2023
We've been populating the services IP pool with a single address (the
external IP configured for Nexus) up til now. This flips that around by
instead requiring a list of IP ranges be given as part of rack
initialization. We can then pull addresses from there for Nexus (and any
other services as needed).

Further work is still needed to support multiple IP pools and allowing
services to utilize addresses across all (or some) of them.

Part of #1530.
leftwo pushed a commit that referenced this issue Nov 19, 2024
No Propolis changes other than to update Crucible

Crucible changes are:
Add debug/timeout to test_memory.sh (#1563)
Consolidate ack checking (#1561)
Rename for crutest: RegionInfo -> DiskInfo (#1562)
Fix dtrace system level scripts (#1560)
Remove `ackable_work`; ack immediately instead (#1552)
No more New jobs, no more New jobs column (#1559)
Remove delay-based backpressure in favor of explicit queue limits (#1515)
Only send flushes when Downstairs is idle; send Barrier otherwise (#1505)
Update Rust crate reqwest to v0.12.9 (#1536)
Update Rust crate omicron-zone-package to 0.11.1 (#1535)
Remove separate validation array (#1522)
Remove more unnecessary `DsState` variants (#1550)
Consolidate `DownstairsClient::reinitialize` (#1549)
Update Rust crate uuid to v1.11.0 (#1546)
Update Rust crate reedline to 0.36.0 (#1544)
Update Rust crate bytes to v1.8.0 (#1541)
Update Rust crate thiserror to v1.0.66 (#1539)
Update Rust crate serde_json to v1.0.132 (#1538)
Update Rust crate serde to v1.0.214 (#1537)
Remove transient states in `DsState` (#1526)
Update Rust crate libc to v0.2.161 (#1534)
Update Rust crate futures to v0.3.31 (#1532)
Update Rust crate clap to v4.5.20 (#1531)
Update Rust crate async-trait to 0.1.83 (#1530)
Update Rust crate anyhow to v1.0.92 (#1529)
Remove obsolete crutest perf test (#1528)
Update dependency rust to v1.82.0 (#1512)
Still more updates to support Volume layer activities. (#1508)
Remove remaining IOPS/bandwidth limiting code (#1525)
Add unit test for VersionMismatch (#1524)
Removing panic paths by only destructuring once (#1523)
Update actions/checkout digest to 11bd719 (#1518)
Switch to using `Duration` for times (#1520)
leftwo added a commit that referenced this issue Nov 20, 2024
No Propolis changes other than to update Crucible

Crucible changes are:
Add debug/timeout to test_memory.sh (#1563)
Consolidate ack checking (#1561)
Rename for crutest: RegionInfo -> DiskInfo (#1562) Fix dtrace system
level scripts (#1560)
Remove `ackable_work`; ack immediately instead (#1552) No more New jobs,
no more New jobs column (#1559)
Remove delay-based backpressure in favor of explicit queue limits
(#1515) Only send flushes when Downstairs is idle; send Barrier
otherwise (#1505) Update Rust crate reqwest to v0.12.9 (#1536)
Update Rust crate omicron-zone-package to 0.11.1 (#1535) Remove separate
validation array (#1522)
Remove more unnecessary `DsState` variants (#1550) Consolidate
`DownstairsClient::reinitialize` (#1549) Update Rust crate uuid to
v1.11.0 (#1546)
Update Rust crate reedline to 0.36.0 (#1544)
Update Rust crate bytes to v1.8.0 (#1541)
Update Rust crate thiserror to v1.0.66 (#1539)
Update Rust crate serde_json to v1.0.132 (#1538)
Update Rust crate serde to v1.0.214 (#1537)
Remove transient states in `DsState` (#1526)
Update Rust crate libc to v0.2.161 (#1534)
Update Rust crate futures to v0.3.31 (#1532)
Update Rust crate clap to v4.5.20 (#1531)
Update Rust crate async-trait to 0.1.83 (#1530)
Update Rust crate anyhow to v1.0.92 (#1529)
Remove obsolete crutest perf test (#1528)
Update dependency rust to v1.82.0 (#1512)
Still more updates to support Volume layer activities. (#1508) Remove
remaining IOPS/bandwidth limiting code (#1525) Add unit test for
VersionMismatch (#1524)
Removing panic paths by only destructuring once (#1523) Update
actions/checkout digest to 11bd719 (#1518)
Switch to using `Duration` for times (#1520)

Co-authored-by: Alan Hanson <alan@oxide.computer>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bootstrap services For those occasions where you want the rack to turn on
Projects
None yet
Development

No branches or pull requests

3 participants