v6.4.1 - Polishing v6.4.0

Ten IXPs have upgraded since we released v6.4.0 just over two weeks ago. Thanks to all of them for their feedback and bug reports - especially Ed, Moritz and Nishal. v6.4.1 reflects their contributions and can be considered a polished, stable, production-ready release.

Release Summary

git --no-pager diff --shortstat v6.4.0 release-v6
 17 files changed, 802 insertions(+), 724 deletions(-)

Upgrade Instructions

The official upgrade instructions can be found here.

🚨 Upgrade to v6.4.0 first following those release notes.

The changes in this release as small improvements and bug fixes. There is one database migration to reset the MySQL VIEWs which does not affect IXP Manager's production use. Upgrading should be straight-forward.

Bug Fixes

• [BF] artisan utils:smtp-mail-test failed as it was referencing Laravel's older and now removed mail library 8473497
• [IM] Telescope caused issues due to excessive undefined array element logging when running under PHP 8.1. Telescope is typically used more as a development aid rather than a production tool and should be disabled by default 6342226
• [DB] While not required for IXP Manager, we maintain the older views as we know some people have integrated their own scripts into using them. The script to refresh these now relies on changes later in the migration path and so we have moved the migration for the view reset. Fixes #894.
• [BF] IXP Manager's PeeringDB interface has changed which broke the PeeringDB helper when adding new members - 7a4e41c
• The verbose warnings under PHP 8.1 for non-IXP Manager library code was raising some queries on the mailing list. Found a way to silence these via a dummy logging driver entry b083dff

v6.4.0 Route Servers - Resilience and UI Based Community Filtering

This release provides significant new features for route server resilience, UI-based community filtering, and many smaller improvements and bug fixes.

Release Summary

git --no-pager diff --shortstat v6.3.1 master
 218 files changed, 21503 insertions(+), 28569 deletions(-)

Upgrade Instructions

🚨 🎥 There is a tutorial video demonstrating this upgrade including all the required changes on route collectors and route servers available on YouTube here.

The official upgrade instructions can be found here. Follow these, including the database migrations.

Edit your .env file and add the following:

TELESCOPE_ENABLED=false

Once that is complete, and assuming you have read these release notes in full, proceed as follows:

1. Edit your router definitions on IXP Manager to set up router pairs.
2. Replace your route server and route collector update scripts with the new scripts linked below.
3. Update the cadence of the router update scripts and enable the route server filtering UI.

Route Server Resiliency

For IXPs, route servers are considered a critical production service and most IXPs deploy them in redundant pairs. This is usually implemented with dedicated hardware (servers with dual PSU, hardware RAID, and out-of-band management access) deployed in different points of presence.

When it comes to updating the configuration of these, the scripts provided by IXP Manager suggested that this be done about four times per day with the timing of the cronjob set so that there is an offset so that each server will not update at the same time. The hope was that if there was an issue, only one server of the resilient pair would be affected, and engineers would have time to react and prevent updates on the other working server. Some IXPs added additional logic to the scripts to check if the other server was functional before performing a reconfiguration, but this was often limited to pings and a simple check to see if Bird was running.

This release adds a significant new resilience mechanism by pairing servers. In the IXP Manager router UI, you can now select another router to pair with the one you are editing. You would select pairs as follows:

• For route servers deployed in pairs, rs1-ipv4 should be paired with rs2-ipv4 and vice versa - be sure to set the paired server in each individual server.
• For route collectors, quarantine route collectors and AS112 services where you would normally have a single instance, you can pair the ipv4 version with the ipv6 version, ensuring at least one will always be running. For example, pair rc1-ipv4 with rc1-ipv6 and vice versa.

Once your pairs are set up, you need to deploy the new router update scripts as follows:

• for route servers: tools/runtime/route-servers/api-reconfigure-example-birdv2.sh
• for route collectors: tools/runtime/route-collectors/reconfigure-rc-bird2.sh

There is no need to use different scripts for route collectors and servers. Traditionally, at INEX, these scripts were developed slightly differently from each other (e.g., the collector script updates both IPv4 and IPv6 versions and provides more informative output, whereas the route server script takes a specific route server handle to update). We may merge these in the future.

You can use these scripts exactly as they are on an Ubuntu server changing only the configuration lines at the top:

APIKEY="your-api-key"
URLROOT="https://ixp.example.com"
BIRDBIN="/usr/sbin/bird"

The collector script takes an additional configuration option for the handles of the servers to update - e.g.:

HANDLES="rc1-ipv4 rc1-ipv6"

These new scripts now work as follows:

1. NEW: Obtain a local script lock preventing more than one update script to execute at a time.
2. NEW: Obtain a configuration lock from IXP Manager.
   • This involves making an API call to /api/v4/router/get-update-lock/$handle, which IXP Manager then processes and returns HTTP code 200 if the lock is acquired and the update can proceed.
   • A lock is not granted if the router is paused for updates within IXP Manager (new per-router option in the router's dropdown menu on the router list page).
   • A lock is not granted if another process has already acquired a configuration lock for this router.
   • A lock is also not granted if the router's partner is locked. This major new resiliency addition prevents two paired route servers from being updated in parallel.
   • The update script will be aborted if IXP Manager is unavailable or in maintenance mode.
3. If a lock is acquired, the script will then download the latest configuration from IXP Manager.
4. The script will do some basic sanity checks on the downloaded configuration:
   • First, check that the HTTP request to pull the new configuration succeeded.
   • Second, check that the downloaded file exists and is non-zero in size.
   • Third, ensure at least two BGP protocol definitions are in the configuration file.
   • Lastly, the script has Bird parse the downloaded file to ensure validity.
5. NEW: The update script will now compare the newly downloaded script to the running configuration.
   • If there are differences, the old configuration is backed up, and the Bird daemon will be reloaded.
   • If no differences exist, the Bird daemon will not be reloaded.
6. A check is performed to ensure the Bird daemon is actually running and, if not, it is started.
7. IMPROVED: A final API call is made to IXP Manager via /api/v4/router/updated/$handle to release the lock and update the timestamp.
   • A significant improvement here is the use of a until api-succeeds, sleep 60, retry construct to ensure the lock is released even when there are transitive network issues / IXP Manager maintenance modes / server maintenance, etc.

Adding step (5) above (only reload on changes) now allows the update script to be safely run as frequently as every few minutes, which is necessary for the UI-based community filtering to be effective.

You should still offset the updates between router pairs, as the script will give up if a lock cannot be obtained. Future improvements could allow for some retries.

For additional information with UI images, see slides 25-30 in this presentation PDF.

Route Server Community Filtering via the UI

Community-based filtering is the standard way to allow route server participants at an IXP to control their routing policy. IXP Manager has supported - and set - the standard across the industry since route servers were introduced at INEX in 2007.

Such filtering is essential to maximise participation with route servers as the member is essentially outsourcing their routing policy to the IXP, and many would be uncomfortable or unable to do this without these basic controls.

Community-based filtering in practice can be difficult for participants at both ends of the network-size scale:

• Small networks rarely touch their border routers and may be both unfamiliar and uncomfortable with the necessary concepts and configuration to use them. This is especially true in a stressful situation when they urgently need to apply communities for the first time.
• Large networks may need cumbersome change control procedures or, in some cases, their automated provisioning pipeline may not even support them.

We must also remember that community filtering is only half the story - the participant will still need to apply route filters to the routes they learn from the route servers (community filtering applies to how their prefixes are propagated by the route servers).

This release of IXP Manager introduces a new feature which allows IXP members to configure route server filtering in a web-based UI. This will move the configuration complexity from the member and their router to the IXP's route servers. The actual mechanism of filtering is unchanged - just where it happens moves:

• The route server will apply community tags to the member's routes immediately at ingress rather than the member doing it on
  egress.
• In the other direction, the route server will filter routes to be advertised to the member on egress rather than the member doing it on ingress.

We expect this to work for >=90% of use cases. A member with a more complex routing policy should handle it on their own routers anyway.

The implementation in IXP Manager uses two database tables - a staging table and a production table. When a member first creates or subsequently edits their filters, this will happen in the staging table. Once they are satisfied their routing intentions are complete, they can commit the changes to the production table. As each router processes its next configuration update, the comparison diff discussed in the above section will show differences, and the router configuration will be updated. It is important, therefore, to have the route servers update on a schedule of at least every 10 minutes.

To allow IXP administrators to update and increase the frequency of their route server update scripts, this UI feature is disabled by default. To enabled it, add the following to your .env file:

IXP_FE...

XSS Security Fixes, Small Bug Fixes and Minor Improvements

This release primarily fixes a number of XSS security issues in IXP Manager. These were discovered and responsibly disclosed by the GRNET IT Security Team and we thank them for that.

This release is a bugfix release and so there are no database schema changes.

Summary:

Release Summary

git --no-pager diff --shortstat v6.3.0 v6.3.1
 78 files changed, 1390 insertions(+), 1155 deletions(-)

Upgrade Instructions

The official upgrade instructions can be found here.

The changes in this release as small improvements and bug fixes. There are no database changes or other complexities. Upgrading should be straight-forward.

Security Fixes

This release includes a fix for five XSS security bugs.

We judge four of these bugs have a CVSS score of CVSS:0.0/AV:N/AC:L/PR:H/UI:R/S:U/C:N/I:N/A:N. These can only be exploited by an authenticated superadmin user who would enter specifically crafted JavaScript code in specific input fields.

The final we judge as CVSS:4.6/AV:N/AC:H/PR:L/UI:R/S:U/C:L/I:L/A:L as an attack exploiting this could be possible from a sufficiently sophisticated and motivated non-admin user who could find a way to inject a XSS payload into a logged database object and could then convince a superadmin to view that database change in the UI log tool. The GRNET IT Security Team have registered CVE-2023-36666 for this.

Credit to the GRNET IT Security Team for responsibly disclosing these issues.

Improvements

• Composer will install the latest OSS_SNMP library making more switches compatible with IXP Manager.
• All PHP framework and libraries used have been updated to latest versions as compatible with PHP 8.0.
• [NF] Filter by ports prewired only in patch panel management 844e16a; closes #814
• missing lladdr ignore no longer available in bird2 - removed from templates b81b89c
• Route server config for bird2 now fixed to allow 32-bit asns via 493ba15)

Bug Fixes

• Ignore reseller ports when generting MRTG configuration via e40be75; closes #855

v6.3.0 - Security hardening, with various improvements and bug fixes

A commercial IT consultancy provider uses IXP Manager in one of their solutions. They had their overall solution reviewed by an internationally respected cyber security and risk assessor. This review included IXP Manager and the commercial IT consultancy responsibly disclosed all of the issues and advice related to IXP Manager.

These have been addressed in this release and are itemised below. We recommend all IXPs that use IXP Manager upgrade to this new version.

We thank the IT consultancy, and those within it that we have been dealing with, for sharing the findings with us.

Release Summary

$  git --no-pager diff --shortstat v6.2.0 release-v6
 87 files changed, 3025 insertions(+), 2513 deletions(-)

Upgrade Instructions

The official upgrade instructions can be found here.

This release does include some minor database migrations - please follow the instructions above.

Post-Upgrade Instructions

We use Laravel's mail system and so we need to keep in sync with their defaults. A recent change means that when sending email via SMTP, tls no longer the default. See #752 for a discussion.

If using SMTP, ensure you test emails via the test tool here.

Security

• Remove web.config from public via e9d0819. Not used and was the framework default so not an issue here, just best practice. Came up as an issue in a security audit and we note this has been removed from Laravel for the same reason: laravel/laravel@4bc502b
• Escape specific instance of HTML content to prevent XSS [ref: 055-9-4] via bc9b14c
• Make response to forgotten password generic [ref: 055-9-8] via 04fe7d8
• Implement a stronger password policy [ref: 055-9-7] via 2889be9
• Prevent XSS / JS interpretation in preview boxes [ref: 555-9-9] via 083d17e
• Disable phpinfo() by default [ref: 055-9-11] via 921f515
• Don't allow user with priv = 0 [ref: 055-9-13] (bug fix) via e5a48ab
• Check for patch panels when deleting racks [ref: 055-9-14] via 5ec406e

Improvements

• Peering Matrix - increase look-back days and make configurable which makes the detection better in some cases via 53333e4
• Garbage collection for macaddress table via 724a05c
• Make member display name formatting configurable in .env to assist with #766
• Preserve IXPM functionality from recent OSS_SNMP change via 1b4cbfa re switch serial number via SNMP being not implemented
• Remove final runtime dependency on views.sql via b414ca1
• Modernise PeeringDB and IX-F links and make them configurable via 0ac782d
• Add patch_panel.colo_pp_type - a new database for the colo end of patch panels via 3fd45df
• Link to reseller in member overview. Use more commonly displayed abbreviatedName. Closes #802
• Update robots.txt - LG can also be referenced using /index.php/lg/. Need to exclude this from search engines via 4e17563
• Enable logout on 2fa required page - fixes #806 and prevents user getting 'locked' in 2fa

Bug Fixes

• Fix display of IPv6 addresses in mac-address/list via d37e285
• Fix duplicated entries in mac-address/list route via daee524
• $sp->isTypePeering() -> $sp->typePeering() via 305a79e
• Could not save associate member edits due to int/string comparison via b707e00
• Fix issue with route display in looking glass via 5f7b338
• Fixing vlan id to close #773
• Incorrect ipv6enabled check - noted by @listerr in #778 - and also bugfix via d1278d5
• ipv4_subnet does not display properly - fixes #783
• Core bundles - need user before checking for permissions via 66d1a60
• Escape returned data and error messages on login forms via c623390
• /statistics/member raises error if not auth'd via f9d2f99
• Fix log viewer - do not crash out if user is deleted / not listed on log via 2ab9102

v6.2.0 - 'Partial Port Speed' Support, New Features and Improvements, Bug Fixes

This release provides a long sought-after feature to allow for a differentiation between the declared port speed (where the IXP provides subrate connectivity) and that configured on the switch which may not match. Among some other small additions, we've also included support for per-facility graphs which are the aggregate of all peering ports excluding core/trunk ports in a given location.

This release also provides a large amount of improvements and bug fixes accumulated since v6.1 was released about two months ago. Most of these relate to the bigger changes in v6.0 and have been found and reported as more IXPs upgraded. Thanks to all how contributed via those reports.

Release Summary

git --no-pager diff --shortstat v6.1.0 release-v6
 174 files changed, 6187 insertions(+), 4775 deletions(-)

Upgrade Instructions

The official upgrade instructions can be found here.

This release does include some minor database migrations - please follow the instructions above.

Post-Upgrade Instructions

NB: If you have skinned the router configuration templates then review the following changes. If you have not skinned these files, you can safely skip this section.

1. As part of #716 (transit ASNs) the following files changed:
   • resources/views/api/v4/router/collector/bird2/filter-transit-networks.foil.php
   • resources/views/api/v4/router/server/bird2/filter-transit-networks.foil.php
2. To implement a fix for the minimum subset size logic from #682, a large number of the template files required minor changes - see 0b7a958.
3. In moving rfc1997 communities config to neighbour template for #730, these two files changed:
   • resources/views/api/v4/router/server/bird2/neighbor-template.foil.php
   • resources/views/api/v4/router/server/bird2/neighbors.foil.php

We further improved the IRRDB efficiency from the route server speed upgrades in v6.1.0. While not necessary, you can prime the cache in advance if you like by running the following:

php artisan irrdb:update-prefix-db -v
php artisan irrdb:update-asn-db -v

Lastly, the IRRDB hosts as set in IXP Manager under IRRDB Configuration in the left hand menu was always ignored by the code which executes BGPQ3 using its default of whois.radb.net. One of the migrations will set all IRRDB sources to whois.radb.net which preserves existing behavior and ensures no production issues. Any changes by an admin there after will work. There was a discussion on the mailing list about this here.

New Feature: Partial Port Speeds

In cases where the IXP provides subrate connectivity, the declared port speed at an IXP may not match the port speed which is configured on the access device. Our first pass at supporting this is now including in v6.2.0 and includes:

• New rate_limit column on the database as the actual request for partial port speeds is really a request to honour rate limits on ports. If there are other use cases this can still be used, the actual physical (production) effect is determined by how individual IXPs use this information in their provisioning systems / as they manually configure switches.
• Rate Limit field when adding and editing a physical interface through the normal virtual interface overview. NB: this means if using the New Interface Wizard, you configure the physical speed and then edit the physical interface afterwards. The wizard is meant to cover the 90% scenario and we'd prefer to keep it de-cluttered.
• This is for member ports; not core ports.
• Speed reported on various member screens as the rate limited speed (and sometimes, where space / appropriate) as x / y and also GUI labels to make it clear when a port is rate limited.
• IX-F Member Export now reports rate_limit speed and adds an IXP Manager specific field called if_phys_speed to show the physical port speed.
• The switch configuration viewer (e.g. INEX's one also updated to support this.
• Port utilisation updated to be congnisant of this.
• Switch provisioner yaml/json output now has rate_limit for peering ports, null means not in use.
• MRTG configuration has not been updated as the MRTG config uses the detected speed from SNMP. We currently do not see any value in swapping this for the rate limited speed as rate limiters can be buggy / misconfigured / etc and we'll want to see that traffic.
• The dashboard statistics also reflect this and include a new section at the end which shows how partial ports were accounted for.
• The physical interface list (left hand menu -> Interfaces -> Physical Interfaces) has a rate limit column to make it easy to see the rate limited ports.

These notes are also included in the documentation.

New Feature: Per-Facility Aggregate Graphs

These are defined and built automatically from the locations / facilities you have defined and the switches you have assigned to them. These graphs are available from the Statistics menu under Facility Aggregate Graphs.

These graphs are the aggregate of all peering ports excluding core/trunk ports in a given location / facility. It is the aggregate of traffic originating and/or terminating in a location rather than traffic simply passing through a location.

You'll find examples of facility graphs on INEX's own IXP Manager where they are public be default.

New Features

• [NF] Export reseller / resold / manrs state in IX-F export - implements #674
• TravisCI has removed the free tier for OSS projects; a number of commits relate to the migration from TravisCI to GitHub Actions
• [NF] Per-Facility / PoP aggregate graphs as requested in #635 (see above)
• [NF] Partial port speeds as requested in #260 (see above)

Improvements

• [IM] Route Servers: Update transit ASN filtering and allow greater control - closes #716 (docs)
• [IM] Route Servers: IRRDB - better sorting consistency and logic in one place
• [IM] Route Servers: tidy logic on "minimum subnet size" filters - closes #682
• [IM] Route Servers: move rfc1997 communities config to neighbour template (#730)
• [IM] Include PeeringDB link on peering request emails
• Refine peeringDB OAuth checks to fix #703
• [BF|IM] Use the IrrdbConfig host rather than defaulting to BGPQ3 - fixes #678
• Include port counts by cabinet - closes #585
• [IM] Allow the switch ports we poll via snmp to be configurable - fixes #695
• Use pollable for mrtg graphing - closes #694 and closes #690
• [IM] Grapher - MRTG - skip manually added dummy ports per #746

Bug Fixes

• [BF] Identify single member LAGs in welcome email - closes #714
• fix routing for statistics@member 0c889e9 / bdece40
• Fix logic in log search controller - closes #733
• [BF] Allow ordering reset on patch panel ports - fixes #639
• [BF] Fix (reimplement) deletion of a user's 2fa by superadmin - fixes islandbridgenetworks/IXP-Manager-Archive-Yann#275
• [BF] Error when adding patch panel without Port name prefix - fixes #738
• [BF] Error when adding patch panel without Port name prefix #738
• [BF] SQL Error when duplicating VLAN Interface if it has linked MAC address - fixes #750
• BF: Weird time format for created_at column. (#749)
• [BF] 500 Error when customer updates logo file #744
• [BF] SQL Error/orphaned record when custadmin user deletes another user #742
• [BF] Configured mac address list doesn't show MAC addresses - fixes #740
• [BF] Moving simplex patch panel ports doesn't work #743

v6.1.0 - Optimising Route Server Configuration Generation

We've known for sometime that the generation time for router configurations was sub-optimal and looking at that has been on the todo list. A regression in v6.0.0 in the move to Eloquent caused an increased latency in the process and explained well by @ichilton in #732.

This release resolves that issue and provides a significant speed increase for generating router configuration - especially where you have members which huge prefix lists.

This is a minor version bump rather than a patch release as we want highlight that if you have skinned the router configuration templates you may need to update your own versions also. Details below. And a reminder - at @inex we use the stock templates always so these are already in production.

Release Summary

git --no-pager diff --shortstat v6.0.1 release-v6
 18 files changed, 174 insertions(+), 36 deletions(-)

Upgrade Instructions

The official upgrade instructions can be found here.

The changes in this release are all code based. There are no database changes or other complexities. Upgrading should be straight-forward. If you have skinned the router configuration templates then see the details below.

You can prime the cache in advance if you like by running the following (these run via the cronjob anyway and the cache would be primed on the first route server config pull):

php artisan irrdb:update-prefix-db -v
php artisan irrdb:update-asn-db -v

Route Server Optimisation

In commit b852c1f through the use of serialising IRRDB prefixes at the time of fetching them from IRRDB databases, we've gotten the speed / usage down as follows:

Iteration	Generation Time	Memory Usage (Real)
v6.0.0	99.5 secs	1,362 MB (1,551 MB)
1st Pass - Query Optimisation	32.0 secs	501 MB (576 MB)
2nd Pass - Add Caching	1.3 secs	210 MB (212 MB)

NB: this is on @barryo's development laptop with local MySQL and webserver. YMMV for production environments but the speed up should be proportional.

Skin Updates

NB: if you have skinned route server configuration templates then you should review the changes in commit c83f8e4 and update your skins.

This commit moves the collection of neighbour prefixes from "all on one go" to a per neighbour processing basis. This has no immediate win but it will allow for future optimisations.

Bug Fixes

• Broken URLs for click-to-search for prefixes in the master[46] table of routers in the looking glass 1b650ad

IXP Manager v6.0.1 - Improving and polishing v6.0.0

Since we released v6.0.0 about six weeks ago, at least 13 IXPs have upgraded.

Thanks to all of those for their feedback and bug reports. v6.0.1 reflects their contributions and can be considered a polished, stable, production ready release.

Yann Robin

This release also reflects @yannrobin's final official contributions to the project (although we have some features worked on by Yann still in development). Yann has relocated back to France and has taken up a position with another company. Yann has been with the project for five years and during that period the number of IXPs using the platform grow from 27 to nearly 200.

The IXP Manager project and INEX both want to sincerely thank Yann for the work he has done and the commitment he has shown to make IXP Manager the success that it is today. There is no doubt the work he has undertaken has helped enable a better internet in so many countries - especially where they were battling against the system to make it work - and his programming fingerprints will endure in these places for many years to come.

We wish Yann every success in the future. Bonne chance.

Release Summary

git --no-pager diff --shortstat v6.0.0 release-v6
 58 files changed, 530 insertions(+), 469 deletions(-)

Upgrade Instructions

The official upgrade instructions can be found here.

The changes in this release as small improvements and bug fixes. There are no database changes or other complexities. Upgrading should be straight-forward.

New Features

• Possible to filter fields on the IX-F Member Export - see documentation here.

Improvements

• Improve the speed of IX-F member export (6x in some cases)
• Allow virtual interface name to have a space at the end in core bundle create form
• [HK] Do not log MAU discovery when logger false
• [DB|IM] eagerly load required entities -> speeds up member overview page [also fixes islandbridgenetworks/IXP-Manager-Archive-Yann#272]
• [IM] - Stripe % from search via 13b4450
• [IM] do not include period with API key as it will cause confusion via 30cb322

Bug Fixes

• ApiMaybeAuthenticate - did not allow access via the browser with a live session.
• Server Error :: Add [peeringdb_id] to fillable property to allow mass assignment on [IXP\Models\User] - fixes #720
• Polling of switch ports were not persisted to the database
• [BF] - fix api/v4/public/content/members/0/list.json via 052a71c
• [BF] - Check for api key grapher api via 92deecf
• Fix migration error by @barryo and complete router update fix per #718
• [BF] - Ports lists not working - fixes #723
• [BF] - xc: cross-connect search broken + Search queries Improvement - fixes islandbridgenetworks/IXP-Manager-Archive-Yann#273
• [BF] - Using 'Aggregate Traffic Statistics' from a member's page opens a different member's page - fixes #725
• Remove INEX-specific content - closes #726
• remove duplicate tab for P2P (#728)
• [BF] Fix show/hide of OOB connections - closes #727
• [BF] replace getId() with $c->id (fixes 500 error) (#731)
• [BF] private VLANs need to be selectable also via 2fbdee0
• [BF] Issue displaying private VLANs without a physical interface via 9d4d46c
• [BF] SwitchPort API broken - fixes #734

IXP Manager v6.0.0

Welcome to the release of IXP Manager v6 and thanks for your patience to date - we know this release was some time in the making.

One of the biggest changes in v6 relates to a significant framework change - Doctrine ORM has been completely removed and replaced with Laravel's Eloquent. This work is thanks to @yannrobin who has done all the heavy-lifting on rewriting all database-related code within the project over the past few months. Using Eloquent, which is a key component of Laravel and integrates fully with the framework, should allow us to develop new features with greater ease and speed.

This release also introduces two new major features: core bundle management (inter-switch links) and database change logging.

Release Summary

git --no-pager diff --shortstat release-v5 release-v6
 1280 files changed, 83436 insertions(+), 138379 deletions(-)

A brief summery of the biggest changes in IXP Manager v6 include:

• Significant framework changes - Doctrine ORM has been completely removed and replaced with Eloquent.
• Core bundles - management of the IXP's own inter-switch links.
• Database change logging - additions, edits and deletions to database objects are now recorded for six months.
• Command line tools for user management - these have been requested a number of times and should prove very useful.

We have also released a video tutorial of the IXP Manager v5.8.0 to v6.0.0 upgrade process which includes upgrading to PHP 8 on an existing system (e.g. Ubuntu 18.04 in the video) and a second option of upgrading to Ubuntu 20.04 and then to PHP 8.0.

Upgrade Instructions

If you are upgrading to IXP Manager v6, this is where to start. We assume that you are upgrading from v5.8.0. No other upgrade paths are supported or recommended.

To help with upgrades, we've cloned INEX's own installation of IXP Manager v5.8.0 running on Ubuntu 18.04 LTS with PHP 7.3 and we've recorded a video of the upgrade process for both IXP Manager to v6.0.0 and to replace PHP 7.3 with PHP 8.0. You can find that video here and these instructions were created when doing test runs for that same video. The last section of that video also demonstrates an upgrade from Ubuntu 18.04 to 20.04 and how to over come a couple issues.

Prerequisite: PHP >= v8.0 is required for IXP Manager v6.0.0. If you are using Ubuntu, we recommend this PPA and video above shows the 7.3 -> 8.0 upgrade process. We would also recommend that you upgrade to Ubuntu 20.04 LTS as part of this process if you are using an older version of Ubuntu (however it will work fine on 18.04 with PHP 8.0). The end of the referenced video includes a 18.04 -> 20.04 upgrade.

The process we describe here is based on a standard installation of IXP Manager using the automated installation scripts. If you have installed IXP Manager in a custom location / using custom settings then just be sure to adjust the following to match your own environment.

We will assume your installation of IXP Manager is installed at a location identified by the shell variable IXPROOT. This is set in step (1) below.

Remember: you can follow along with the video for a more detailed discussion on these steps.

1. Set up some variables and ensure directory permissions are okay:

   # set this to your IXP Manager installation directory
   IXPROOT=/srv/ixpmanager
   
   # fix as appropriate to your operating system. ubuntu/debian is fine with this:
   MY_WWW_USER=www-data

2. Let's take some backups of both the IXP Manager installation and the database:

   # Start with the IXP Manager installation
   tar -jcf /srv/ixpmanager-v5.8.tar.bz2 $IXPROOT
   
   # And now the database:
   source $IXPROOT/.env
   mysqldump --no-tablespaces -h $DB_HOST -u $DB_USERNAME -p$DB_PASSWORD $DB_DATABASE | \
       bzip2 -9 >/srv/ixpmanager-database-v5.8.sql.bz2

   You should copy both of these files off the server before proceeding.

   Before continuing, you should also ensure your IXP Manager MySQL user has SUPER privileges. This needs to be done as the MySQL root user. The command you need to execute with MySQL can be determined as follows:

   source $IXPROOT/.env
   echo "GRANT SUPER ON *.* TO \`$DB_USERNAME\`@\`$DB_HOST\`;"

   This command will need to be run on MySQL as the root user.

3. Enable maintenance mode to let your users know what's happening:

   cd $IXPROOT
   php artisan down --message='Upgrading to IXP Manager v6. Check back after 23:00 BST.'

4. Shut down the sflow p2p data collection system and safely flush the rrdcached process if this is being used:

   killall sflowtool
   killall -USR1 rrdcached

5. Ensure our operating system is fully up to date:

   apt update
   apt upgrade

6. We now need to upgrade PHP to 8.0 and install any other required packages:

   # We use a Ubuntu PPA for the 8.0 packages. You may already be using this but it can be set-up as follows:
   apt-get install -yq software-properties-common
   add-apt-repository -y ppa:ondrej/php
   apt update
   
   # With that set-up, we install all the packages required by IXP Manager v6.0.0.
   # (you may already have a lot of these installed - taken from https://docs.ixpmanager.org/install/manually/):
   apt install -qy apache2 php8.0 php8.0-intl php-rrd php8.0-cgi php8.0-cli          \
       php8.0-snmp php8.0-curl  php-memcached libapache2-mod-php8.0 mysql-server     \
       mysql-client php8.0-mysql memcached snmp php8.0-mbstring php8.0-xml php8.0-gd \
       php8.0-bcmath bgpq3 php-memcache unzip php8.0-zip git php-yaml                \
       php-ds libconfig-general-perl libnetaddr-ip-perl mrtg  libconfig-general-perl \
       libnetaddr-ip-perl rrdtool librrds-perl curl composer

7. The above will possibly have left PHP in a bit of a mess. Best to review the video but essentially, we want to remove any crud from older versions. In the example I'm using, our crud relates to PHP 7.3.

   # find old packages and, if nothing looks unusual, purge them:
   dpkg -l | grep php7.3
   dpkg --purge `dpkg -l | grep php7.3 | awk '{print $2}'`
   
   # see if there are any other versions lingering around (php-xxx packages are fine):
   dpkg -l | grep php
   # ... and remove and purge them if there are.
   
   # check your php version now and it should be >= 8.0.0:
   php -v

   If you have more than ~20 members or members with huge route server prefix lists, then you should edit /etc/php/8.0/apache2/php.ini and set memory_limit to >= 1024 and max_execution_time to 300.

   In our test runs, Apache was not setup to run PHP 8.0 after the above. This is corrected with:

   apt install libapache2-mod-php8.0
   a2enmod php8.0
   systemctl restart apache2.service

8. Upgrade IXP Manager:

   # pull the latest code
   git fetch --all
   # check out the version you are upgrading to
   git checkout v6.0.0

9. Install Composer locally. This is important because Ubuntu uses v1 at the moment which is not compatible with PHP 8.0. These instructions come straight from getcomposer.org and you should browse to here and follow the latest as the hash in this example may be out of date:

   php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
   php -r "if (hash_file('sha384', 'composer-setup.php') === '756890a4488ce9024fc62c56153228907f1545c228516cbf63f885e036d37e9a59d27d63f46af1d4d07ee0f76181c7d3') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
   php composer-setup.php
   php -r "unlink('composer-setup.php');"

10. Install the required libraries via composer:

    # this assumes composer.phar is in the IXP Manager install directory. YMMV - see notes on the upgrade instructions.
    sudo -u $MY_WWW_USER bash -c "HOME=${IXPROOT}/storage && cd ${IXPROOT} && php ./composer.phar install --no-dev --prefer-dist"

11. Restart Memcached and clear the cache. Do not forget / skip this step!

    # (assuming we're still in $IXPROOT)
    systemctl restart memcached.service
    php artisan cache:clear

12. Update the database schema:

    # (assuming we're still in $IXPROOT)
    # (you really should take a mysqldump of your database first if you haven't already)
    # see what will change:
    php artisan migrate:status
    
    # migrate:
    php artisan migrate --force

13. Ensure file permissions are still correct.

    chown -R $MY_WWW_USER: ${IXPROOT}/storage $IXPROOT/vendor $IXPROOT/bootstrap/cache
    chmod -R u+rwX ${IXPROOT}/storage $IXPROOT/vendor $IXPROOT/bootstrap/cache

14. Clear out all caches:

    # (assuming we're still in $IXPROOT)
    php artisan cache:clear
    php artisan config:clear
    php artisan route:clear
    php artisan view:clear

...

Various small fixes and improvements to cap the v5 branch

This release has various small fixes and improvements and is intended to be the last significant release for the v5 branch. All future development effort, new features and fixes will be committed to the soon-to-be-released v6 branch. All IX's running < v5.8.0 are advised to upgrade as this will also be the staging point for upgrading to v6.

Summary:

git --no-pager diff --shortstat --no-merges v5.7.0 v5.8.0
 62 files changed, 37478 insertions(+), 20965 deletions(-)

Upgrade Instructions

Please follow the official upgrade documentation without skipping any steps.

Additionally, please run the following MySQL command:

UPDATE cust SET `type` = 3 WHERE `type` = 5;

This can be easily ran as follows:

cd /srv/ixpmanager # or wherever you have IXP Manager installed
source .env
mysql -u $DB_USERNAME -p$DB_PASSWORD $DB_DATABASE -e 'UPDATE cust SET `type` = 3 WHERE `type` = 5;'

Small Features and Improvements

• New tool for testing email - see the documentation here
• Disabled users and closed customers not managed properly - fixes islandbridgenetworks/IXP-Manager-Archive-Yann#268
• Autocomplete enabled on 2FA code input - fixes #713
• Document store require login for members-only folders - fixes #709
• AS-SET-s stored in IXP manager visible/manageable by members - closes #708
• More detail on Document Store extensions - addresses #686
• update exit codes for check-perl-dependencies.pl 8ce718c
• updates for recent perl dependencies [FreeBSD 12, Debian9/10, Ubuntu 18.04/20.04] ecbbbaf
• Cisco / Huawei use ifType l2vlan for subinterfaces 2c0dd11
• Log successful logins 2968292

Bug Fixes

• Filtered prefixes tab route server button fails when route limit is higher than the maximum set in birdseye - fixes #707
• prevent getSource() bombing out if the customer has no irrdb set 80c3986
• IPv6Address::hostname() should return ipv6hostname, not ipv4hostname 030261a
• fix for v6 peering matrix 418b4ff
• Bug: 500 Error Undefined index trying to list any contact group that is not 'ROLE' type - fixes #706
• Error 500 when changing password - fixes #679
• work around bash bug which leaks named pipe descriptors 4f4026a

Security Fix, Small Bug Fixes and Minor Improvements / Features

This release primarily fixes a XSS security issue in IXP Manager. It also has a small number of bug fixes and improvements. All IX's running < v5.7.0 are advised to upgrade. This release has a minor version bump as there are two small database schema changes.

Summary:

git --no-pager diff --shortstat --no-merges v5.6.0 v5.7.0
 152 files changed, 13874 insertions(+), 8307 deletions(-)

Upgrade Instructions

Please follow the official upgrade documentation without skipping any steps.

There are no additional release specific steps required.

Security Fix

This release includes a fix for a XSS security bug in the looking glass feature.

The bug allows a potential attacker to provide an IXP Manager user or administrator a crafted URL which would result in the execution of supplied JavaScript within the user's browser.

If you are running IXP Manager with the looking glass feature enabled, you are advised to upgrade. If you wish to delay the upgrade and mitigate the risk in the mean time then you could:

1. set the looking glass access privileges to SUPERUSER in each of your router configurations;
2. advise your SUPERADMINS to examine any externally provided IXP Manager URL for the presence of potential XSS code.

Credit to Bart Vrancken (AbuseIO CERT) for responsibly disclosing this issue.

Small Features and Improvements

• New Artisan command to reindex switch ports' ifIndex based on ifName. This is useful when a port's ifIndex changes in a switch operating system update. See this documentation for more information.
  • And suplemantal to this, we can now also exclude a switch from polling (via 00ccf4d).
• IX-F Member Export: improvements include: (7286616)
  • Provides a more user friendly error message if the schema-required IX-F IXP ID is not set.
  • Allows the poller to provide an IX-F ID per infrastructure if one is not set via the parameter: &ixfid_1=xx&ixfid_2=yy.
  • Allows the poller to ignore the missing IX-F ID and set it to zero via the parameter: ?ignore_missing_ixfid=1.
  • Tag IXP Manager as the generator of the IX-F JSON document (4185fe6)
• Better member logo layouts (c10c712) and option to add a background colour to check transparency (8a0ce56)

Bug Fixes

• Can not update IRRDB if only IPv6 is configured. #662
• Insufficient permissions error downloading crossconnect documents #663
• VLAN Tagging should be warned/enforced when >2 vlan interfaces exist #667
• ASN max length too short in IRRDB database due to the 32-bit ASN integer representation in database being signed - fixes #664
• Admin log on as this user updates last login date when it shouldn't - fixes #652
• Rack field in patch panel port verification page is blank (f95a893)
• Off by one couting issue for admin dashboard - ports by location (4a10448)