[next-devel] rebase to Fedora 33 #640

dustymabe · 2020-09-24T15:41:14Z

These commits perform the rebase to Fedora 33.

Part of the work for coreos/fedora-coreos-tracker#611

dustymabe · 2020-09-24T15:42:27Z

Right now zincati isn't in the f33 repos so you'll need to download the latest rpm for f33 into the overrides/rpm/ folder before doing a loca cosa build with:

$ cosa fetch --with-cosa-overrides && cosa build

dustymabe · 2020-10-01T13:53:51Z

current important changes when building locally right after building f32:

Removed:                                                                                                              
  dbxtool-8-12.fc32.x86_64                                                                                            
  libdb-utils-5.3.28-40.fc32.x86_64                                                                                                                                                                                                          
  mozjs60-60.9.0-5.fc32.x86_64                             
  ncurses-6.1-15.20191109.fc32.x86_64                                                                                                                                                                                                        
Added:                                                                                                                
  ModemManager-glib-1.14.2-1.fc33.x86_64                                                                                                                                                                                                     
  dmidecode-1:3.2-8.fc33.x86_64                                                                                                                                                                                                              
  fedora-release-identity-coreos-33-0.15.noarch            
  fedora-repos-modular-33-0.13.noarch                                                                                                                                                                                                        
  flashrom-1.2-3.fc33.x86_64                                                                                          
  fwupd-1.4.6-1.fc33.x86_64                                
  glib-networking-2.66.0-1.fc33.x86_64                                                                                
  gsettings-desktop-schemas-3.38.0-1.fc33.x86_64                                                                                                                                                                                             
  hwdata-0.339-1.fc33.noarch                                                                                          
  libeconf-0.3.8-4.fc33.x86_64                                                                                        
  libftdi-1.4-3.fc33.x86_64                                                                                           
  libgcab1-1.4-3.fc33.x86_64                                                                                          
  libgudev-234-1.fc33.x86_64                                                                                                                                                                                                                 
  libgusb-0.3.5-1.fc33.x86_64                                                                                                                                                                                                                
  libibverbs-31.0-1.fc33.x86_64                                                                                                                                                                                                              
  libjcat-0.1.3-3.fc33.x86_64                                                                                                                                                                                                                
  libmbim-1.24.2-1.fc33.x86_64                                                                                                                                                                                                               
  libmodman-2.0.1-23.fc33.x86_64                                                                                      
  libproxy-0.4.15-24.fc33.x86_64                                                                                      
  libqmi-1.26.4-1.fc33.x86_64                                                                                                                                                                                                                
  libsmbios-2.4.2-10.fc33.x86_64                                                                                                                                                                                                             
  libsoup-2.72.0-3.fc33.x86_64                                                                                                                                                                                                               
  libxmlb-0.2.1-1.fc33.x86_64                                                                                         
  mozjs78-78.3.0-1.fc33.x86_64                                                                                        
  pciutils-3.6.4-2.fc33.x86_64                                                                                        
  pciutils-libs-3.6.4-2.fc33.x86_64                                                                                                                                                                                                          
  rdma-core-31.0-1.fc33.x86_64                                                                                                                                                                                                               
  samba-libs-2:4.13.0-11.fc33.x86_64                                                                                  
  shared-mime-info-2.0-3.fc33.x86_64

We can discuss each one of these in the file diff as well.

dustymabe · 2020-10-01T13:59:51Z

Right now zincati isn't in the f33 repos so you'll need to download the latest rpm for f33 into the overrides/rpm/ folder before doing a loca cosa build with:
$ cosa fetch --with-cosa-overrides && cosa build

This is fixed now with https://bodhi.fedoraproject.org/updates/FEDORA-2020-2fbcdf6262

dustymabe · 2020-10-01T17:14:57Z

manifest-lock.x86_64.json

@@ -1,1145 +1,1220 @@
 {
  "packages": {
+    "ModemManager-glib": {


needed by fwupd

$ rpm -e ModemManager-glib warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: libmm-glib.so.0()(64bit) is needed by (installed) fwupd-1.4.6-1.fc33.x86_64

dustymabe · 2020-10-01T17:17:28Z

manifest-lock.x86_64.json

    },
    "dbus-libs": {
-      "evra": "1:1.12.20-1.fc32.x86_64"
-    },
-    "dbxtool": {


in Fedora 32 this was needed by shim:

[core@localhost ~]$ rpm -e dbxtool error: Failed dependencies: dbxtool >= 0.6-3 is needed by (installed) shim-x64-15-8.x86_64

in Fedora 33 dbxtool is provided by the fwupd rpm and not the dbxtool rpm.

It's not clear to me that shim should hard require fwupd.

Now...shipping an OS without a way to update the DBX is probably not a good idea, but fwupd is now doing a lot.

dustymabe · 2020-10-01T17:18:57Z

manifest-lock.x86_64.json

    },
    "diffutils": {
      "evra": "3.7-4.fc32.x86_64"
    },
+    "dmidecode": {


needed by flashrom which is needed by fwupd:

```$ rpm -e dmidecode warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: dmidecode is needed by (installed) flashrom-1.2-3.fc33.x86_64

dustymabe · 2020-10-01T17:20:14Z

manifest-lock.x86_64.json

-      "evra": "32-3.noarch"
+      "evra": "33-0.15.noarch"
+    },
+    "fedora-release-identity-coreos": {


new package not sure how it automagically gets installed but it looks like we do want it.

I think it's coming from fedora-release-common requiring fedora-release-identity which fedora-release-identity-coreos provides. But yeah, seems harmless.

dustymabe · 2020-10-01T17:20:36Z

manifest-lock.x86_64.json

-      "evra": "32-6.noarch"
+      "evra": "33-0.13.noarch"
+    },
+    "fedora-repos-modular": {


new package. fedora-repos-modular was broken into a subpackage

Related: we should sanity-check our disablement still works there.

yeah. it does I think. Part of the reason I knew I needed to add them back is because the "disablement" code was failing :)

dustymabe · 2020-10-01T19:34:26Z

manifest-lock.x86_64.json

+    "pciutils": {
+      "evra": "3.6.4-2.fc33.x86_64"
+    },
+    "pciutils-libs": {


needed by pciutils (leads to iptables) and flashrom (leads to fwupd):

[core@cosa-devsh ~]$ rpm -e pciutils-libs warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: libpci.so.3()(64bit) is needed by (installed) pciutils-3.6.4-2.fc33.x86_64 libpci.so.3()(64bit) is needed by (installed) flashrom-1.2-3.fc33.x86_64 libpci.so.3(LIBPCI_3.0)(64bit) is needed by (installed) pciutils-3.6.4-2.fc33.x86_64 libpci.so.3(LIBPCI_3.0)(64bit) is needed by (installed) flashrom-1.2-3.fc33.x86_64 libpci.so.3(LIBPCI_3.1)(64bit) is needed by (installed) pciutils-3.6.4-2.fc33.x86_64 libpci.so.3(LIBPCI_3.3)(64bit) is needed by (installed) pciutils-3.6.4-2.fc33.x86_64 libpci.so.3(LIBPCI_3.3)(64bit) is needed by (installed) flashrom-1.2-3.fc33.x86_64 libpci.so.3(LIBPCI_3.5)(64bit) is needed by (installed) pciutils-3.6.4-2.fc33.x86_64 libpci.so.3(LIBPCI_3.5)(64bit) is needed by (installed) flashrom-1.2-3.fc33.x86_64 libpci.so.3(LIBPCI_3.6)(64bit) is needed by (installed) pciutils-3.6.4-2.fc33.x86_64 libpci.so.3(LIBPCI_3.7)(64bit) is needed by (installed) pciutils-3.6.4-2.fc33.x86_64 pciutils-libs = 3.6.4-2.fc33 is needed by (installed) pciutils-3.6.4-2.fc33.x86_64

dustymabe · 2020-10-01T19:37:12Z

manifest-lock.x86_64.json

-      "evra": "2:2.1.0-2.fc32.x86_64"
+      "evra": "2:2.1.0-0.179.dev.git43f2771.fc33.x86_64"
+    },
+    "podman-plugins": {


will be dropped soon when new build of podman hits stable that only recommends and doesn't require podman-plugins.

dustymabe · 2020-10-01T19:38:01Z

manifest-lock.x86_64.json

-      "evra": "4.0.2-5.fc32.x86_64"
+      "evra": "4.0.2-6.fc33.x86_64"
+    },
+    "rdma-core": {


ultimately needed by iptables:

[core@cosa-devsh ~]$ rpm -e rdma-core warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: rdma-core(x86-64) = 31.0-1.fc33 is needed by (installed) libibverbs-31.0-1.fc33.x86_64 [core@cosa-devsh ~]$ rpm -e libibverbs warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: libibverbs.so.1()(64bit) is needed by (installed) libpcap-14:1.9.1-5.fc33.x86_64 libibverbs.so.1(IBVERBS_1.0)(64bit) is needed by (installed) libpcap-14:1.9.1-5.fc33.x86_64 libibverbs.so.1(IBVERBS_1.1)(64bit) is needed by (installed) libpcap-14:1.9.1-5.fc33.x86_64 [core@cosa-devsh ~]$ rpm -e libpcap warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: libpcap.so.1()(64bit) is needed by (installed) iptables-libs-1.8.5-3.fc33.x86_64 libpcap.so.1()(64bit) is needed by (installed) nmap-ncat-2:7.80-5.fc33.x86_64 [core@cosa-devsh ~]$ rpm -e iptables-libs warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: iptables-libs(x86-64) = 1.8.5-3.fc33 is needed by (installed) iptables-1.8.5-3.fc33.x86_64 iptables-libs(x86-64) = 1.8.5-3.fc33 is needed by (installed) iptables-nft-1.8.5-3.fc33.x86_64 libip4tc.so.2()(64bit) is needed by (installed) systemd-246.6-3.fc33.x86_64 libip4tc.so.2()(64bit) is needed by (installed) iptables-1.8.5-3.fc33.x86_64 libip6tc.so.2()(64bit) is needed by (installed) iptables-1.8.5-3.fc33.x86_64 libxtables.so.12()(64bit) is needed by (installed) iptables-1.8.5-3.fc33.x86_64 libxtables.so.12()(64bit) is needed by (installed) iproute-tc-5.8.0-1.fc33.x86_64 libxtables.so.12()(64bit) is needed by (installed) nftables-1:0.9.3-6.fc33.x86_64 libxtables.so.12()(64bit) is needed by (installed) iptables-nft-1.8.5-3.fc33.x86_64

dustymabe · 2020-10-01T19:39:04Z

manifest-lock.x86_64.json

-      "evra": "2:4.12.7-0.fc32.x86_64"
+      "evra": "2:4.13.0-11.fc33.x86_64"
+    },
+    "samba-libs": {


new requirement of samba-client-libs:

[core@cosa-devsh ~]$ rpm -e samba-libs warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: libdcerpc-server-core.so.0()(64bit) is needed by (installed) samba-client-libs-2:4.13.0-11.fc33.x86_64 libdcerpc-server-core.so.0(DCERPC_SERVER_CORE_0.0.1)(64bit) is needed by (installed) samba-client-libs-2:4.13.0-11.fc33.x86_64

dustymabe · 2020-10-01T19:39:31Z

manifest-lock.x86_64.json

-      "evra": "2:4.8.1-2.fc32.x86_64"
+      "evra": "2:4.8.1-4.fc33.x86_64"
+    },
+    "shared-mime-info": {


fwupd:

[core@cosa-devsh ~]$ rpm -e shared-mime-info warning: Found bdb Packages database while attempting sqlite backend: using bdb backend. error: Failed dependencies: shared-mime-info is needed by (installed) libxmlb-0.2.1-1.fc33.x86_64 shared-mime-info is needed by (installed) fwupd-1.4.6-1.fc33.x86_64

cgwalters · 2020-10-01T19:43:37Z

OK so a lot of this is fwupd which was previously discussed here coreos/fedora-coreos-tracker#449

jlebon

As mentioned, a lot of stuff from fwupd here which I think needs more investigation.

jlebon · 2020-10-01T19:50:34Z

manifest-lock.x86_64.json

-      "evra": "32-3.noarch"
+      "evra": "33-0.15.noarch"
+    },
+    "fedora-release-identity-coreos": {


I think it's coming from fedora-release-common requiring fedora-release-identity which fedora-release-identity-coreos provides. But yeah, seems harmless.

jlebon · 2020-10-01T19:50:54Z

manifest-lock.x86_64.json

-      "evra": "32-6.noarch"
+      "evra": "33-0.13.noarch"
+    },
+    "fedora-repos-modular": {


Related: we should sanity-check our disablement still works there.

jlebon · 2020-10-01T19:59:17Z

manifest-lock.x86_64.json

-      "evra": "0.7.14-1.fc32.x86_64"
+      "evra": "0.7.14-4.fc33.x86_64"
+    },
+    "libsoup": {


This one is unfortunate. We were avoiding pulling in both libcurl and libsoup by having OSTree compiled against the former.

jlebon · 2020-10-01T20:00:18Z

manifest-lock.x86_64.json

    },
    "mpfr": {
-      "evra": "4.0.2-5.fc32.x86_64"
-    },
-    "ncurses": {


Hmm, this has reset which can be super useful for saving a broken terminal (and tput/tset which I think can be useful too though I've only used it a few times). Tempted to add that one back.

dustymabe · 2020-10-02T13:34:49Z

While we still have some deps that we want to investigate as part of this change (for example fwupd) we need to make progress here. We can track down remaining issues and dependencies for f33 while it bakes in next.

The two biggest offenders on bringing in new packages were fwupd (already covered by previously linked issue) and the iptables depchain because libpcap added support for rdma.

dustymabe · 2020-10-02T13:34:59Z

This is ready for final review now.

dustymabe · 2020-10-02T16:18:39Z

issues with podman in F33 (yay) are causing CI failures see: containers/podman#5364 (comment)

jlebon · 2020-10-02T16:41:26Z

manifest-lock.overrides.aarch64.yaml

+  # Fast-track systemd with the networkd subpackage split
+  # <insert link to bodhi update here when it is created>
+  systemd:
+    evra: 246.4-3.fc33.aarch64


This should be 246.6-3, right?

fixed in most recent push

dustymabe · 2020-10-05T15:02:45Z

Force pushed a new update with newer podman.

jlebon · 2020-10-05T16:44:57Z

Re. the root reprovisioning test failures, reproduced it locally and it seems like ignition-ostree-dracut-rootfs is failing with ENOSPC. Even bumping it to 8G fails which tells me something else is going on there. We may need to bump the memory requirement across the board as #658 does, but for now can you just add another exclude while I investigate this?

dustymabe · 2020-10-05T16:56:38Z

Re. the root reprovisioning test failures, reproduced it locally and it seems like ignition-ostree-dracut-rootfs is failing with ENOSPC. Even bumping it to 8G fails which tells me something else is going on there. We may need to bump the memory requirement across the board as #658 does, but for now can you just add another exclude while I investigate this?

SGTM: #665

dustymabe · 2020-10-05T22:41:27Z

ok.. hopefully #667 is the only remaining piece before this starts passing CI and we can merge this.

This also includes updating the `fedora-next` repo so we can pull from the development development location since we are still in development. We also add back in the modular repos, which were split out into a subpackage in Fedora 33. Fixes coreos/fedora-coreos-tracker#611

Bodhi says some of these are stable already but the nightly composes are failing so they aren't actually in the yum repos yet so let's put them in as overrides for now. Also update the generated lockfile with the output of a local run of `cosa fetch --update-lockfile` to pre-seed file with F33 content.

dustymabe · 2020-10-06T18:55:43Z

🤞 that CI passes this time!

jlebon

Nice work on this!

dustymabe · 2020-10-06T19:37:14Z

Yay CI passed! I'm going to drop the last commit from this PR and put them in a new PR against testing-devel (#670) since those files will get synced over by config bot. Since CI already passed I'm going to merge after dropping that last commit.

dustymabe added the hold label Sep 24, 2020

dustymabe force-pushed the dusty-next-f33 branch from 697ffea to 0676ea5 Compare September 24, 2020 15:44

This was referenced Sep 24, 2020

Move the next stream to Fedora 33 coreos/fedora-coreos-tracker#611

Closed

manifest: Include systemd-resolved for F33 change #575

Closed

jlebon mentioned this pull request Sep 30, 2020

next: new release on 2020-10-07 (33.20201006.1.0) coreos/fedora-coreos-streams#197

Closed

35 tasks

dustymabe force-pushed the dusty-next-f33 branch from 0676ea5 to ad37d90 Compare October 1, 2020 13:56

dustymabe commented Oct 1, 2020

View reviewed changes

cgwalters mentioned this pull request Oct 1, 2020

ship fwupd coreos/fedora-coreos-tracker#449

Closed

dustymabe mentioned this pull request Oct 1, 2020

tracker: Fedora 33 rebase work coreos/fedora-coreos-tracker#609

Closed

jlebon reviewed Oct 1, 2020

View reviewed changes

dustymabe force-pushed the dusty-next-f33 branch from 2918c70 to 2b11457 Compare October 2, 2020 13:18

dustymabe removed the hold label Oct 2, 2020

dustymabe changed the title ~~WIP: [next-devel] rebase to Fedora 33~~ [next-devel] rebase to Fedora 33 Oct 2, 2020

cgwalters approved these changes Oct 2, 2020

View reviewed changes

dustymabe force-pushed the dusty-next-f33 branch from 2b11457 to 36f99e0 Compare October 2, 2020 16:23

jlebon reviewed Oct 2, 2020

View reviewed changes

dustymabe force-pushed the dusty-next-f33 branch 2 times, most recently from de26e6a to c372d8e Compare October 2, 2020 17:55

dustymabe mentioned this pull request Oct 2, 2020

F33: chrony.dhcp-propagation test failing coreos/fedora-coreos-tracker#643

Closed

dustymabe force-pushed the dusty-next-f33 branch from c372d8e to 8a8edf7 Compare October 2, 2020 21:39

jlebon force-pushed the dusty-next-f33 branch from 8a8edf7 to c10a801 Compare October 5, 2020 14:39

dustymabe force-pushed the dusty-next-f33 branch from c10a801 to bff87c5 Compare October 5, 2020 15:00

jlebon mentioned this pull request Oct 5, 2020

tests/kola/root-reprovision: bump the amount of RAM to 6G as 4G is not enough on ppc64le #658

Closed

dustymabe mentioned this pull request Oct 5, 2020

F33: ext.config.root-reprovision* tests failing coreos/fedora-coreos-tracker#644

Closed

dustymabe force-pushed the dusty-next-f33 branch from bff87c5 to 80aea54 Compare October 5, 2020 22:40

dustymabe force-pushed the dusty-next-f33 branch from 80aea54 to 4a7272f Compare October 6, 2020 02:33

jlebon force-pushed the dusty-next-f33 branch 4 times, most recently from b6beb54 to d57a787 Compare October 6, 2020 15:37

dustymabe added 3 commits October 6, 2020 14:54

lockfiles: reset lockfiles to start fresh for f33

2c96f57

dustymabe force-pushed the dusty-next-f33 branch from d57a787 to 589aeba Compare October 6, 2020 18:55

jlebon approved these changes Oct 6, 2020

View reviewed changes

dustymabe force-pushed the dusty-next-f33 branch from 589aeba to de7e7ad Compare October 6, 2020 19:37

dustymabe merged commit 2addf27 into coreos:next-devel Oct 6, 2020

dustymabe deleted the dusty-next-f33 branch October 6, 2020 19:37

[next-devel] rebase to Fedora 33 #640

[next-devel] rebase to Fedora 33 #640

Conversation

dustymabe commented Sep 24, 2020

dustymabe commented Sep 24, 2020

dustymabe commented Oct 1, 2020 • edited Loading

dustymabe commented Oct 1, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cgwalters commented Oct 1, 2020

jlebon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dustymabe commented Oct 2, 2020

dustymabe commented Oct 2, 2020

dustymabe commented Oct 2, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dustymabe commented Oct 5, 2020

jlebon commented Oct 5, 2020

dustymabe commented Oct 5, 2020

dustymabe commented Oct 5, 2020

dustymabe commented Oct 6, 2020

jlebon left a comment

Choose a reason for hiding this comment

dustymabe commented Oct 6, 2020

dustymabe commented Oct 1, 2020 •

edited

Loading