Status/2024Q4/pkg-pkgbase.adoc: Add report

Sponsored by:	The FreeBSD Foundation

website/content/en/status/report-2024-10-2024-12/pkg-pkgbase.adoc

@@ -0,0 +1,32 @@
+=== PkgBase-motivated improvements to pkg
+
+Contact: Isaac Freund <ifreund@freebsdfoundation.org>
+
+Some problems blocking progress on the link:https://wiki.freebsd.org/PkgBase[PkgBase project] are caused by shortcomings of man:pkg[8].
+The primary goal of my work on pkg is to unblock PkgBase progress.
+However, all users of pkg will benefit even if they do not use PkgBase.
+
+The scheduler for pkg's install/upgrade/delete jobs has been link:https://github.com/freebsd/pkg/pull/2330[rewritten], motivated by solving link:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259785[PR259785].
+The new scheduler models the scheduling problem as a directed graph and splits upgrade jobs into delete/install halves only when necessary to break a cycle in the graph.
+This formal model gives strong guarantees about ordering that the old scheduler was not able to provide and prevents unnecessary splitting of upgrade jobs.
+It also fixes longstanding bugs where the old scheduler would bail out and cause the entire upgrade to fail.
+The new scheduler is included in pkg version 1.21.99.3 (pkg-devel).
+
+The rest of my work this quarter has been related to pkg's automatic tracking of shared library dependencies, which PkgBase heavily relies on.
+The initial motivating problem was link:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265061[PR265061] but it was necessary to make more fundamental changes to how pkg tracks shlibs before cleanly solving that problem became possible.
+
+When a package is created with man:pkg-create[8], pkg scans the included files and generates shlibs_provided/shlibs_required lists based on the executables/shared libraries found.
+Before my changes, pkg would use the elf hints file of the host system as an input to pkg-create in order to filter out shlibs provided by the base system from the generated shlibs_required list.
+An ALLOW_BASE_SHLIBS option disabled this filtering for the purpose of building PkgBase packages.
+After my changes, pkg-create no longer reads the elf hints file of the host system and base system shlibs are included in the generated shlibs_required list.
+When man:pkg-install[8]/man:pkg-upgrade[8]/etc. invoke the solver on an non-PkgBase system, pkg generates a list of shlibs provided by the base system as an input to the solver by scanning /lib and /usr/lib.
+On a PkgBase system, the PkgBase packages provide all base system shlibs.
+This allows the ALLOW_BASE_SHLIBS option to be eliminated.
+It also gives better integration between the ports packages and PkgBase packages as shlib dependencies of ports packages on PkgBase packages are now tracked rather than ignored.
+Finally, this change significantly simplifies the pkg codebase and improves portability.
+This change was implemented in link:https://github.com/freebsd/pkg/pull/2386 and is not yet included in a pkg release.
+
+With that change and other link:https://github.com/freebsd/pkg/pull/2376[internal improvements] I was able to add support for tracking lib32 and Linuxulator shlibs, which should resolve the problem that originally motivated my work on pkg's shlib handling (link:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265061[PR265061]).
+This support is implemented in link:https://github.com/freebsd/pkg/pull/2387 and is not yet included in a pkg release.
+
+Sponsor: The FreeBSD Foundation