New storage proposal

doc/auto_storage.md

@@ -37,8 +37,8 @@ Storage
  volumeGroups <VolumeGroup[]>
  mdRaids <MdRaid[]>
  btrfsRaids <BtrfsRaid[]>
- bcacheDevices <BCache[]>
  nfsMounts <NFS[]>
+ boot [BootSettings]
  guided <Guided>
 ```
 
@@ -67,6 +67,7 @@ Drive
  format [<FormatAction>]
  mount [<MountAction>]
  ptableType [<string>]
+ space <'delete'|'resize'|'keep'>
  partitions [<Partition[]>]
 
 VolumeGroup
@@ -89,6 +90,7 @@ MdRaid
  format [<FormatAction>]
  mount [<MountAction>]
  ptableType [<string>]
+ space <'delete'|'resize'|'keep'>
  partitions [<Partition[]>]
  delete [<boolean=false>]
 
@@ -162,6 +164,10 @@ Size <'default'|string|SizeRange>
 SizeRange
  min <string>
  max <string>
+
+BootSettings
+ configure <boolean>
+ device <string|Search>
 ```
 
 To illustrate how all that fits together, let's see the following example in which the first disk of
@@ -420,6 +426,41 @@ above, it would be possible to use the key as name of the property, resulting in
 }
 ```
 
+## Making Space and Specifying what to do with Existing Partitions (under discussion)
+
+The `space` subsection of each drive or RAID can be used to specify what to do with existing
+partitions, if any. That can also be combined with more specific actions indicated by "searching" a
+given partition within the `partitions` subsection.
+
+Theoretically, we could do the following actions for each given partition during the algorithm
+execution. We just need to decide how to define all that using `space` and `partitions`.
+
+- Delete the partition (mandatory action executed as soon as we process the disk)
+- Shrink the partition to a given size (same than above, note we need to clearly define the way to
+ specify a partition must be resized or grown and likely both things will happen at different
+ stages of the algorithm).
+- Shrink the partition if needed (optional action done if needed and with a calculated target size).
+- Delete the partition if needed (optional action done if needed).
+- Shrink or delete if needed (first an optional resize will be attemped, deleting if it's not
+ enough).
+
+Note also we may need to consider which resize actions are possible depending on the content of the
+partition, the filesystem type, etc.
+
+Maybe a `space` action is not needed since the same behavior can be specified only using something
+like this:
+
+```json
+"storage": {
+ "drives": [
+ {
+ "partitions":
+ { "search": {}, "delete": true }
+ }
+ ]
+}
+```
+
 ## Referencing Other Devices
 
 Sometimes is necessary to reference other devices as part of the specification of an LVM volume
@@ -518,28 +559,21 @@ system (so the same conditions can be matched by a disk, a partition, an LVM dev
 
 ## Partitions needed for Booting
 
-When relying on the Agama proposal (see below), there are some options to configure whether (and
-where) Agama should calculate and create the extra partitions needed for booting.
-
-If the proposal is not used, Agama will always try to calculate and create those partitions taking
-the location of the root file system as a reference. That's the same approach that AutoYaST has
-followed for years.
+The `boot` section can be used to configure whether (and where) Agama should calculate and create
+the extra partitions needed for booting. If the device is not specified, Agama will take the
+location of the root file system as a reference.
 
 ## Using the Automatic Proposal
 
 Agama can rely on the process known as Guided Proposal to calculate all the needed partitions, LVM
-devices and file systems based on some general product settings and some user preferences. That
-mechanism can also be used as part of the profile and will be executed as a last step, after
-processing all the explicit sections that describe devices.
+devices and file systems based on some general product settings and some user preferences.
 
 The `guided` section conforms to the following specification.
 
 ```
 Guided
  device [TargetDevice]
- boot [BootSettings]
  encryption [EncryptionSettings]
- space <'delete'|'resize'|'keep'>
  volumes [Volume[]]
 
 TargetDevice <string|TargetDisk|TargetNewLvm|TargetReusedLvm>
@@ -553,10 +587,6 @@ TargetNewLvm
 TargetReusedLvm
  reusedLvmVg <string|Search>
 
-BootSettings
- configure <boolean>
- device <string|Search>
-
 EncryptionSettings
  password <string>
  method <string>

service/lib/agama/storage/config.rb

@@ -0,0 +1,196 @@
+# frozen_string_literal: true
+
+# Copyright (c) [2024] SUSE LLC
+#
+# All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of version 2 of the GNU General Public License as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+# more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, contact SUSE LLC.
+#
+# To contact SUSE LLC about this file by physical or electronic mail, you may
+# find current contact information at www.suse.com.
+
+require "agama/storage/configs"
+
+module Agama
+ module Storage
+ # Settings used to calculate an storage proposal.
+ class Config
+ # Boot settings.
+ #
+ # @return [Configs::Boot]
+ attr_accessor :boot
+
+ # @return [Array<Configs::Drive>]
+ attr_accessor :drives
+
+ # @return [Array]
+ attr_accessor :volume_groups
+
+ # @return [Array]
+ attr_accessor :md_raids
+
+ # @return [Array]
+ attr_accessor :btrfs_raids
+
+ # @return [Array]
+ attr_accessor :nfs_mounts
+
+ def initialize
+ @boot = Configs::Boot.new
+ @drives = []
+ @volume_groups = []
+ @md_raids = []
+ @btrfs_raids = []
+ @nfs_mounts = []
+ end
+
+ # Creates a config from JSON hash according to schema.
+ #
+ # @param config_json [Hash]
+ # @param product_config [Agama::Config]
+ #
+ # @return [Storage::Config]
+ def self.new_from_json(config_json, product_config:)
+ ConfigConversions::FromJSON.new(config_json, product_config: product_config).convert
+ end
+
+ # Name of the device that will presumably be used to boot the target system
+ #
+ # @return [String, nil] nil if there is no enough information to infer a possible boot disk
+ def boot_device
+ explicit_boot_device || implicit_boot_device
+ end
+
+ # Device used for booting the target system
+ #
+ # @return [String, nil] nil if no disk is explicitly chosen
+ def explicit_boot_device
+ return nil unless boot.configure?
+
+ boot.device
+ end
+
+ # Device that seems to be expected to be used for booting, according to the drive definitions
+ #
+ # @return [String, nil] nil if the information cannot be inferred from the list of drives
+ def implicit_boot_device
+ # NOTE: preliminary implementation with very simplistic checks
+ root_drive = drives.find do |drive|
+ drive.partitions.any? { |p| p.filesystem.root? }
+ end
+
+ root_drive&.found_device&.name
+ end
+
+ # Sets min and max sizes for all partitions and logical volumes with default size
+ #
+ # @param volume_builder [VolumeTemplatesBuilder] used to check the configuration of the
+ # product volume templates
+ def calculate_default_sizes(volume_builder)
+ default_size_devices.each do |dev|
+ dev.size.min = default_size(dev, :min, volume_builder)
+ dev.size.max = default_size(dev, :max, volume_builder)
+ end
+ end
+
+ private
+
+ # return [Array<Configs::Filesystem>]
+ def filesystems
+ (drives + partitions).map(&:filesystem).compact
+ end
+
+ # return [Array<Configs::Partition>]
+ def partitions
+ drives.flat_map(&:partitions)
+ end
+
+ # return [Array<Configs::Partitions>]
+ def default_size_devices
+ partitions.select { |p| p.size&.default? }
+ end
+
+ # Min or max size that should be used for the given partition or logical volume
+ #
+ # @param device [Configs::Partition] device configured to have a default size
+ # @param attr [Symbol] :min or :max
+ # @param builder [VolumeTemplatesBuilder] see {#calculate_default_sizes}
+ def default_size(device, attr, builder)
+ path = device.filesystem&.path || ""
+ vol = builder.for(path)
+ return fallback_size(attr) unless vol
+
+ # Theoretically, neither Volume#min_size or Volume#max_size can be nil
+ # At most they will be zero or unlimited, respectively
+ return vol.send(:"#{attr}_size") unless vol.auto_size?
+
+ outline = vol.outline
+ size = size_with_fallbacks(outline, attr, builder)
+ size = size_with_ram(size, outline)
+ size_with_snapshots(size, device, outline)
+ end
+
+ # TODO: these are the fallbacks used when constructing volumes, not sure if repeating them
+ # here is right
+ def fallback_size(attr)
+ return Y2Storage::DiskSize.zero if attr == :min
+
+ Y2Storage::DiskSize.unlimited
+ end
+
+ # @see #default_size
+ def size_with_fallbacks(outline, attr, builder)
+ fallback_paths = outline.send(:"#{attr}_size_fallback_for")
+ missing_paths = fallback_paths.reject { |p| proposed_path?(p) }
+
+ size = outline.send(:"base_#{attr}_size")
+ missing_paths.inject(size) { |total, p| total + builder.for(p).send(:"#{attr}_size") }
+ end
+
+ # @see #default_size
+ def size_with_ram(initial_size, outline)
+ return initial_size unless outline.adjust_by_ram?
+
+ [initial_size, ram_size].max
+ end
+
+ # @see #default_size
+ def size_with_snapshots(initial_size, device, outline)
+ return initial_size unless device.filesystem.btrfs_snapshots?
+ return initial_size unless outline.snapshots_affect_sizes?
+
+ if outline.snapshots_size && outline.snapshots_size > DiskSize.zero
+ initial_size + outline.snapshots_size
+ else
+ multiplicator = 1.0 + (outline.snapshots_percentage / 100.0)
+ initial_size * multiplicator
+ end
+ end
+
+ # Whether there is a separate filesystem configured for the given path
+ #
+ # @param path [String, Pathname]
+ # @return [Boolean]
+ def proposed_path?(path)
+ filesystems.any? { |fs| fs.path?(path) }
+ end
+
+ # Return the total amount of RAM as DiskSize
+ #
+ # @return [DiskSize] current RAM size
+ def ram_size
+ @ram_size ||= Y2Storage::DiskSize.new(Y2Storage::StorageManager.instance.arch.ram_size)
+ end
+ end
+ end
+end

service/lib/agama/storage/config_conversions.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+# Copyright (c) [2024] SUSE LLC
+#
+# All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of version 2 of the GNU General Public License as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+# more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, contact SUSE LLC.
+#
+# To contact SUSE LLC about this file by physical or electronic mail, you may
+# find current contact information at www.suse.com.
+
+require "agama/storage/config_conversions/block_device"
+require "agama/storage/config_conversions/drive"
+require "agama/storage/config_conversions/encrypt"
+require "agama/storage/config_conversions/filesystem"
+require "agama/storage/config_conversions/format"
+require "agama/storage/config_conversions/from_json"
+require "agama/storage/config_conversions/mount"
+require "agama/storage/config_conversions/partition"
+require "agama/storage/config_conversions/partitionable"
+require "agama/storage/config_conversions/size"
+
+module Agama
+ module Storage
+ # Conversions for the storage config.
+ module ConfigConversions
+ end
+ end
+end

service/lib/agama/storage/config_conversions/block_device.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+# Copyright (c) [2024] SUSE LLC
+#
+# All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of version 2 of the GNU General Public License as published
+# by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+# more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, contact SUSE LLC.
+#
+# To contact SUSE LLC about this file by physical or electronic mail, you may
+# find current contact information at www.suse.com.
+
+require "agama/storage/config_conversions/block_device/from_json"
+
+module Agama
+ module Storage
+ module ConfigConversions
+ # Conversions for block device.
+ module BlockDevice
+ end
+ end
+ end
+end