update minio to 2022-10-24T18-35-07Z #5555

fgma · 2023-01-08T20:27:23Z

Description

Update minio to 2022-10-24T18-35-07Z. This is the last release supporting the legacy FS backend. Details see #5549

Checklist

Build rule all-supported completed successfully
New installation of package completed successfully
Package upgrade completed successfully (Manually install the package again)
Package functionality was tested
Any needed documentation is updated/created

Type of change

hgy59 · 2023-01-21T18:49:22Z

@fgma as #5430 is merged, it is possible to have space (and other special characters) in minio password entered in the install- or upgrade-wizard.
The format of the custom environment-variables has changed and I have updated the changelog in spk/minio/Makefile. Please keep this text when resolving the conflict for this tile.

fgma · 2023-01-21T19:38:17Z

Sorry, I've made a mistake and messed up the merge.

hgy59 · 2023-01-21T19:47:24Z

Sorry, I've made a mistake and messed up the merge.

Yes, I think you still want to update to 2022-10-24T18-35-07Z for compatibility with current package.

fgma · 2023-01-21T20:25:59Z

Should be OK now.

fgma · 2023-02-22T19:19:30Z

Anything left here?

hgy59 · 2023-02-22T19:49:02Z

Anything left here?

We should add some information to the changelog, regarding that future versions will not be compatible with this one.
And we should know how the update/migration will work.

fgma · 2023-02-24T19:57:49Z

I've updated the changelog. My idea was to get this update out so everyone has at least this version and new installations will not need a migration. Migration is a separate step for me.

Any ideas on how we should handle this? I don't see any chance doing this automatically because one would need the same amount of temporary storage used by data stored in minio (at least in the biggest bucket) which is not possible to guarantee.

fgma · 2023-09-02T21:55:28Z

I've upgraded my whole setup to this version for production use. How are we going to continue from here? I'd like to continue upgrading to the latest minio release as soon as possible.

th0ma7 · 2023-09-04T20:21:03Z

@fgma and @hgy59 is there a hard requirement to #5649 or can this be merged and published as-is?

fgma · 2023-09-04T20:29:15Z

@th0ma7 Not for me. I've used the manual shared folder creation on my system as a workaround.

hgy59 · 2023-09-04T20:29:38Z

@fgma and @hgy59 is there a hard requirement to #5649 or can this be merged and published as-is?

The target of #5649 will be an independent solution to fix the shared folders.
By independent I mean that I will have to create new Makefile variables to not touch existing implementations.
So this is not a blocker for any package that use the old version.
(but AFAICR @fgma wants to have the new version to benefit from the feature to create shares on diferent volumes in advance).

fgma · 2023-09-04T20:44:28Z

@hgy59 I'm don't really need #5649 but I want to upgrade to the latest minio release which uses a new data format and the release in this PR ist the last one which supports the old format and the new format at the same time. For details have a look at #5549

fgma · 2023-11-02T19:45:53Z

Anything left for me to do do get this finally merged?

The currently available version has at least two problems:

No security updates for over a year.
Everyone who stores data in the old data format has to do a manual migration at some point in time. The longer this update takes the more users are affected.

We need a strategy to get this finally resolved.

th0ma7 · 2023-11-04T18:30:42Z

I may be able to provide a review over the weekend although from a strategy standpoint it needs a volunteer to take it over (unless that would be you, haven't followed this thread in particular)

fgma · 2023-11-04T20:01:40Z

I can't decide how the migration path should look like. I've opened a thread for discussing the issue almost a year ago which received no reaction at all: #5549

After doing my own migration I'm pretty sure any automation would result in a disaster. It took me multiple weeks for a few terrabytes because tools would timeout all the time and miss file etc. Maybe we need to create a new package or something like that.

Getting this update out would at least would save new users from a migration their data.

th0ma7

LGTM

th0ma7 · 2023-11-05T23:33:58Z

So considering the other changes are in (thnx @hgy59) i guess this is a rather trivial update. What I would suggest is adding a wizard with a single page to explain that this will be the LAST version compatible with the older format. You would need to add that as well to the changelog so it is more obvious as info would be hands-on even prior to installing.

On top of that, I'd suggest creating a "new" package for the "new" format, perhaps called minio-mc or something similar to make a distinction between the two. So for the older package you clearly state that this is the last version... then in the new package you explain that this is the newer one.

Guessing there will be issues to be sorted out relatively to ports in usage or else? You could then create a wiki page in spksrc github section to explain how to migrate from minio to minio-mc ...

Otherwise, as-is it looks fine. The above proposal can be done at a later while creating the "new" package where the older one can be re-published with the added wizard and info in the changelog.

My two cents...

fgma · 2023-11-10T21:03:35Z

I've added a step in the wizard explaining the situation to the user. This ready for me.

After the merge I would create a new package based on a copy of the released version and upgrade it to the current minio release.

spk/minio/src/wizard/install_uifile

spk/minio/src/wizard/upgrade_uifile.sh

hgy59 · 2023-11-11T11:24:45Z

@fgma sorry for my late review

Looking at the details of the migration I got some new ideas.

This minio package provides the wizard options to chose the shared folder independent of new installation or package update. In either case, you can use a folder with or without a minio configuration.

Since the current version and the version in this PR supports the legacy FS and the SNSD (Single-Node Single-Drive) mode, we should now use the SNSD-Mode when the shared folder does not contain a minio configuration.

The installer could check whether the shared folder already contains a minio configuration and either configure minio server for legacy FS or SNSD mode.
If the existing configuration is not already SNSD, this Minio version (2022-10-24T18-35-07Z) will display a deprecation warning in the installation wizard, and future versions will refuse the installation without former migration.

I suppose the minio server mode can be recognized with the file .minio.sys/format.json in the shared folder. For the legacy FS mode this looks like:

{
  "version": "1",
  "format": "fs",
  "id": "05583150-3d04-4058-bc30-7e8600932b87",
  "fs": {
    "version": "2"
  }
}

I highly recommend that this update uses the SNSD mode by default so it is useful for the migration of existing installations.

@fgma what do you think about it?

EDIT:
There is no need to configure the minio server for SNSD. When a new shared folder is selected, the the file .minio.sys/format.json looks like this:

{
  "version": "1",
  "format": "xl-single",
  "id": "83ac901d-9fc8-42a5-afd8-dc2e694795bb",
  "xl": {
    "version": "3",
    "this": "46b08f42-a295-4b04-9d43-a8d99dddfc8d",
    "sets": [
      [
        "46b08f42-a295-4b04-9d43-a8d99dddfc8d"
      ]
    ],
    "distributionAlgo": "SIPMOD+PARITY"
  }
}

So for future updates there is no need to create a new package (with a different name), but the installer must refuse the installation/update when the selected shared folder exists and contains an incompatible (legacy FS) configuration.

hgy59 · 2023-11-11T12:07:13Z

@fgma The minio console displays the version DEVELOPMENT.GOGET under metrics - info - server.
Isn't there a way to build an official version that displays 2022-10-24T18-35-07Z?

hgy59 · 2023-11-11T12:15:02Z

@fgma can you rebase this PR to current master so we have #5649 to apply the redesigned shared folder handling?

fgma · 2023-11-14T23:19:33Z

@hgy59 I'll try to get the fixes done during the next days.

Concerning bucket versioning: I've verified what you already noticed. When creating a new install even older versions of minio already create SNSD based data by default for a new installation. I've just tested this on version 2022-10-24T18-35-07Z and 2022-09-07T22-25-02Z (current version packaged) using official minio binaries on windows. The issue is mainly a problem for all long term users already having data stored inside minio using quite old releases for the initial install.

How does one gracefully stop an installation? We would need to dynamically detect the data version after the user has selected the storage location and then decide if we can continue or stop with an error. Is there is something similar in the codebase already?

fgma · 2023-11-18T20:12:55Z

@fgma The minio console displays the version DEVELOPMENT.GOGET under metrics - info - server. Isn't there a way to build an official version that displays 2022-10-24T18-35-07Z?

@fgma can you rebase this PR to current master so we have #5649 to apply the redesigned shared folder handling?

fixed in 6727bb7

fgma · 2023-12-04T19:10:37Z

Any feedback on this?

d8x · 2024-03-26T11:34:56Z

How can we move forward with this?

hgy59 · 2024-03-26T20:38:00Z

This PR is lacking migration to redesigned shared folder handling.

SERVICE_WIZARD_SHARE = wizard_data_directory
USE_DATA_SHARE_WORKER = yes

should be replaced by

SERVICE_WIZARD_SHARENAME = wizard_data_directory

Then the install_uifile has to be adjusted (remove volume configuration)
and service-setup.sh must use SHARE_PATH variable instead of wizard_data_directory

probably it is better to rename the variable wizard_data_directory to wizard_share_name to reflect the fact that it is the name of the share and not a full path anymore.

finally upgrade_uifile.sh and service-setup.sh have to be adjusted for the shared folder.

There is a good example of modifications in spk/owncloud/ by @mreid-tt

hgy59 · 2024-03-26T20:50:36Z

How does one gracefully stop an installation? We would need to dynamically detect the data version after the user has selected the storage location and then decide if we can continue or stop with an error. Is there is something similar in the codebase already?

Sorry for my late reply.
There are the functions validate_preinst and validate_preupgrade that can be used for this kind of validation.

Just echo an "ERROR TEXT" and use "exit 1" in a validate function to refuse the installation or upgrade of the package (you find commented examples in spk/demoservice/src/service-setup.sh).

All such functions (service_* and validate_*) are called after wizard pages are completed and the installation or upgrade is started.

… minio-2023-01-02

- adjust wizard for shared folders - use SHARE_PATH variable instead of WIZARD_DATA_VOLUME and WIZARD_DATA_DIRECTORY

- add notes to the installation wizard that the "Migration notes" apply only when the shared folder already contains a minio data structure - evaluate existing data format in upgrade wizard and show related "Migration notes" - reusing an existing installation with the new format will have no upgrade issues - reusing an existing installation with the filesystem format will not get any future update - when changing the shared folder on upgrade, the data format is unknown and it is unknown whether future updates will work - add link to the 'Migrate from Gateway or Filesystem Mode' page

hgy59 · 2024-09-08T01:00:01Z

@fgma glad to tell you that this old pending PR is about to be ready to merge.

The redesinged shared folder handling is implemented and wizards are adjusted.

I shortened the Information about migration in the wizards and created a wiki page showing the detailed migration procedure.
(https://github.com/SynoCommunity/spksrc/wiki/MinIO)

Future versions only need a validation of the data format to refuse installation/upgrade if FS format is found in the shared folder.

I know you can't test this since you've already migrated your installation, but having a second set of eyes on it is still welcome.

th0ma7

@hgy59 just got back, I havent build & tested but overall changes looked good to me. And makes me wonder if the vcs` related build options wouldn't be of interest to use for one of my other PR relatively to IGC stack.

IngmarStein · 2024-10-17T19:04:02Z

Is this ready to go (followed by #6221)?

fgma force-pushed the minio-2023-01-02 branch from 5ec0b5f to bb20bb4 Compare January 21, 2023 19:24

fgma force-pushed the minio-2023-01-02 branch from bb20bb4 to 70cba4a Compare January 21, 2023 20:16

hgy59 mentioned this pull request Jan 22, 2023

Fix compatibility with POSIX shell in SRM (ash) #5570

Merged

10 tasks

hgy59 mentioned this pull request Feb 27, 2023

redesign shared folder handling #5649

Merged

6 tasks

th0ma7 approved these changes Nov 5, 2023

View reviewed changes

hgy59 reviewed Nov 11, 2023

View reviewed changes

spk/minio/src/wizard/install_uifile Outdated Show resolved Hide resolved

hgy59 reviewed Nov 11, 2023

View reviewed changes

spk/minio/src/wizard/upgrade_uifile.sh Outdated Show resolved Hide resolved

fgma added 2 commits November 18, 2023 20:45

minio: update to 2022-10-24T18-35-07Z

932ff38

minio: update changelog

6f8b0b5

fgma added 2 commits November 18, 2023 20:45

minio: end of life note in install wizard

db39b38

minio: fix typos in wizard

5453909

fgma force-pushed the minio-2023-01-02 branch from 6ecaa70 to 5453909 Compare November 18, 2023 19:45

minio: fix version information in binaries

6727bb7

hgy59 mentioned this pull request Sep 6, 2024

Update go to 1.23.2 #6221

Open

10 tasks

hgy59 added 2 commits September 7, 2024 15:18

Merge branch 'master' of https://github.com/synocommunity/spksrc into…

cc49054

… minio-2023-01-02

migrate to redesigned shared folder handing

b52e343

- adjust wizard for shared folders - use SHARE_PATH variable instead of WIZARD_DATA_VOLUME and WIZARD_DATA_DIRECTORY

hgy59 mentioned this pull request Sep 7, 2024

how to get log output during development of wizard pages? #6224

Closed

fix upgrade wizard for DSM 6

9a0ab9c

hgy59 requested a review from th0ma7 September 10, 2024 19:07

th0ma7 approved these changes Sep 14, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update minio to 2022-10-24T18-35-07Z #5555

update minio to 2022-10-24T18-35-07Z #5555

fgma commented Jan 8, 2023 •

edited

Loading

hgy59 commented Jan 21, 2023

fgma commented Jan 21, 2023 •

edited

Loading

hgy59 commented Jan 21, 2023

fgma commented Jan 21, 2023

fgma commented Feb 22, 2023

hgy59 commented Feb 22, 2023

fgma commented Feb 24, 2023

fgma commented Sep 2, 2023

th0ma7 commented Sep 4, 2023

fgma commented Sep 4, 2023

hgy59 commented Sep 4, 2023

fgma commented Sep 4, 2023

fgma commented Nov 2, 2023

th0ma7 commented Nov 4, 2023

fgma commented Nov 4, 2023

th0ma7 left a comment

th0ma7 commented Nov 5, 2023

fgma commented Nov 10, 2023

hgy59 commented Nov 11, 2023 •

edited

Loading

hgy59 commented Nov 11, 2023

hgy59 commented Nov 11, 2023

fgma commented Nov 14, 2023

fgma commented Nov 18, 2023 •

edited

Loading

fgma commented Dec 4, 2023

d8x commented Mar 26, 2024

hgy59 commented Mar 26, 2024

hgy59 commented Mar 26, 2024

hgy59 commented Sep 8, 2024

th0ma7 left a comment

IngmarStein commented Oct 17, 2024

update minio to 2022-10-24T18-35-07Z #5555

Are you sure you want to change the base?

update minio to 2022-10-24T18-35-07Z #5555

Conversation

fgma commented Jan 8, 2023 • edited Loading

Description

Checklist

Type of change

hgy59 commented Jan 21, 2023

fgma commented Jan 21, 2023 • edited Loading

hgy59 commented Jan 21, 2023

fgma commented Jan 21, 2023

fgma commented Feb 22, 2023

hgy59 commented Feb 22, 2023

fgma commented Feb 24, 2023

fgma commented Sep 2, 2023

th0ma7 commented Sep 4, 2023

fgma commented Sep 4, 2023

hgy59 commented Sep 4, 2023

fgma commented Sep 4, 2023

fgma commented Nov 2, 2023

th0ma7 commented Nov 4, 2023

fgma commented Nov 4, 2023

th0ma7 left a comment

Choose a reason for hiding this comment

th0ma7 commented Nov 5, 2023

fgma commented Nov 10, 2023

hgy59 commented Nov 11, 2023 • edited Loading

hgy59 commented Nov 11, 2023

hgy59 commented Nov 11, 2023

fgma commented Nov 14, 2023

fgma commented Nov 18, 2023 • edited Loading

fgma commented Dec 4, 2023

d8x commented Mar 26, 2024

hgy59 commented Mar 26, 2024

hgy59 commented Mar 26, 2024

hgy59 commented Sep 8, 2024

th0ma7 left a comment

Choose a reason for hiding this comment

IngmarStein commented Oct 17, 2024

fgma commented Jan 8, 2023 •

edited

Loading

fgma commented Jan 21, 2023 •

edited

Loading

hgy59 commented Nov 11, 2023 •

edited

Loading

fgma commented Nov 18, 2023 •

edited

Loading