Migration using NSS2CSS 12/2024

[bourgeoa]

• Some data :

  • 20241211
    accounts 62932 email 51834
    58326 from 2020-12-31
  • size : data + .db/users/users : 66.5 GB

• test migration from solidcommunity.net backup on 2024/12/15

  • step1 on initialization reading NSS accounts 10 mn
  • step 2 CSS accounts creation 1s per account => expected 16 hours
    • to investigate : Do not fully succeed. only copy 24805 pods (in 14 hours).
    • podname's account converted to lower case (NSS url use lower case also, but not the account)
  • step 3 : copy users folders/files : 1 hour (need verification)
  • step 4 : update password to NSS one (seems OK was able to login to bourgeoa and bourgeoa1)
  • step 5 : replace oidcIssuier in WebID seems OK (ending with '/')
  • step 6 : replace acl:defaultForNew with acl:default seems OK
  • step 7 : replace links : this broke at first account
  • issues
    • step 2
      • pod with name's including blank or @ sign cannot be created on CSS (not allowed in an email)
      • on account creation uppercase converted to lower case (email request ?)
      • only 24805 pods created out of 62932 pods
      • account created even if there is no data folder this induces ENOENT error in the steps 4 to 8
    • step 7 : replace links on files
      • pod name beginning with '+' (+devient) broke step 7 on code error
      • improve to be able re-run on all situations
      • actually only on .ttl .acl and .meta extensions
      • add .json and .jsonld extensions

	7️⃣  CSS: Update 24805 pods : replace links on files content
	file:///home/solid/NSS2CSS/copy-pods-to-css.mjs:292
			const patt = new RegExp(source)
						 ^

	SyntaxError: Invalid regular expression: /+deviant.solidcommunity.net/: Nothing to repeat
		at new RegExp (<anonymous>)
		at file:///home/solid/NSS2CSS/copy-pods-to-css.mjs:292:22

	Node.js v20.18.0

• can we reduce the migration duration
  • reduce the list of migrated accounts
    • call users for a list of pods to migrate
    • check pod used for example on last 2 years
      may this be done by checking last modified file in user folder
  • We can we check NSS accounts on date.
    https://stackoverflow.com/questions/26111471/get-last-modified-file-date-in-a-folder-structure
    https://stackoverflow.com/questions/4997242/in-linux-terminal-how-do-i-show-the-folders-last-modification-date-taking-its
  • replace cp copy pods by move mv
    to do that we can make a rsync backup that uses only about 1 hour
• after initial migration : migrating pods on request can be done on demand with no difficulty

[bourgeoa]

Temporarily Proposals : (do not create when there is an issue)

1. do not create account if the data folder do not exist
2. blank and @ in podName either do not copy or %encode --> temporarily do not create
3. do not create account with uppercase