Evaluate Minio

[ctengel]

• Fix the Go install issue
  • https://docs.min.io/docs/minio-quickstart-guide.html
  • https://maelvls.dev/go111module-everywhere/
  • https://stackoverflow.com/questions/24878737/what-is-the-difference-between-go-get-and-go-install
• Look at the on disk format
• Last chance for competition
  • https://hypercore-protocol.org/
  • https://zenko.readthedocs.io/en/latest/operation/Architecture/index.html
  • https://s3-server.readthedocs.io/en/latest/ARCHITECTURE.html
  • https://zenko.readthedocs.io/en/latest/

[ctengel]

Opened issue - minio/mc#4100

[ctengel]

Openio? Openmediavault? Owncloud?

[ctengel]

Client - boto3 (more) or minio-py

[ctengel]

Going to use boto3 - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#service-resource

[ctengel]

Note since then we hit another major minio issue with auto upgrading and losing backward compatibility

In happy news lots of benefits from the new version hopefully

See notes in #7

[ctengel]

We need better documentation of how to setup MINIO buckets etc - see #8

[ctengel]

1. Free up laptop space
2. Reattempt go install
3. Reread docs in Get one node up #8 about setting up buckets
4. Attempt to setup
5. Document in README
6. Close this issue and move on to CLI tool to import local data #11

[ctengel]

• https://www.linkedin.com/pulse/build-your-own-private-cloud-home-raspberry-pi-minio-huerta-arias
• https://min.io/docs/minio/linux/index.html?ref=docs-redirect
• https://min.io/docs/minio/linux/administration/identity-access-management.html?ref=docs-redirect
• https://stackoverflow.com/questions/42616518/minio-hows-bucket-policy-related-to-anonymous-authorized-access

[ctengel]

From https://www.linkedin.com/pulse/build-your-own-private-cloud-home-raspberry-pi-minio-huerta-arias

$ cat /boot/config.txt
# Increase power available to USB ports
max_usb_current=1

Sharable URLs

$ systemctl list-units | grep '/home/pi/data' | awk '{ print $1 }'
home-pi-data.mount
$ cat  /etc/systemd/system/minio.service
[Unit]
Description=Minio Storage Service
After=network-online.target home-pi-data.mount
[Service]
ExecStart=/home/pi/start.sh
WorkingDirectory=/home/pi
StandardOutput=inherit
StandardError=inherit
Restart=always
User=pi

[Install]
WantedBy=multi-user.target
$ sudo systemctl start minio
$ sudo systemctl status minio
$ sudo systemctl enable minio

[ctengel]

From https://min.io/docs/minio/linux/index.html?ref=docs-redirect

• https://min.io/docs/minio/linux/administration/identity-access-management.html#minio-authentication-and-identity-management
  • also included above
• https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#deploy-minio-distributed

https://stackoverflow.com/questions/42616518/minio-hows-bucket-policy-related-to-anonymous-authorized-access

[ctengel]

Consider Fedora, now supported officially on Raspbery Pi 4 as of Fedora 37

[ctengel]

this is the infra piece - let's execute on #5 (comment)

[ctengel]

https://github.com/minio/console

[ctengel]

trying to figure out still -

• how to handle user access to bucket
  • https://blog.nikhilbhardwaj.in/2020/02/25/minio-bucket-policy/
  • Support for User access control through Principal in Bucket policies minio/minio#9530
  • https://min.io/docs/minio/linux/administration/identity-access-management/policy-based-access-control.html
  • https://stackoverflow.com/questions/74603734/how-to-restrict-access-by-particular-user-to-bucket-using-bucket-level-policy-in
• anonymous bucket access https://stackoverflow.com/questions/42616518/minio-hows-bucket-policy-related-to-anonymous-authorized-access
• do we need MINIO_DISK_USAGE_CRAWL???
• do we need https://github.com/minio/console
• final rpi setup
  • usb power
  • systemd

figure out and document above, then close this issue

fedora question can stick with #2

[ctengel]

I believe access is solved, need to add example JSON and commands to README

[ctengel]

After doing and splitting some stuff off, the only thing left to do is to get minio running on rpi, including usb power fix and systemd

[ctengel]

Others:

• https://github.com/juicedata/juicefs
• https://github.com/moosefs/moosefs
• https://filiphanes.sk/en/dont-reinvent-file-object-storage-api/

[ctengel]

testing now for streaming

[ctengel]

mp4 streaming seems to work (actual mp4 and not mp2t disguised as mp4) fantastically - firefox requests appropriate sections and minio sends them - test success!

[ctengel]

More stability/maturity issues -

mc: <ERROR> Deprecated command. Please use 'mc admin policy create' instead.

upon calling MinioAdmin.policy_add() which seems to run mc admin policy add

Essentially the CLI deprecated a usage to the point of breaking it without even letting the official minio python wrapper/library catch up...

[ctengel]

See minio/minio-py#1293

[ctengel]

Seems fixed in latest, with pip3 install https://github.com/minio/minio-py/archive/refs/heads/master.zip

[ctengel]

At this point minio-py seems fixed as of 7.1.16

Remaining

• formal test of 7.1.16
• systemd
  • see Evaluate Minio #5 (comment) - document this
  • perhaps docker?
• power fixes
  • see Evaluate Minio #5 (comment) - document this
  • do we need a hub? monitor dmesg

[ctengel]

dmesg is clean and so is minio log for over a week - that is good

Just need to determine if any power tweaks (software/firmware or hardware (usb hub etc)) are needed...

[ctengel]

It appears max usb current no longer needs to be tweaked https://www.raspberrypi.com/documentation/computers/config_txt.html#max_usb_current

One less thing for us to do however bad news in that it seems any power woes have to be fixed in hardware

[ctengel]

Will monitor and open separate HW issue as needed. Minio is it for now!