Noobish question: where does the data /actually/ live in the OVF deployment?

[voyager529]

So, I spun up Cortx on my ESXi host...but having read basically-everything I could find, I'm still missing an a-HA! moment somewhere...

For a bit of context, I have an ESXi host that lives on my lab LAN, along with a FreeNAS build. I've got Minio running on the FreeNAS via its S3 communications option. The ESXi doesn't have a meaningful amount of storage available to it, so the ultimate goal is to compare Cortx to Minio and see how it fares. I'm not a super huge enterprise user or anything, but I am interested in seeing Cortx work...I'm just missing a thing or two and I might not be the only one...

1. The WebUI doesn't seem to allow the ability to define or monitor the storage on the backend. While I was able to figure out how to define buckets, assigning them to a storage volume didn't seem to be readily apparent in the WebUI.

2. I'm assuming that it's intended to have the capacity to point data to something other than the included hard disks, as 20/50/50GB disk volumes do not a massive storage server make. It'd make a bit more sense that I would be expected to put the VMDK volumes on an NFS or iSCSI share if one of them was a thin-provisioned 16EB volume or something, but that does not seem to be the case.

3. The WebUI frontend (nor any of the documentation I have read thus far) does not seem to indicate the ability to create an iSCSI/NFS connection.

Really, what I'm missing here is that if Cortx is supposed to send data to hard disks (ideally Seagate ones, obviously), then I would assume that there would be some amount of that function that's handled by Cortx instead of letting VMWare do it. I readily admit to not-being super well versed in this sort of storage functionality, but it may well be worth addressing in the FAQ if this concern is 100% a PEBKAC issue.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label question to this issue, with a confidence of 0.90. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[gregnsk]

@voyager529 , thank you for your interest in CORTX.

The VM is provided just to demonstrate basic CORTX functionality but is not intended for production use. It doesn't provide high availability, performance or capacity and there is no guarantee for long term data preservation.

CORTX is an object storage layer which doesn't care much about the underlying storage. We focus on making it shine with particular hardware for mass capacity use cases (we call it Lyve Rack), but CORTX may work with any general purpose local drives. If a customer decides to use an FC or an iSCSI target instead of local drives - it should work too. However, as these options are practically unlimited we're not planning to provide special integrations for them. If you're interested in a particular configuration, you're welcome to test it and provide an installation documentation which we'll gladly publish in CORTX git repository. See an example in https://github.com/Seagate/cortx/tree/main/doc/scaleout

[justinzw]

@voyager529 Thanks for asking this. I moved it to a discussion in case others run have a similar question.

I'd also like to help and assist you with your goals for CORTX.
Are you on our slack channel yet? We are on there and will be happy to help you out!

Here's a link to join: https://bit.ly/3609FAb

[voyager529]

hey @justinzw and @gregnsk,

Thanks for getting back to me and for the Slack invite. I took a look through the channel earlier today, and it seemed that at least one other person was in a similar boat.

I understand that Cortx isn't looking to be a file system or deal with making RAID volumes or anything like that. I'm not expecting to find a FreeNAS clone being implemented.

However, at some level, that's going to come into play. When I use Minio on a Synology, I can define the mount point to which the data goes as a part of the Docker config. When I use Minio on FreeNAS, once again, there's a place where Minio is ultimately told what folder to send the data transferred via S3.

If such a function isn't present, then that needs to be taken into account, and the data can, presumably, only be sent to a secondary volume by making changes to /etc/fstab...which is fine, but the absence of that sort of requirement in the 'getting started' documentation, I submit, is a glaring omission. Conversely, it might be worth adding a "what volume do you want the data to live on" as a part of the initial setup wizard, allowing the user to specify local storage, an NFS share, or have all the boxes to stipulate an iSCSI volume.

Respectfully,

We focus on making it shine with particular hardware for mass capacity use cases (we call it Lyve Rack), but CORTX may work with any general purpose local drives.

That comes across as a somewhat contradictory statement, especially given that it's being provided as a VM rather than a Docker container. If the point of Cortx is to be the building block or a substantial module of the Lyve Rack firmware, great...but that still leaves two possibilities: the first is that the LR firmware is intended to be proprietary like Syn and QNAP, which makes Cortx itself limited in its utility; in such a case it may well be worth slapping in a Docker module for users to add Minio or just git cloning Minio directly. Moreover, testing Cortx in isolation would probably be able to give reliable feedback as to whether the S3 API is implemented correctly, but little else. It's possible that Cortx throws up with an 8TB file (entirely possible for a VM), but we can't test that because of the inability to put more than 50GB on the "spherical cow in a vacuum" VM.

Alternatively, the LR firmware may be intended to be released as part of an 'official' bundle with Lyverack hardware and an 'unofficial bundle' that'll more directly compete with FreeNAS, at which point testing Cortx in isolation seems equally-limited since it's still just one module.

I'm certainly fine with any of these things, but as much as I'm not expecting Cortx to be a ZFS/BTRFS/EXT4 replacement, the inability to so much as point buckets to a folder and let the OS handle the file system and even the iSCSI/NFS negotiation makes it super difficult to give any feedback other than "the UI is pretty", sadly.

I hope that helps explain things a bit more, and I appreciate the continued discussion.

[gregnsk]

Hi @voyager529, let me make a few clarifying statements here.

1. Some S3 implementations (like Minio or older versions of Ceph) lay data on top of a regular file system. This adds an extra abstraction layer which may impact performance. We don't rely on any file system in CORTX. Our engine, Motr, works with directly with block devices to store data.
2. You can certainly configure CORTX to utilize particular devices on your server - however, for that purpose you may need to install CORTX from the source code.
3. The VM is provided as a quick functional reference, it allows ISVs and partners to test their applications with CORTX APIs without spending too much time installing open source or getting a Lyve Rack system in their data center. It's not intended for anything else.
4. Containerized version of CORTX - it's something that we're planning to offer at some point. Currently CORTX has some kernel-level dependencies which can't be containerized. It's going to be addressed soon, and will pave the way to container image for CORTX

Thank you for your interest in CORTX, let us know if you'd like to have a phone call to provide more feedback