Installing the OSDF Cache¶
+This document describes how to install a Pelican-based Open Science Data Federation (OSDF) Cache service via RPMs. +This service allows a site or regional network to cache data frequently used in Open Science Pool jobs, +reducing data transfer over the wide-area network and decreasing access latency.
+Before Starting¶
+Before starting the installation process, consider the following requirements:
+-
+
- Operating system: A RHEL 8 or RHEL 9 or compatible operating system. +
- User IDs: If it does not exist already, the installation will create the Linux user named
xrootdfor running daemons.
+ - File Systems: The cache should have a partition of its own for storing data and metadata. +
- Host certificate: Required for authentication. See note below. +
- Network ports: The cache service requires the following ports open: +
- Inbound TCP port 8443 for file access via the HTTP(S) and XRoot protocols. +
- (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; + if enabled, access to this port should be restricted to the LAN. +
- Service requirements:
-
+
- A cache serving the OSDF federation as a regional cache should have at least:
-
+
- 8 cores +
- 40 Gbps connectivity +
- 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, + if you cannot fit that much disk into a single chassis +
- 24 GB of RAM +
+ - A cache being used to serve data from the OSDF to a single site should have at least:
-
+
- 8 cores +
- 40 Gbps connectivity +
- 2 TB of NVMe disk for the cache partition +
- 24 GB of RAM + We suggest that several gigabytes of local disk space be available for log files, + although some logging verbosity can be reduced. +
+
+ - A cache serving the OSDF federation as a regional cache should have at least:
As with all OSG software installations, there are some one-time steps to prepare in advance:
+-
+
- Obtain root access to the host +
- Prepare the required Yum repositories +
OSG 23
+In OSG 23, the Pelican-based OSDF RPMs are only available in the "osg-upcoming" repositories.
+Host certificates
+Caches should use a CA that is accepted by major browsers and operating systems, +such as InCommon RSA or Let's Encrypt. +IGTF certs are not recommended because clients are not configured to accept them by default. +Note that you will need the full certificate chain, not just the certificate.
+The following locations should be used (note that they are in separate directories):
+-
+
- Host Certificate Chain:
/etc/pki/tls/certs/pelican.crt
+ - Host Key:
/etc/pki/tls/private/pelican.key
+
Installing the Cache¶
+The cache service is provided by the osdf-cache RPM.
+Install it using one of the following commands:
OSG 24: +
root@host # yum install osdf-cache --enablerepo=osg-testing
+OSG 23: +
root@host # yum install --enablerepo=osg-upcoming-testing osdf-cache
+Configuring the Cache Server¶
+osdf-cache 7.11.1
+This configuration requires version 7.11.1 or newer of the osdf-cache
+and pelican RPMs.
In /etc/pelican/config.d/20-cache.yaml, set Cache.LocalRoot, Cache.DataLocation and Cache.MetaLocation as follows,
+replacing <CACHE PARTITION> with the mount point of the partition you will use for the cache.
+
Cache:
+ LocalRoot: "<CACHE PARTITION>/namespaces"
+ DataLocation: "<CACHE PARTITION>/data"
+ MetaLocation: "<CACHE PARTITION>/meta"
+Preparing for Initial Startup¶
+-
+
-
+
The cache identifies itself to the federation via public key authentication; +before starting the cache for the first time, it is generate a keypair.
++ +root@host$ cd /etc/pelican +root@host$ pelican generate keygen +The newly created files,
+issuer.jwkandissuer-pub.jwksare the private and public keys, respectively.
+ -
+
Save these files; if you lose them, your cache will need to be re-approved.
+
+
Validating the Cache Installation¶
+Do the following steps to verify that the cache is functional:
+-
+
-
+
Start the cache using the following command:
++ +root@host$ systemctl start osdf-cache +
+ -
+
Download a test file from the OSDF through your cache (replacing
+CACHE_HOSTNAMEwith the host name of your cache)+ +root@host$ osdf object get -c CACHE_HOSTNAME:8443 /ospool/uc-shared/public/OSG-Staff/validation/test.txt /tmp/test.txt +root@host$ cat /tmp/test.txt + +Hello, World! +If the download fails, rerun the above
+osdf object getcommand with the-dflag added; +additional debugging information is located in/var/log/pelican/osdf-cache.log. +See this page for requesting assistance; please include the log file +and theosdf object get -doutput in your request.
+
Joining the Cache to the Federation¶
+The cache must be registered with the OSG prior to joining the data federation. +Send mail to help@osg-htc.org requesting registration; provide the following information:
+-
+
- Cache hostname +
- Administrative and security contact(s) +
- Institution that the cache belongs to +
OSG Staff will register the cache and respond with the Resource Name that the cache was registered as.
+Once you have that information, edit /etc/pelican/config.d/15-osdf.yaml, and set XRootD.Sitename:
+
XRootD:
+ Sitename: <RESOURCE NAME REGISTERED WITH OSG>
+Then, restart the cache by running
+root@host$ systemctl restart osdf-cache
+Let OSG Staff know that you have restarted the cache with the updated sitename, +so they can approve the new cache.
+Managing the Cache Service¶
+Use the following SystemD commands as root to start, stop, enable, and disable the OSDF Cache.
+| To... | +Run the command... | +
|---|---|
| Start the cache | +systemctl start osdf-cache |
+
| Stop the cache | +systemctl stop osdf-cache |
+
| Enable the cache to start on boot | +systemctl enable osdf-cache |
+
| Disable the cache from starting on boot | +systemctl disable osdf-cache |
+
Getting Help¶
+To get assistance, please use the this page.
+ + +