Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Software-6012: OSG-24 #201

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/common/ca.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,9 +219,7 @@ To modify the times that `fetch-crl-cron` runs, edit `/etc/cron.d/fetch-crl`.

| Software | Service name | Notes |
|:----------|:-----------------------------|:-------------------------------------------|
| Fetch CRL | `fetch-crl.timer` (EL8-only) | Runs `fetch-crl` every 6 hours and on boot |
| | `fetch-crl-cron` (EL7-only) | Runs `fetch-crl` every 6 hours |
| | `fetch-crl-boot` (EL7-only) | Runs `fetch-crl` immediately and on boot |
| Fetch CRL | `fetch-crl.timer` (EL8+) | Runs `fetch-crl` every 6 hours and on boot |

Start the services in the order listed and stop them in reverse order. As a reminder, here are common service commands (all run as `root`):

Expand Down
2 changes: 1 addition & 1 deletion docs/common/help.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ support inquiry:

* Troubleshooting sections or pages for the problematic software
* Recent OSG Software release notes
- [OSG 24](../release/osg-24.md)
- [OSG 23](../release/osg-23.md)
- [OSG 3.6](../release/osg-36.md)
* [Outage](https://status.opensciencegrid.org/) information for OSG services

### Submitting support inquiries ###
Expand Down
71 changes: 14 additions & 57 deletions docs/common/yum.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@ OSG's RPM packages also rely on external packages provided by supported OSes and
You must have the following repositories available and enabled:

- OS repositories, including the following ones that aren't enabled by default:
- `extras` (SL 7, CentOS 7, CentOS Stream 8, Rocky Linux 8, AlmaLinux 8)
- `Server-Extras` (RHEL 7)
- `powertools` (CentOS Stream 8, Rocky Linux 8, AlmaLinux 8)
- `CodeReady Builder` (RHEL 8) or `crb` (all EL9 variants)
- EPEL repositories
Expand All @@ -52,8 +50,8 @@ Their names start with `osg-upcoming` and have the same structure as our standar
as well as the same guarantees of quality and production-readiness.

There are separate sets of upcoming repositories for each release series.
For example, the [OSG 23 repos](https://repo.opensciencegrid.org/osg/23-main/) have corresponding
[23-upcoming repos](https://repo.opensciencegrid.org/osg/23-upcoming/).
For example, the [OSG 24 repos](https://repo.osg-htc.org/osg/24-main/) have corresponding
[24-upcoming repos](https://repo.osg-htc.org/osg/24-upcoming/).
The upcoming repositories are meant to be layered on top of our standard repositories:
installing software from the upcoming repositories requires also enabling the standard repositories from the same release.

Expand All @@ -67,33 +65,15 @@ supported by the OSG.

The definitive list of software in the contrib repository can be found here:

- [OSG 23 EL8 contrib software repository](https://repo.opensciencegrid.org/osg/23-contrib/el8/x86_64/)
- [OSG 23 EL9 contrib software repository](https://repo.opensciencegrid.org/osg/23-contrib/el9/x86_64/)
- [OSG 24 EL8 contrib software repository](https://repo.opensciencegrid.org/osg/24-contrib/el8/x86_64/)
- [OSG 24 EL9 contrib software repository](https://repo.opensciencegrid.org/osg/24-contrib/el9/x86_64/)

If you would like to distribute your software in the OSG `contrib` repository, please [contact us](../common/help.md) with a
description of your software, what users it serves, and relevant RPM packaging.

Installing Yum Repositories
---------------------------

### Install the Yum priorities plugin (EL7)

The Yum priorities plugin is used to tell Yum to prefer OSG packages over EPEL or OS packages.
It is important to install and enable the Yum priorities plugin before installing OSG Software to ensure that you are
getting the OSG-supported versions.

This plugin is built into Yum on EL8 and EL9 distributions.

1. Install the Yum priorities package:

:::console
root@host # yum install yum-plugin-priorities

1. Ensure that `/etc/yum.conf` has the following line in the `[main]` section:

:::file
plugins=1

### Enable additional OS repositories

Some packages depend on packages that are in OS repositories not enabled by default.
Expand All @@ -104,15 +84,6 @@ The repositories to enable, as well as the instructions to enable them, are OS-d
or if the `enabled` line is missing
(i.e. it is enabled unless specified otherwise.)

#### SL 7

- Install the `yum-conf-extras` RPM package.
- Ensure that the `sl-extras` repo in `/etc/yum.repos.d/sl-extras.repo` is enabled.

#### CentOS 7

- Ensure that the `extras` repo in `/etc/yum.repos.d/CentOS-Base.repo` is enabled.

#### CentOS Stream 8

- Ensure that the `extras` repo in `/etc/yum.repos.d/CentOS-Stream-Extras.repo` is enabled.
Expand All @@ -128,10 +99,6 @@ The repositories to enable, as well as the instructions to enable them, are OS-d
- Ensure that the `extras` repo in `/etc/yum.repos.d/almalinux.repo` is enabled.
- Ensure that the `powertools` repo in `/etc/yum.repos.d/almalinux-powertools.repo` is enabled.

#### RHEL 7

- Ensure that the `Server-Extras` channel is enabled.

#### RHEL 8

- Ensure that the `CodeReady Linux Builder` channel is enabled.
Expand All @@ -157,8 +124,6 @@ You must install and enable these first.
- Install the EPEL repository, if not already present. Choose the right version to match your OS version.

:::console
## EPEL 7 (For RHEL 7, CentOS 7, and SL 7)
root@host # yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
## EPEL 8 (For RHEL 8 and CentOS Stream 8, Rocky Linux 8, AlmaLinux 8)
root@host # yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
## EPEL 9 (For RHEL 9 and CentOS Stream 9, Rocky Linux 9, AlmaLinux 9)
Expand All @@ -185,15 +150,15 @@ For instructions on upgrading from one OSG series to another, see the
1. Install the OSG repository for your OS version and the [OSG release series](../release/release_series.md) that you wish to
use:

- OSG 23 EL8:
- OSG 24 EL8:

:::console
root@host # yum install https://repo.opensciencegrid.org/osg/23-main/osg-23-main-el8-release-latest.rpm
root@host # yum install https://repo.opensciencegrid.org/osg/24-main/osg-24-main-el8-release-latest.rpm

- OSG 23 EL9:
- OSG 24 EL9:

:::console
root@host # yum install https://repo.opensciencegrid.org/osg/23-main/osg-23-main-el9-release-latest.rpm
root@host # yum install https://repo.opensciencegrid.org/osg/24-main/osg-24-main-el9-release-latest.rpm


1. The only OSG repository enabled by default is the release one.
Expand All @@ -202,15 +167,14 @@ For instructions on upgrading from one OSG series to another, see the

:::file hl_lines="7"
[osg-testing]
name=OSG Software for Enterprise Linux 7 - Testing - $basearch
#baseurl=https://repo.opensciencegrid.org/osg/23-main/el8/testing/$basearch
mirrorlist=https://repo.opensciencegrid.org/osg/23-main/el8/testing/$basearch
name=OSG Software for Enterprise Linux 9 - Testing - $basearch
#baseurl=https://repo.opensciencegrid.org/osg/24-main/el9/testing/$basearch
mirrorlist=https://repo.opensciencegrid.org/osg/24-main/el9/testing/$basearch
failovermethod=priority
priority=98
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-OSG
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-OSG-2
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-OSG-24-developer

Optional Configuration
----------------------
Expand All @@ -226,20 +190,13 @@ Therefore we recommend security-only automatic updates or disabling automatic up

To enable only security related automatic updates:

- On EL 7 variants, edit `/etc/yum/yum-cron.conf` and set `update_cmd = security`

- On EL8 and EL9 variants, edit `/etc/dnf/automatic.conf` and set `upgrade_type = security`

CentOS 7, CentOS Stream 8, and CentOS Stream 9 do not support security-only automatic updates;
doing any of the above steps will prevent automatic updates from happening at all.

To disable automatic updates entirely:

- On EL7 variants, run:

:::console
root@host # service yum-cron stop

- On EL8 and EL9 variants, run:

:::console
Expand Down Expand Up @@ -270,10 +227,10 @@ Add the following to a file in `/etc/cron.d`:
Or, to mirror only a single repository:

:::file
<RANDOM> * * * * root rsync -aH rsync://repo-rsync.opensciencegrid.org/osg/<OSG_RELEASE>/el9/development /var/www/html/osg/<OSG_RELEASE>/el7
<RANDOM> * * * * root rsync -aH rsync://repo-rsync.opensciencegrid.org/osg/<OSG_RELEASE>/el9/development /var/www/html/osg/<OSG_RELEASE>/el9


Replace `<OSG_RELEASE>` with the OSG release you would like to use (e.g. `23-main`) and `<RANDOM>` with a number between 0
Replace `<OSG_RELEASE>` with the OSG release you would like to use (e.g. `24-main`) and `<RANDOM>` with a number between 0
and 59.

On your worker node, you can replace the `baseurl` line of `/etc/yum.repos.d/osg.repo` with the appropriate URL for your
Expand Down
20 changes: 10 additions & 10 deletions docs/compute-element/install-htcondor-ce.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Before Starting
---------------

Before starting the installation process, consider the following points, consulting the upstream references as needed
([HTCondor-CE 23](https://htcondor.com/htcondor-ce/v23/reference/)):
([HTCondor-CE 24](https://htcondor.com/htcondor-ce/v24/reference/)):

- **User IDs:** If they do not exist already, the installation will create the Linux users `condor` (UID 4716) and
`gratia`
Expand All @@ -43,7 +43,7 @@ Before starting the installation process, consider the following points, consult
- **Access point/login node:** HTCondor-CE should be installed on a host that already has the ability to submit jobs
into your local cluster
- **File Systems**: Non-HTCondor batch systems require a
[shared file system](https://htcondor.com/htcondor-ce/v23/configuration/local-batch-system/#sharing-the-spool-directory)
[shared file system](https://htcondor.com/htcondor-ce/v24/configuration/local-batch-system/#sharing-the-spool-directory)
between the HTCondor-CE host and the batch system worker nodes.

As with all OSG software installations, there are some one-time (per host) steps to prepare in advance:
Expand Down Expand Up @@ -113,7 +113,7 @@ For more advanced configuration, see the section on [optional configurations](#o
### Configuring the local batch system ###

To configure HTCondor-CE to integrate with your local batch system,
please refer to the [upstream documentation](https://htcondor.com/htcondor-ce/v23/configuration/local-batch-system/).
please refer to the [upstream documentation](https://htcondor.com/htcondor-ce/v24/configuration/local-batch-system/).

### Configuring authentication ###

Expand All @@ -137,7 +137,7 @@ To accept RARs from a particular collaboration:
SCITOKENS /^https\:\/\/scitokens\.org\/osg\-connect,/ osgpilot

For more details of the mapfile format, consult the "SciTokens" section of the
[upstream documentation](https://htcondor.com/htcondor-ce/v23/configuration/authentication/#scitokens).
[upstream documentation](https://htcondor.com/htcondor-ce/v24/configuration/authentication/#scitokens).

#### Bannning a collaboration

Expand Down Expand Up @@ -188,8 +188,8 @@ In addition to the configurations above, you may need to further configure how p
before they are submitted to your local batch system or otherwise change the behavior of your CE.
For detailed instructions, please refer to the upstream documentation:

- [Configuring the Job Router](https://htcondor.com/htcondor-ce/v23/configuration/job-router-overview/)
- [Optional configuration](https://htcondor.com/htcondor-ce/v23/configuration/optional-configuration/)
- [Configuring the Job Router](https://htcondor.com/htcondor-ce/v24/configuration/job-router-overview/)
- [Optional configuration](https://htcondor.com/htcondor-ce/v24/configuration/optional-configuration/)

#### Accounting with multiple CEs or local user jobs

Expand All @@ -208,16 +208,16 @@ Starting and Validating HTCondor-CE
-----------------------------------

For information on how to start and validate the core HTCondor-CE services, please refer to the
[upstream documentation](https://htcondor.com/htcondor-ce/v23/operation/)
[upstream documentation](https://htcondor.com/htcondor-ce/v24/operation/)

Troubleshooting HTCondor-CE
---------------------------

For information on how to troubleshoot your HTCondor-CE, please refer to the upstream documentation:

- [Common issues](https://htcondor.com/htcondor-ce/v23/troubleshooting/common-issues/)
- [Debugging tools](https://htcondor.com/htcondor-ce/v23/troubleshooting/debugging-tools/)
- [Helpful logs](https://htcondor.com/htcondor-ce/v23/troubleshooting/logs/)
- [Common issues](https://htcondor.com/htcondor-ce/v24/troubleshooting/common-issues/)
- [Debugging tools](https://htcondor.com/htcondor-ce/v24/troubleshooting/debugging-tools/)
- [Helpful logs](https://htcondor.com/htcondor-ce/v24/troubleshooting/logs/)

Registering the CE
------------------
Expand Down
9 changes: 1 addition & 8 deletions docs/data/external-oasis-repos.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Before Starting

The host OS must be:

- RHEL7 or RHEL8 (or equivalent).
- RHEL8 or RHEL9 (or equivalent).

Additionally,

Expand All @@ -36,13 +36,6 @@ Additionally,
repository itself will be done as an unprivileged user.
- **Yum** will need to be [configured to use the OSG repositories](../common/yum.md).

!!! warning "Overlay-FS limitations"
CVMFS on RHEL7 only supports Overlay-FS if the underlying filesystem is `ext3` or `ext4`; make sure
`/var/spool/cvmfs` is one of these filesystem types.

If this is not possible, add `CVMFS_DONT_CHECK_OVERLAYFS_VERSION=yes` to your CVMFS configuration. Using
`xfs` will work if it was created with `ftype=1`

Installation
------------

Expand Down
6 changes: 3 additions & 3 deletions docs/data/run-frontier-squid-container.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ To run a Frontier Squid container with the defaults:
user@host $ docker run --rm --name frontier-squid \
-v <HOST CACHE PARTITION>:/var/cache/squid \
-v <HOST LOG PARTITION>:/var/log/squid \
-p <HOST PORT>:3128 opensciencegrid/frontier-squid:23-release
-p <HOST PORT>:3128 hub.opensciencegrid.org/osg-htc/frontier-squid:24-release
```

You may pass configuration variables in `KEY=VALUE` format with either
Expand Down Expand Up @@ -112,8 +112,8 @@ TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker stop %n
ExecStartPre=-/usr/bin/docker rm %n
ExecStartPre=/usr/bin/docker pull opensciencegrid/frontier-squid:23-release
ExecStart=/usr/bin/docker run --rm --name %n --publish 3128:3128 -v /tmp/squid:/var/cache/squid -v /tmp/log:/var/log/squid --env-file /opt/xcache/.env opensciencegrid/frontier-squid:23-release
ExecStartPre=/usr/bin/docker pull hub.opensciencegrid.org/osg-htc/frontier-squid:24-release
ExecStart=/usr/bin/docker run --rm --name %n --publish 3128:3128 -v /tmp/squid:/var/cache/squid -v /tmp/log:/var/log/squid --env-file /opt/xcache/.env hub.opensciencegrid.org/osg-htc/frontier-squid:24-release


[Install]
Expand Down
33 changes: 1 addition & 32 deletions docs/data/stashcache/install-cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,19 +247,6 @@ To use HTTPS:

1. Uncomment `set EnableVoms = 1` in `/etc/xrootd/config.d/10-osg-xrdvoms.cfg`

!!! note "Upgrading from OSG 3.5"
If upgrading from OSG 3.5, you may have a file with the following contents in `/etc/xrootd/config.d`:

# Support HTTPS access to unauthenticated cache
if named stash-cache
http.cadir /etc/grid-security/certificates
http.cert /etc/grid-security/xrd/xrdcert.pem
http.key /etc/grid-security/xrd/xrdkey.pem
http.secxtractor /usr/lib64/libXrdLcmaps.so
fi

You must delete this config block or XRootD will fail to start.


Manually Setting the FQDN (optional)
------------------------------------
Expand Down Expand Up @@ -313,7 +300,7 @@ As a reminder, here are common service commands (all run as `root`):
|--------------|------------------|-----------|
| XRootD | `xrootd@stash-cache.service` | The XRootD daemon, which performs the data transfers |
| XCache | `xcache-reporter.timer` | Reports usage information to collector.opensciencegrid.org |
| Fetch CRL |EL8: `fetch-crl.timer` <br> EL7: `fetch-crl-boot` and `fetch-crl-cron` | Required to authenticate monitoring services. See [CA documentation](../../common/ca.md#managing-fetch-crl-services) for more info |
| Fetch CRL | `fetch-crl.timer` | Required to authenticate monitoring services. See [CA documentation](../../common/ca.md#managing-fetch-crl-services) for more info |
| | `stash-authfile@stash-cache.service` | Generate authentication configuration files for XRootD (public cache instance) |
| | `stash-authfile@stash-cache.timer` | Periodically run the above service (public cache instance) |

Expand Down Expand Up @@ -375,24 +362,6 @@ STASHCACHE_DaemonVersion = "1.0.0"
```


Updating to OSG 3.6
-------------------

The OSG 3.5 series reached end-of-life on May 1, 2022.
Admins are strongly encouraged to move their caches to OSG 3.6.

See [general update instructions](../../release/updating-to-osg-36.md).

Unauthenticated caches (`xrootd@stash-cache` service) do not need any configuration changes,
unless HTTPS access has been enabled.
See the ["enable HTTPS on the unauthenticated cache" section](#enable-https-on-the-unauthenticated-cache))
for the necessary configuration changes.

Authenticated caches (`xrootd@stash-cache-auth` service) may need the configuration changes described in the
[updating to OSG 3.6 section](../xrootd/xrootd-authorization.md#updating-to-osg-36)
of the XRootD authorization configuration document.


Getting Help
------------

Expand Down
15 changes: 1 addition & 14 deletions docs/data/stashcache/install-origin.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Before Starting

Before starting the installation process, consider the following requirements:

* __Operating system:__ A RHEL 7 or RHEL 8 or compatible operating systems.
* __Operating system:__ A RHEL 8 or RHEL 9 or compatible operating systems.
* __User IDs:__ If they do not exist already, the installation will create the Linux user IDs `condor` and `xrootd`;
only the `xrootd` user is utilized for the running daemons.
* __Host certificate:__ Required for authentication.
Expand Down Expand Up @@ -328,19 +328,6 @@ See the page on [getting your VO's data into OSDF](vo-data.md).

Specifying the DN of your origin is not required but it is useful for testing.

Updating to OSG 3.6
-------------------

The OSG 3.5 series reached end-of-life on May 1, 2022.
Admins are strongly encouraged to move their origins to OSG 3.6.

See [general update instructions](../../release/updating-to-osg-36.md).

Unauthenticated origins (`xrootd@stash-origin` service) do not need any configuration changes.

Authenticated origins (`xrootd@stash-origin-auth` service) may need the configuration changes described in the
[updating to OSG 3.6 section](../xrootd/xrootd-authorization.md#updating-to-osg-36)
of the XRootD authorization configuration document.

Getting Help
------------
Expand Down
Loading
Loading