kubectl-rook-ceph

Provide common management and troubleshooting tools for the Rook Ceph storage provider as a Krew plugin.

Install

Note: This requires kubectl krew to be installed.

To install the plugin, run:

kubectl krew install rook-ceph

To check plugin version kubectl krew list this will list all krew plugin with their current version.

Update

kubectl krew upgrade rook-ceph

Usage

kubectl rook-ceph <root-args> <command> <command-args>

Root args

These are args currently supported:

1. -h|--help: this will print brief command help text.

   kubectl rook-ceph --help

2. -n|--namespace='rook-ceph': the Kubernetes namespace in which the CephCluster resides. (optional, default: rook-ceph)

   kubectl rook-ceph -o test-operator -n test-cluster rook version

3. -o|--operator-namespace : the Kubernetes namespace in which the rook operator resides, when the arg -n is passed but -o is not then -o will equal to the -n. (default: rook-ceph)

   kubectl rook-ceph -o test-operator -n test-cluster rook version

4. --context: the name of the Kubernetes context to be used (optional).

   kubectl rook-ceph --context=$(kubectl config current-context) mons

Commands

• ceph <args> : Run a Ceph CLI command. Supports any arguments the ceph command supports. See Ceph docs for more.

• rbd <args> : Call a 'rbd' CLI command with arbitrary args

• mons : Print mon endpoints

  • restore-quorum <mon-name> : Restore the mon quorum based on a single healthy mon since quorum was lost with the other mons

• health : check health of the cluster and common configuration issues

• operator

  • restart : Restart the Rook-Ceph operator
  • set <property> <value> : Set the property in the rook-ceph-operator-config configmap.

• rook

  • version : Print the version of Rook
  • status : Print the phase and/or conditions of every CR in the namespace
  • status all : Print the phase and conditions of all CRs
  • status <CR> : Print the phase and conditions of CRs of a specific type, such as cephobjectstore, cephfilesystem, etc
  • purge-osd <osd-id> [--force] : Permanently remove an OSD from the cluster. Multiple OSDs can be removed with a comma-separated list of IDs.

• debug : Debug a deployment by scaling it down and creating a debug copy. This is supported for mons and OSDs only

  • start <deployment-name> [--alternate-image <alternate-image>] : Start debugging a deployment with an optional alternative ceph container image
  • stop <deployment-name> : Stop debugging a deployment

• dr :

  • health [ceph status args]: Print the ceph status of a peer cluster in a mirroring-enabled environment thereby validating connectivity between ceph clusters. Ceph status args can be optionally passed, such as to change the log level: --debug-ms 1.

• restore-deleted <CRD> [CRName]: Restore the ceph resources which are stuck in deleting state due to underlying resources being present in the cluster

• help : Output help text

Documentation

Visit docs below for complete details about each command and their flags uses.

1. Running ceph commands
2. Running rbd commands
3. Get mon endpoints
4. Get cluster health status
5. Update configmap rook-ceph-operator-config
6. Restart operator pod
7. Get rook version
8. Get all CR status
9. Get cephCluster CR status
10. Get specific CR status
11. To purge OSD
12. Debug OSDs and Mons
13. Restore mon quorum
14. Disaster Recovery
15. Restore deleted CRs
16. Destroy cluster

Examples

Run a Ceph Command

Any ceph command can be run with the plugin. This example gets the ceph status:

kubectl rook-ceph ceph status

 cluster:
   id:     a1ac6554-4cc8-4c3b-a8a3-f17f5ec6f529
   health: HEALTH_OK

 services:
   mon: 3 daemons, quorum a,b,c (age 11m)
   mgr: a(active, since 10m)
   mds: 1/1 daemons up, 1 hot standby
   osd: 3 osds: 3 up (since 10m), 3 in (since 8d)

 data:
   volumes: 1/1 healthy
   pools:   6 pools, 137 pgs
   objects: 34 objects, 4.1 KiB
   usage:   58 MiB used, 59 GiB / 59 GiB avail
   pgs:     137 active+clean

 io:
   client:   1.2 KiB/s rd, 2 op/s rd, 0 op/s wr

Restart the Operator

kubectl rook-ceph operator restart

deployment.apps/rook-ceph-operator restarted

Rook Version

kubectl rook-ceph rook version

rook: v1.8.0-alpha.0.267.g096dabfa6
go: go1.16.13

Ceph Versions

kubectl rook-ceph ceph versions

{
    "mon": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 1
    },
    "mgr": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 1
    },
    "osd": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 1
    },
    "mds": {},
    "overall": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 3
    }
}

Contributing

We welcome contributions. See the Rook Contributing Guide to get started.

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, please open an issue.

Contact

Please use the following to reach members of the community:

• Slack: Join our slack channel
• Forums: rook-dev
• Twitter: @rook_io
• Email (general topics): cncf-rook-info@lists.cncf.io
• Email (security topics): cncf-rook-security@lists.cncf.io

Licensing

Rook is under the Apache 2.0 license.