When it comes to network accessible storage in compute stacks, there’s no shortage of use cases. Whether you’re deploying multi-homed services that share common files or you just need the increased performance offered by a network file system, Ceph is a great tool for building cost-effective, hyper-converged storage solutions. This tutorial should give you a very easy start in utilizing Ceph to provide network shares for your virtual machines that operate like native storage.
Ceph Configuration
The first step to this process involves configuring Ceph and extracting relevant information needed to setup your virtual machine client. Open a command shell to one of the hosts running Ceph and elevate your access to a user that has permissions to execute Ceph configuration commands.
Once you have an active command shell to the Ceph server, copy and paste the following script block into the terminal and then execute it with a return as required. This will install the APT package “jq” which is used for extracting some of the configuration settings.
(
sudo apt install -y jq
FS_NAME_DEF="cephfs"
FS_PATH_DEF="/"
FS_PERM_DEF="rw"
CLIENT_NAME_DEF="vm-client"
FS_NAME=""
FS_PATH=""
FS_PERM=""
CLIENT_NAME=""
get_fs_name () {
clear
echo 'Enter the name of the Ceph file system you wish to use and then press return ['"$FS_NAME_DEF"']:'
read FS_NAME
FS_NAME=$(echo "$FS_NAME" | xargs)
if [[ ${#FS_NAME} -eq 0 ]]; then
FS_NAME="$FS_NAME_DEF"
fi
}
get_fs_path () {
clear
echo 'Enter the absolute path within the Ceph file system you wish to use and then press return ['"$FS_PATH_DEF"']:'
read FS_PATH
FS_PATH=$(echo "$FS_PATH" | xargs)
if [[ ${#FS_PATH} -eq 0 ]]; then
FS_PATH="$FS_PATH_DEF"
fi
}
get_fs_perm () {
clear
echo 'Enter the access permissions for the previously provided path you wish to use and then press return ['"$FS_PERM_DEF"']:'
read FS_PERM
FS_PERM=$(echo "$FS_PERM" | xargs)
if [[ ${#FS_PERM} -eq 0 ]]; then
FS_PERM="$FS_PERM_DEF"
fi
}
get_client_name () {
clear
echo 'Enter the name of the Ceph client node and then press return ['"$CLIENT_NAME_DEF"']:'
read CLIENT_NAME
CLIENT_NAME=$(echo "$CLIENT_NAME" | xargs)
if [[ ${#CLIENT_NAME} -eq 0 ]]; then
CLIENT_NAME="$CLIENT_NAME_DEF"
fi
}
get_fs_name
get_fs_path
get_fs_perm
get_client_name
ceph fs authorize $FS_NAME client.$CLIENT_NAME $FS_PATH $FS_PERM > /dev/null
CLIENT_SECRET=$(ceph auth get-or-create-key client.$CLIENT_NAME)
CEPH_CONF=$(ceph config generate-minimal-conf)
MON_TARGET=$(ceph mon dump -f json 2>/dev/null|jq --raw-output .mons[0].public_addrs.addrvec[0].addr)
readarray -d : -t MON_IP<<<"$MON_TARGET"
clear
echo 'export FS_NAME='"$FS_NAME"
echo 'export FS_PATH='"$FS_PATH"
echo 'export CLIENT_NAME='"$CLIENT_NAME"
echo 'export CLIENT_SECRET='"$CLIENT_SECRET"
echo 'export MON_IP='"$MON_IP"
echo 'sudo tee /etc/ceph/ceph.conf &> /dev/null <<EOF
'"$CEPH_CONF"'
EOF
'
)
This script will prompt you for the following information;
- Ceph file system name that should be used.
- Ceph file system path that should be used. This should be an absolute path within the given file system.
- Ceph file system permissions that should be granted for the previously given file system path. This can be one of “r” for read-only, “w” for write-only, or “rw” for both read and write permissions.
- Ceph client name that should be used. This is what will be used to create a user for Ceph authentication so it should only contain alphanumeric characters as well as underscores and hyphens.
The rest of the required information will be automatically extracted by the script. If everything executes correctly, you should be left with a Bash script that looks something like this:
export FS_NAME=cephfs
export FS_PATH=/vm/docker-stacks
export CLIENT_NAME=docker-node1
export CLIENT_SECRET=AQC4YUhjP+NbOxAAH+PViHPlzP4joKruS3qkXg==
export MON_IP=172.22.1.4
sudo tee /etc/ceph/ceph.conf &> /dev/null <<EOF
# minimal ceph.conf for 20fc650f-25a4-42c6-957c-efb594ce1f01
[global]
fsid = 20fc650f-25a4-42c6-957c-efb594ce1f01
mon_host = [v2:172.22.1.2:3300/0,v1:172.22.1.2:6789/0] [v2:172.22.1.3:3300/0,v1:172.22.1.3:6789/0] [v2:172.22.1.4:3300/0,v1:172.22.1.4:6789/0]
EOFCopy the Bash script to a text editor for use in the next step.
Virtual Machine Configuration
For the remaining steps, open a command shell to the virtual machine that will act as a Ceph client for your application. Once you have an active command shell to the virtual machine, paste the Bash script that was generated from the previous step into the command shell and then execute it with a return as required.
Now the virtual machine should be ready for configuration. Using the same command shell that the previous Bash script was executed in, paste the following script and then execute it with a return as required.
(
MNT_PATH_DEF="/mnt/cephfs"
MNT_PATH=""
create_mount_path () {
echo "The given path $MNT_PATH does not exist. Would you like to create it automatically? Enter yes or no:"
read CREATE_PATH
CREATE_PATH=$(echo "$CREATE_PATH" | xargs)
if [[ "$CREATE_PATH" == "yes" ]] || [[ "$CREATE_PATH" == "y" ]]; then
sudo mkdir $MNT_PATH
else
clear
echo "The given path $MNT_PATH was not automatically created so you must specify an existing directory to use!"
echo ''
get_mount_path
fi
}
get_mount_path () {
echo 'Enter the absolute path of the location you wish to mount the Ceph file system to and then press return ['"$MNT_PATH_DEF"']:'
read MNT_PATH
MNT_PATH=$(echo "$MNT_PATH" | xargs)
if [[ ${#MNT_PATH} -eq 0 ]]; then
MNT_PATH="$MNT_PATH_DEF"
fi
if [[ ! -d $MNT_PATH ]] && [[ ! -f $MNT_PATH ]]; then
clear
create_mount_path
elif [[ -f $MNT_PATH ]]; then
clear
echo "The given path $MNT_PATH is not a directory!"
echo ''
get_mount_path
fi
}
clear
get_mount_path
sudo apt update && sudo apt install -y software-properties-common
wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -
echo "deb https://download.ceph.com/debian-pacific/ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/ceph.list > /dev/null
echo "# deb-src https://download.ceph.com/debian-pacific/ $(lsb_release -cs) stable" | sudo tee -a /etc/apt/sources.list.d/ceph.list > /dev/null
sudo apt update
sudo apt install -y ceph-common
sudo tee /etc/ceph/ceph.client.$CLIENT_NAME.keyring &> /dev/null <<EOF
[client.${CLIENT_NAME}]
key = ${CLIENT_SECRET}
EOF
sudo tee /etc/ceph/ceph.client.$CLIENT_NAME.keyring.value &> /dev/null <<EOF
${CLIENT_SECRET}
EOF
echo "$MON_IP:$FS_PATH $MNT_PATH ceph name=$CLIENT_NAME,fs=$FS_NAME,recover_session=clean,_netdev 0 2" | sudo tee -a /etc/fstab &> /dev/null
sudo mount $MNT_PATH
)This script will prompt you for the following information:
- Absolute mount path that should be used for mounting the network file system on the virtual machine.
Once you have provided a valid answer to the previous prompt, the script will proceed to setup the required APT repository, install the required Ceph packages, create the Ceph client credential files, setup an fstab entry, and then mount the configured file system at the given mount path.
If everything executed as expected, you should now find that the Ceph file system is mounted and ready for use!
