Cryptography in Charmed Ceph

This section provides a comprehensive overview of the security features and cryptographic technologies available in Charmed Ceph. The implementation of these features may vary depending on specific configurations and use cases.

Charmed Ceph is built upon Ubuntu Ceph packages, inheriting many of their security capabilities. This section covers cryptographic approaches and security features in Ubuntu Ceph as well as Charmed Ceph-specific security features.

Ubuntu Ceph package cryptography

Here, we look at the security features inherent to Ubuntu Ceph packages, which form the foundation of Charmed Ceph’s security infrastructure.

Ceph internal communication

Ceph Messenger protocol

The Ceph Messenger v2 protocol, also known as msgr2, supports encryption in secure mode. The msgr2 protocol also ensures data integrity during transmission. This is done by using the AES-128-GCM stream cipher.

Cephx protocol

Authentication

The cephx protocol is used to authenticate components inside a Ceph cluster. Cephx, similar to Kerberos, uses shared secret keys for mutual authentication between clients and Ceph MONs. The protocol employs session keys and time-limited authentication tickets for ongoing communication with OSDs and metadata servers.

  • Shared secret keys: the core of the cephx protocol is based on shared secret keys for authentication. Both the client and Ceph Monitors (MONs) maintain a copy of the client’s secret key. This setup allows mutual authentication.
  • Session keys: when a client initiates communication with a Ceph MON, the monitor generates a session key. This session key is encrypted using the client’s permanent secret key. The client may contact any MON for this operation.
  • Authentication Tickets: similar to Kerberos, cephx uses an authentication ticket system. After initial authentication, clients receive a ticket from the monitor, which they use to authenticate themselves with Object Storage Daemons (OSDs) and metadata servers. These tickets have expiration times to prevent misuse if intercepted.
  • Message signatures: once authenticated, ongoing communications between clients and Ceph daemons are protected by message signatures. This ensures that messages are not tampered with during transmission

Authentication is implemented using symmetric key cryptography based on AES CBC (AES-128). Message signatures are implemented using HMAC-SHA256.

Authorization

To perform a specific operation on a cluster, a user must, first, be authenticated, and secondly, needs to possess the required privileges for the specific operation.

Ceph uses the term “capabilities”, or caps, to describe the permissions granted to an authenticated user to exercise the functionality of the monitors, OSDs, and metadata servers. Capabilities can also restrict access to data within a pool, a namespace within a pool, or a set of pools based on their application tags. A Ceph administrative user specifies the capabilities of a user when creating or updating that user.

Caps are specific to services, i.e. MON, OSD, Ceph Manager (MGR) and Ceph Metadata Server (MDS) services. In general, permissions for read, write and execute are defined for each service. Also, all services can restrict access to specific networks (in CIDR notation).

Additional permissions:

  • MON: no extra permissions defined.
  • OSD: besides the standard restrictions, OSDs also define class-read and class-write permissions. Access can be restricted to specific pools.
  • MGR: besides the standard restrictions, managers can also restrict access to specific commands for commands exported by manager modules.
  • MDS: access to CephFS can also be restricted by filesystem and path. Access to filesystem information, and the layout and snapshot features can also be restricted via caps. It’s also possible to restrict superuser access (root squash).

Cephadm management

Cephadm is a tool for deploying and managing Ceph clusters using containers. It can perform initial cluster setup, service and lifecycle management.

Cephadm uses Secure Shell (SSH) to connect to remote hosts; authentication is done via key authentication. By default SSH keys are auto-generated, and stored in the Ceph MON. Stored keys can be managed using the cephadm tool as well.

Cephadm also supports using Certificate Authority (CA) signed keys for SSH authentication across cluster nodes. In this setup, instead of needing a private key and public key, we instead need a private key and certificate created by signing that private key with a CA key.

Data at rest encryption

Charmed Ceph supports encryption of data at rest using the following solutions:

Linux Unified Key Setup-on-disk-format (LUKS): Ceph uses LUKS version 1, or LUKS1 for encrypting logical volumes associated with OSDs.

dm-crypt: This is the underlying disk encryption subsystem used by LUKS. When setting up encryption, dm-crypt is used to encrypt logical volumes. This will use AES-XTS ciphers.

Key management: The encryption keys used by LUKS are managed by Ceph.

Storage types

Ceph will provide three types of storage to clients via specific components listed below. Common to all these components is the ability to take advantage of msgr2 encryption if configured (see above).

RADOS Gateway object storage

Accessing Ceph object storage happens via the Rados Gateway (RGW) service. This service supports transport security via Secure Sockets Layer/Transport Layer Security (SSL/TLS) for encrypting client traffic; to do this it will need to be configured with certificate material for SSL/TLS.

Keystone authentication

RGW can be configured to authenticate against an OpenStack Keystone service. Using this, user-provided tokens would be validated against a Keystone service.

For this integration, the RGW has to be authenticated against Keystone, itself. This can be accomplished either by configuring an admin token, or, preferably, by setting up a service tenant, user and password for RGW in Keystone.

RGW can also support S3-like access and service secrets for authentication against Keystone.

RGW also supports the use of service tokens to authenticate against Keystone. Enabling this will cause an expired token given in the X-Auth-Token header to be allowed if coupled with a X-Service-Token header that contains a valid token with the accepted roles. This can allow long running processes using a user token in X-Auth-Token to function beyond the expiration of the token.

LDAP authentication

RGW can delegate the Ceph Object Gateway authentication to an LDAP server. This can be configured for native RGW as well as for S3-compatible authentication. Typically, SSL/TLS transport encryption would be configured to secure communication with an LDAP server. RGW must be configured with credentials to be able to bind to an LDAP server.

Multi-factor authentication

RGW’s S3 multi-factor authentication (MFA) feature allows users to require the use of a one-time password (OTP) when removing objects from certain buckets. The buckets need to be configured with versioning and be MFA-enabled. This can be done through the S3 API.

OTP tokens can be assigned to a user as well. Each token has a secret seed, and a serial ID that is assigned to it. Tokens are added to the user; they can be listed, removed, and re-synchronized.

STS authentication

The RGW service supports Amazon Secure Token Service (STS) by implementing a subset of AWS STS APIs. This functionality allows for the creation of temporary security credentials that can be used for identity and access management. These credentials enable users to make authenticated S3 API calls to the Ceph Object Gateway.

The AssumeRole API provides temporary credentials for cross-account access. The permissions of these credentials are determined by both the permission policies attached to the role and any policy attached to the AssumeRole API call. Parameters for this API include RoleArn, RoleSessionName, Policy, DurationSeconds, ProviderId, and WebIdentityToken.

RGW also supports an extension of STS named STS-Lite. This extension of STS reduces the load on external identity providers like Keystone and LDAP by providing temporary credentials after initial authentication. The GetSessionToken API is part of STS-Lite and returns temporary credentials that can be used for subsequent S3 calls.

Keycloak authentication

RGW integrates with Keycloak for identity and access management for federated authentication. The integration works like this:

  1. Via OpenID Connect (OIDC): RGW uses Keycloak via the OIDC protocol to authenticate users. Keycloak serves as the identity provider. Users authenticate with Keycloak, which issues a token that RGW verifies for access control.

  2. By using the web token returned by the authentication process, a mobile app or web app can call AssumeRoleWithWebIdentity, receive a set of temporary S3 credentials, and use those credentials to make S3 calls.

Server-side encryption

The RGW service supports server-side encryption (SSE) according to the Amazon SSE specifications. Keys are managed with one of five mechanisms:

  1. Customer keys: The SSE-C specification prescribes use of customer-managed keys. It will make use of AES256 symmetric encryption. RGW implements this as AES256-CBC. Also per the SSE-C specification, keys may be provided as a 128-bit MD5 digest.

  2. Amazon KMS: Per the SSE-KMS specification keys for encryption are managed by the Amazon KMS. RGW will encrypt object with AES256 either CBC or ECB using these keys

  3. Vault: RGW can be integrated with Hashicorp Vault to manage keys. As above, symmetric encryption will be performed with AES256. Integration with Vault requires the RGW service to be authenticated. Currently, RGW supports authentication via token authentication or a Vault Agent.

  4. Barbican: RGW can be integrated with Barbican, OpenStack’s Key Manager Service (KMS). To authenticate with Barbican, RGW must be set up for integration with Keystone as Barbican depends on this service for authentication and access control. When configured, server-side encryption keys will be managed in Barbican.

  5. KMIP: RGW can be integrated with a Key Management Interoperability Protocol (KMIP)-compatible service. Authentication is supported through mutual TLS, authenticating the KMIP server and RGW against each other. Server-side encryption using AES256 would then be implemented using the SSE-KMS specification.

For testing purposes, it’s also possible to hard-code a key to use for symmetric encryption via AES256. However, this should not be used in production.

QAT acceleration

RGW supports encryption and compression via the Intel Quick Assist Technology (QAT) accelerators. This can be set up to accelerate server-side encryption using AES256-CBC symmetric encryption.

Encryption via QAT can also be combined with compression for RGW objects.

CephFS file storage

CephFS provides filesystem storage to clients. Client access is regulated using the Ceph-native cephx protocol which performs authentication and authorization for clients. Ceph supports access control to specific filesystem and filesystem subtrees as well.

RBD block storage

The RADOS Block Device (RBD), Ceph’s block storage component can provide block devices backed by Ceph storage. Access to RBD is regulated using the native cephx protocol for authentication and authorization.

RBD encryption: Users can instruct Ceph to encrypt block device images utilizing the rbd encryption format commands. RBD supports AES128 and AES256 algorithms, with AES256 XTS-plain64 being the default.

Dashboard

The Ceph dashboard provides administrative capabilities for Ceph. Access to the dashboard can be secured via SSL/TLS.

The Ceph dashboard module also exposes an API, the Ceph RESTful API. Like regular dashboard access, this can be secured through SSL/TLS. The RESTful API can make use of JSON Web Tokens (JWTs) using the HMAC-SHA256 algorithm.

The Ceph dashboard can be configured to authenticate users against a SAML2.0 service.

SAML2.0 can make use of a host of cryptographic technologies; which ones are actually used largely depends on the backend:

  • AES128, AES192, AES256 and Triple-DES can be used for symmetric encryption
  • RSA-SHA family of algorithms for signatures
  • RSA-v1.5, RSA-OAEP for key transport

Summary of cryptographic components

The following is a list of cryptographic libraries and tools used in Ubuntu Ceph:

  • dm-crypt
  • libldap
  • LUKS
  • OpenSSL
  • pyca/cryptography library
  • Python hvac library
  • Python bcrypt
  • python-jwt
  • python-onelogin library
  • python-openssl
  • SSH
  • Vault

Charm cryptography

Charmed Ceph builds upon Ubuntu Ceph’s security features, providing additional configuration options and integration mechanisms. This section details the security enhancements specific to Charmed Ceph, and any differences, in support, to Ubuntu Ceph.

Juju

As a charmed product, Charmed Ceph naturally builds on the security features of Juju.

Juju is an open-source orchestration engine for managing application deployments. It incorporates several security features and cryptographic practices to safeguard data and operations, including:

  • TLS-encrypted communication: Juju uses TLS encryption for all communication between its controller’s API server and clients, the charmhub, and deployed application units.
  • User authentication: Juju supports several methods for user authentication, including macaroons, JWTs (for Juju with JAAS), SSH keys, and passwords.
  • Agent authentication: each Juju agent authenticates itself with the controller using a password, adding a layer of security to agent-controller interactions.
  • Rate limiting: authentication requests from Juju unit agents to the controller are rate-limited to prevent abuse and potential denial-of-service (DOS) attacks.
  • Database security: access to the database by controllers, agents, or administrators requires authentication. Moreover, passwords stored in the database are hashed and salted to protect against unauthorized access.
  • Filesystem permissions: a minimum access policy is enforced to restrict filesystem permissions, reducing the risk of unauthorized data access.
  • Secrets management: Juju supports managing sensitive configurations as secrets. Secrets can also be managed via Hashicorp Vault.

Ceph internals

Charmed Ceph can make use of the security features of the Ceph messenger protocol. Refer to Cryptography in Ubuntu Ceph above for details around cryptographic techniques used.

Data at rest encryption

Charmed Ceph supports data at rest encryption (for details, refer to Cryptography in Ubuntu Ceph above).

While Charmed Ceph can use native Ceph mechanisms for OSD key management, reference deployments leverage the Vault charm for enhanced security.

Key management with Vault

Reference configurations use Hashicorp Vault for data-at-rest key management. The implementation relies on tools built on the hvac Python library, particularly the vaultlocker tool, and the Vault charm:

  • Authentication: uses AppRole with a secret_id
  • Access Control: employs CIDR-based control to restrict access
  • Key Generation: vaultlocker creates 4096-bit keys for dmcrypt/LUKS

Vault charm integration

The Vault charm centralizes key management, utilizing the Vault snap to manage Hashicorp Vault software. Access to Vault is secured using SSL/TLS client certificates.

Storage types

As with native Ceph, Charmed Ceph will provide three types of storage to clients via specific components as outlined below.

RGW object storage

Accessing Ceph object storage happens via the RGW service deployed by the ceph-radosgw charm. This charm supports transport security via SSL/TLS for encrypting client traffic. Certificate material for SSL/TLS can be configured directly on the charm or alternately, ceph-radosgw can make use of the tls-certificates interface to deploy certificates from a tls-certificate provider, e.g. the Vault charm. The ceph-radosgw charm would configure an Apache2 server with those certificates to terminate SSL/TLS.

The ceph-radosgw charm supports authenticating against OpenStack Keystone via the identity-service charm interface.

See Cryptography in Ubuntu Ceph for more details on security and the RGW service.

CephFS file storage

Access to CephFS is supported via the ceph-fs charm. Access to CephFS is regulated through cephx authentication and authorization. See Cryptography in Ubuntu Ceph for details.

RBD block storage

The Ceph RBD component can provide block devices backed by Ceph storage. The ceph-mon charm provides support for this via the ceph-client interface. As for CephFS, access is regulated via cephx authentication and authorization. RBD also supports block device encryption.

NFS file storage

See Cryptography in Ubuntu Ceph for details.

NVMe block devices

SeeCryptography in Ubuntu Ceph for details.

Dashboard

The Ceph dashboard provides administrative capabilities for Ceph. In Charmed Ceph this is supported via the ceph-dashboard charm.

Access to the dashboard can be secured via SSL/TLS. Certificate material can be configured either directly on the ceph-dashboard charm, or can be configured via the tls-certificate interface by a provider such as the Vault charm.

The Ceph dashboard module also exposes an API, the Ceph RESTful API. Like regular dashboard access, this can be secured through SSL/TLS through the same mechanisms.

The ceph-dashboard charm also supports authentication against a SAML2.0 backend.

For more details on dashboard security, refer to Cryptography in Ubuntu Ceph.

Previous Explanation Next Enabling full disk encryption

This page was last modified 7 days ago. Help improve this document in the forum.