Canonical and Intel introduce hardware acceleration in Ceph to reduce CPU load and enhance storage efficiency in data centers.
Efficient storage management is crucial for performance, scalability, and cost reduction in IT infrastructures. As data volumes grow exponentially, traditional redundancy strategies such as replication and erasure coding (EC) are no longer sufficient to optimize storage resources. To address this, Canonical and Intel have integrated Intel® Quick Assist Technology (QAT) into Ceph, significantly reducing CPU consumption for compression tasks without sacrificing performance.
The Challenge of Compression in Storage Systems
Data compression has long been used to reduce storage costs, but it traditionally comes at the expense of CPU usage, which can degrade system performance. This issue is especially critical in software-defined storage solutions like Ceph, where each node must maintain an optimal balance between performance, capacity, and energy efficiency.
Intel® QAT offloads compression tasks from the primary CPU, allowing storage nodes to process data faster while freeing up computing resources for other workloads. With the release of 4th and 5th Gen Intel® Xeon® processors, QAT is now built directly into the CPU, eliminating the need for additional hardware and making it easier to deploy across data centers and cloud environments.
Performance Benchmark: How QAT Improves Ceph Efficiency
Canonical conducted performance testing in a four-node Ceph cluster using NVMe disks, with MinIO Warp simulating object storage workloads at a 4:3 compression ratio (25% storage savings).
Key Performance Results
| Metric | No Compression | Compression (No QAT) | Compression + QAT |
|---|---|---|---|
| Write Bandwidth | 4.66 GBps | 50% reduction | 5.05 GBps (+8%) |
| Read Bandwidth | 21.86 GBps | 11.28 GBps (-48%) | 20.88 GBps (-4%) |
| CPU Utilization | Normal | +150% increase | No additional load |
By integrating Intel® QAT, Ceph maintains high performance, and in some cases, even improves write bandwidth, while preventing the CPU from becoming a bottleneck.
When is Compression Efficient?
Not all data benefits equally from compression. While uncompressed formats can yield significant savings, already compressed files such as JPEG or H.264 show little to no improvement.
| Data Type | Compression Ratio | Storage Savings |
|---|---|---|
| Structured Data (CSV, JSON, Text) | 1.33 | 25% |
| Compressed Images (JPEG) | 1.01 | 1% |
| Uncompressed Video (RAW YUV) | 3.13 | 68% |
| Compressed Video (H.264) | 1.00 | 0% |
For system administrators, this means QAT-enabled compression is most effective in environments with large volumes of structured data, logs, databases, and uncompressed video, whereas its benefits are limited for repositories of already compressed images and videos.
Deploying Intel® QAT in Ceph
For testing or production deployment, Canonical recommends using MicroCeph, a lightweight Ceph implementation that simplifies setup and management.
Steps to Enable Intel® QAT in Ceph
- Install MicroCeph with QAT support:
snap install microceph --channel=squid/edge/qat --devmode - Install the QAT engine on each storage node:
sudo apt-get -y install qatengine - Enable QAT compression in Ceph:
ceph config set client.radosgw.gateway qat_compressor_enabled true - Configure compression for object storage:
radosgw-admin zone placement modify \ --rgw-zone default \ --placement-id default-placement \ --storage-class STANDARD \ --compression zlib - Verify compression status with:
radosgw-admin bucket stats
These steps allow storage administrators to significantly reduce storage usage while maintaining high performance, freeing up CPU cycles for other tasks.
Benefits for Data Centers and Cloud Providers
The integration of QAT with Ceph delivers key benefits for data centers and cloud infrastructure:
- Lower operational costs by reducing the hardware required for storage.
- Higher storage density, enabling greater capacity without increasing server count.
- Optimized performance, maintaining high data throughput without overloading CPUs.
- Energy efficiency, lowering power consumption and improving sustainability.
With data volumes surging across industries such as telecom, banking, healthcare, and e-commerce, these optimizations are essential for scalable and cost-effective cloud infrastructures.
Conclusion: A New Standard for Enterprise Storage
The combination of Ceph, Intel® QAT, and Ubuntu provides system administrators and data center operators with a highly efficient, scalable, and high-performance storage solution. By lowering storage costs without sacrificing performance, this integration becomes a key tool for cloud and enterprise environments.
