Jul
ZFS Second Level ARC - L2ARC - Testing shows 20x thoughtput with 8x better latency
Testing The Technology that allows ZFS to exploit Flash
A new blog posting by Brendan Gregg, co-author of “Solaris Performance and Tools”, explains an exciting new ZFS feature called L2ARC - the second level ARC.
ARC - Adaptive Replacement Cache - is the algorithm that ZFS uses to keep pages in memory to improve performance. Think of the ARC as that section of DRAM reserved by ZFS for buffering. Brendan observer, “An ARC read miss would normally read from disk, at millisecond latency (especially random reads). The L2ARC sits in-between, extending the main memory cache using fast storage devices - such as flash memory based SSDs (solid state disks).”
Brendan uses a lab machine to showcase the use of flash and L2ARC:
| Layer | Medium | Total Capacity |
| ARC | DRAM | 128 Gbytes |
| L2ARC | 6 x SSDs | 550 Gbytes |
| Storage Pool | 44 Disks | 17.44 Tbytes (mirrored |
In Brendan’s work load, 20 clients access the lab server over NFS. The test does read random workload of 8 Kbyte records across 500 GBytes. (More that will fit into the ARC.)
With iostat, he reports that the I/Os take between 9 and 10 milliseconds to complete. Many of the I/Os are waiting for disks heads to seek and platters to rotate.
With the L2ARC configures, there is quite an improvement. The throughput is 8 times greater and the latency is 20x faster - 0.4 to 0.6 milliseconds.
More Information
Brendan Gregg’s ZFS L2ARC Blog posting
Adam Leventhal - Can Flash Memory Become the Foundation for a New Tier in the Storage Hierarchy?
