Leveraging Data Deduplication to Improve the Performance of Primary Storage Systems in the Cloud

Leveraging Data Deduplication to Improve the Performance of Primary Storage Systems in the Cloud

Abstract— Leveraging Data Deduplication to Improve the Performance of Primary Storage Systems in the Cloud. With the explosive growth in data volume, the I/O bottleneck has become an increasingly daunting challenge for big data < Final Year Projects 2016 > analytics in the Cloud. Recent studies have shown that moderate to high data redundancy clearly exists in primary storage systems in the Cloud. Our experimental studies reveal that data redundancy exhibits a much higher level of intensity on the I/O path than that on disks due to relatively high temporal access locality associated with small I/O requests to redundant data. Moreover, directly applying data deduplication to primary storage systems in the Cloud will likely cause space contention in memory and data fragmentation on disks.