分布式文件系统存储效率优化研究-计算机系统结构专业论文.docxVIP

华 华 中 科 技 大 学 硕 士 学 位 论 文 II II Abstract With the rapid increasing data produced by users, the storage pressure for distributed file systems has never been so challenging. Major company usually uses cluster systems to store massive amounts of data, and since the cluster scale is growing up to thousands of nodes, the cost for data storage is becoming more and more expensive. To achieve high reliability, distributed file system usually uses triplication, but it wastes a lot of disk space and has low storage efficiency, so the optimization of storage efficiency is becoming a hot area of research. To improve storage efficiency, this paper provides a storage strategy based on erasure code. Through the parity blocks, erasure code can ensure the data reliability with only one copy of the source data. But the disadvantages of this approach are the extra time of encode process and decode process, and the delay time when large amount of concurrent clients access the only copy of source data at the same time. In order to reduce the effect of erasure code’s latency, the strategy only encoding cold data in the system. Since the new data are most likely to be the hot data, we use triplication to store the file. When the data has been determined to be cold, it will be coded to generate parity blocks and reduce the number of source blocks to only one copy. In other words, this strategy combined the strengths of both triplication and erasure coding. And the encoding process with an independent unit called “striping”. The result shows this strategy has upgraded the storage efficiency and fault-tolerant capability of distributed file system. And the average client response time is similar to the system that uses triplication. In conclusion, this erasure code based storage strategy achieved balance in storage cost, data reliability and system performance. Key words： Storage Efficiency, Distributed File System, Erasure Code, Reliability III III 目 录 摘 要 I HYPERLINK \l _bookmark0 Abstra