Utilizing real service simulation techniques to investigate distributed storage systems' latency performance based on an examination of Amazon S3
Abstract
Current erasure codes rely heavily on data nodes to generate the parity nodes. The greater the tolerance for error, and the more "If we can increase the number of parity nodes, we may increase our chances of restoring the original data. As the number of parity nodes grows, the storage overhead will rise, and the repair burden on data nodes will rise as well, because data nodes are queried often to help in the repair of parity nodes. If a global parity node fails in LRC [25, 26], for instance, all data nodes must be fixed. It will take more time to process read requests for data nodes as a result of the "increasing demands on the network's data nodes. An application where frequent data retrievals are unwelcome is a Google search.
In an effort to cut down on waiting time, "produces both data and parity nodes, the latter of which can take over part of the repair work normally done by the former. In other words, the number of data nodes that may be accessed remains constant, regardless of whether or not a parity node is functioning. It would appear that parity nodes incur additional storage costs. Generating parity nodes using parity nodes can assist decrease access latency without raising or decreasing the storage requirements if the architecture is sound ", as we shall show in the following sections [27, 28], above your head.
In this research, we'll compare and contrast the effectiveness of "Hierarchical Tree Structure Code (HTSC) and High Failure-tolerant Hierarchical Tree Structure Code (FH HTSC)."
Keywords
Hierarchical Tree Structure, Data Retrieval, Data NodesHow to Cite
References
. S. Ghemawat, H. Gobioff, and S.-T. Leung, “The Google file system,” in ACM SIGOPS Operating Systems Review, vol. 37, no. 5. ACM, 2003, pp. 29–43.
. M. Foley, “High availability HDFS,” in 28th IEEE Conference on Massive Data Storage, MSST, vol. 12, 2012.
. C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li,S. Yekhanin et al., “Erasure coding in Windows Azure storage,” in USENIX ATC, 2012, pp. 15–26.
. N. B. Shah, K. Lee, and K. Ramchandran, “The MDS queue: Analysing the latency performance of erasure codes,” in IEEE International Symposium on Information Theory (ISIT), 2014, pp. 861–865.
. B. Y. Kong, J. Jo, H. Jeong, M. Hwang, S. Cha, B. Kim, and I.-C. Park, “Low- complexity low-latency architecture for matching of data encoded with hard systematic error-correcting codes,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 22, no. 7, pp. 1648–1652, 2014.
. K. Rashmi, N. B. Shah, D. Gu, H. Kuang, D. Borthakur, and K. Ramchan- dran, “A solution to the network challenges of data recovery in erasure-coded distributed storage systems: A study on the Facebook warehouse cluster,” in Presented as part of the 5th USENIX Workshop on Hot Topics in Storage and File Systems. USENIX, 2013.
. A. Fikes, “Storage architecture and challenges,” Talk at the Faculty Summit, 2010.
. D. Ford, F. Labelle, F. I. Popovici, M. Stokely, V.-A. Truong, L. Barroso,C. Grimes, and S. Quinlan, “Availability in globally distributed storage sys- tems.” in OSDI, 2010, pp. 61–74.
. A. G. Dimakis, P. B. Godfrey, Y. Wu, M. J. Wainwright, and K. Ramchan- dran, “Network coding for distributed storage systems,” IEEE Transactions on Information Theory, vol. 56, no. 9, pp. 4539–4551, 2010.
. K. V. Rashmi, N. B. Shah, and P. V. Kumar, “Optimal exact-regenerating codes for distributed storage at the msr and mbr points via a product-matrix construction,” IEEE Transactions on Information Theory, vol. 57, no. 8, pp. 5227–5239, 2011.
. V. R. Cadambe, S. A. Jafar, H. Maleki, K. Ramchandran, and C. Suh, “Asymp- totic interference alignment for optimal repair of MDS codes in distributed data storage,” 2011.
. N. B. Shah, K. Rashmi, P. V. Kumar, and K. Ramchandran, “Explicit codes minimizing repair bandwidth for distributed storage,” in Information Theory Workshop (ITW), 2010 IEEE. IEEE, 2010, pp. 1–5.
. V. R. Cadambe, S. A. Jafar, and H. Maleki, “Distributed data storage with minimum storage regenerating codes-exact and functional repair are asymp- totically equally efficient,” arXiv preprint arXiv:1004.4299, 2010.
. N. B. Shah, K. Rashmi, P. V. Kumar, and K. Ramchandran, “Interference alignment in regenerating codes for distributed storage: Necessity and code constructions,” IEEE Transactions on Information Theory, vol. 58, no. 4, pp. 2134–2158, 2012.
. A. Duminuco and E. Biersack, “A practical study of regenerating codes for peer-to-peer backup systems,” in 29th IEEE International Conference on Dis- tributed Computing Systems. IEEE, 2009, pp. 376–384.
. A. Duminuco and E. W. Biersack, “Hierarchical codes: A flexible trade-off for erasure codes in peer-to-peer storage systems,” Peer-to-peer Networking and Applications, vol. 3, no. 1, pp. 52–66, 2010.
. M. Sathiamoorthy, M. Asteris, D. Papailiopoulos, A. G. Dimakis, R. Vadali,S. Chen, and D. Borthakur, “Xoring elephants: Novel erasure codes for big data,” in Proceedings of the 39th international conference on Very Large Data Bases. VLDB Endowment, 2013, pp. 325–336.
. J. Li and B. Li, “Erasure coding for cloud storage systems: A survey,” Ts- inghua Science and Technology, vol. 18, no. 3, pp. 259–272, 2013.
. A. G. Dimakis, K. Ramchandran, Y. Wu, and C. Suh, “A survey on network codes for distributed storage,” Proceedings of the IEEE, vol. 99, no. 3, pp. 476–489, 2011.
. A. Rudra, P. K. Dubey, C. S. Jutla, V. Kumar, J. R. Rao, and P. Rohatgi, “Ef- ficientRijndael encryption implementation with composite field arithmetic,” in Cryptographic Hardware and Embedded Systems?CHES 2001. Springer, 2001, pp. 171–184.
. J. Brutlag, “Speed matters for Google web search,” Google. June, 2009.
. L. Huang, S. Pawar, H. Zhang, and K. Ramchandran, “Codes can reduce queueing delay in data centers,” in IEEE International Symposium on Infor- mation Theory (ISIT), 2012, pp. 2766–2770.
. N. B. Shah, K. Lee, and K. Ramchandran, “When do redundant requests re- duce latency?” in the 51st Annual Allerton Conference on Communication, Control, and Computing. IEEE, 2013, pp. 731–738.
. G. Liang and U. C. Kozat, “TOFEC: Achieving optimal throughput-delay trade-off of cloud storage using erasure codes,” in Proceedings of INFOCOM. IEEE, 2014, pp. 826–834.
. G. Joshi, Y. Liu, and E. Soljanin, “On the delay-storage trade-off in content download from coded distributed storage systems,” IEEE Journal on Selected Areas in Communications, vol. 32, no. 5, pp. 989–997, 2014.
. Y. Xiang, T. Lan, V. Aggarwal, and Y. F. R. Chen, “Joint latency and cost optimization for erasurecoded data center storage,” ACM SIGMETRICS Per- formance Evaluation Review, vol. 42, no. 2, pp. 3–14, 2014.
. B. Li, A. Ramamoorthy, and R. Srikant, “Mean-field-analysis of coding versus replication in cloud storage systems,” in Proceedings of INFOCOM. IEEE, 2016.
. G. Liang and U. C. Kozat, “Fast Cloud: Pushing the envelope on delay perfor- mance of cloud storage with coding,” IEEE/ACM Transactions on Network- ing, vol. 22, no. 6, pp. 2012–2025, 2014.
. G. Ananthanarayanan, S. Agarwal, S. Kandula, A. Greenberg, I. Stoica,D. Harlan, and E. Harris, “Scarlett: coping with skewed content popularity in mapreduce clusters,” in Proceedings of the sixth conference on Computer systems. ACM, 2011, pp. 287–300.
. A. Kala Karun and K. Chitharanjan, “A review on hadoophdfs infrastructure extensions,” in Conference on Information & Communication Technologies (ICT). IEEE, 2013, pp. 132–137.
. M. Harchol-Balter, Performance Modeling and Design of Computer Systems: Queueing Theory in Action. Cambridge University Press, 2013.
. H. A. David and H. N. Nagaraja, Order statistics. Wiley Online Library, 1981.
. M. Rahman and L. Pearson, “Moments for order statistics in shift parameter exponential distribution,” Journal of Statistical Research, vol. 36, no. 1, pp. 75–83, 2002.
. S. B. Wicker and V. K. Bhargava, Reed-Solomon codes and their applications. John Wiley & Sons, 1999.
. Q. Shuai, V. O. K. Li, and Y. Zhu, “Performance models of access latency in cloud storage systems,” in Fourth Workshop on Architectures and Systems for Big Data, 2014.
. M. Blaum, J. Brady, J. Bruck, and J. Menon, “Evenodd: An efficient scheme for tolerating double disk failures in raid architectures,” IEEE Transactions on Computers, vol. 44, no. 2, pp. 192–202, 1995.
. L. Xu and J. Bruck, “X-code: Mds array codes with optimal encoding,” IEEE Transactions on Information Theory, vol. 45, no. 1, pp. 272–276, 1999.
. P. Corbett, B. English, A. Goel, T. Grcanac, S. Kleiman, J. Leong, andS. Sankar, “Row-diagonal parity for double disk failure correction,” in Pro- ceedings of the 3rd USENIX Conference on File and Storage Technologies, 2004, pp. 1–14.
License
Copyright (c) 2024 SU RIGUGE, DR MOHAMMAD NIZAMUDDIN
This work is licensed under a Creative Commons Attribution 4.0 International License.
Individual articles are published Open Access under the Creative Commons Licence: CC-BY 4.0.
