sibsutisВестник СибГУТИThe Herald of the Siberian State University of Telecommunications and Information Science1998-6920СибГУТИ10.55648/1998-6920-2022-16-1-77-88sibsutis-122Research ArticleСтатьиАнализ эффективности потоково-локальной сборки мусора в распределённых системах хранения и обработки данныхApplication of Thread-Local Garbage Collection to Distributed Systems for Large-Scale Data ProcessingФилатовА. Ю.FilatovA. Yu.

Филатов Александр Юрьевич, ассистент кафедры программирования ММФ НГУ; ведущий инженер Новосибирского исследовательского центра Huawei

Новосибирск

Alexander Yu. Filatov, Assistant, Novosibirsk State University; Leading engineer, Huawei Research Center

Novosibirsk

a.filatov@g.nsu.ruМихеевВ. В.MikheevV. V.

Михеев Виталий Витальевич, старший преподаватель кафедры программирования ММФ НГУ; старший эксперт Новосибирского исследовательского центра Huawei

Новосибирск

Vitaly V. Mikheev, Lecturer, Novosibirsk State University; Senior expert, Huawei Research Center

Novosibirsk

mikheev.vitaly@huawei.comНГУ; Новосибирский исследовательский центр HuaweiРоссияNovosibirsk State University; Huawei Research CenterRussian Federation202213052022017788Copyright © Филатов А.Ю., Михеев В.В., 20222022Филатов А.Ю., Михеев В.В.Filatov A.Y., Mikheev V.V.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.sibsutis.ru/jour/article/view/122

В данной работе рассматривается потоково-локальная сборка мусора – техника автоматического управления памятью, которая использует свойство объекта быть доступным только одному потоку исполнения. Возможность выделять и переиспользовать динамическую память независимо в каждом потоке, т.е. без дорогостоящих операций синхронизации, улучшает производительность и масштабируемость прикладных программ. В работе анализируется применимость данной техники к распределённым системам хранения и обработки данных, используемым в промышленности. Результаты свидетельствуют о том, что потоковолокальное управление памятью позволяет в полной мере задействовать ресурсы вычислительной системы и является многообещающим подходом к сборке мусора в высоконагруженных средах исполнения.

Thread-local garbage collection (TLGC) is a technique of automatic memory management that associates memory locations with a specific application thread. These memory areas (thread-local heaps) support independent processing allowing other mutator threads to be concurrently executed. Improved scalability and throughput make thread-local memory manager an attractive alternative to conventional GC algorithms. This paper discusses effectiveness of thread-local GC applied to distributed systems for large-scale data processing. Experimental results show that the proposed approach increases overall system throughput and proves that TLGC is a suitable choice for memory-intensive fault-tolerant distributed systems.

сборка мусоравиртуальная машинапотоково-локальные кучиNUMAJVMgarbage collectionJVMthread local heapsNUMAReferencesDoligez D., Leroy X. A concurrent, generational garbage collector for a multithreaded implementation of ml // Proc. 20th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, 1993. P. 113–123.Doligez D., Leroy X. A concurrent, generational garbage collector for a multithreaded implementation of ml. Proc. 20th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, 1993, pp. 113–123.Domani T., Goldshtein G., Kolodner E., Lewis E., Petrank E, Sheinwald D. Thread-local heaps for java. // In SIGPLAN Not, 2002. P. 76–87.Domani T., Goldshtein G., Kolodner E., Lewis E., Petrank E, Sheinwald D. Thread-local heaps for java. In SIGPLAN Not, 2002, pp. 76–87.Apache Hadoop® official website [Электронный ресурс] // Apache Hadoop: [сайт]. URL: https://hadoop.apache.org/ (дата обращения: 01.01.2022).Apache Hadoop® official website [Electronic resource]. Apache Hadoop: [website]. URL: https://hadoop.apache.org/ (access date: 01.01.2022).Apache Spark official website [Электронный ресурс] // Apache Spark: [сайт]. URL: https://spark.apache.org/ (дата обращения: 01.01.2022).Apache Spark official website [Electronic resource]. Apache Spark: [website]. URL: https://spark.apache.org/ (access date: 01.01.2022).Filatov A., Mikheev V. Evaluation of thread-local garbage collection // Proc. 2020 Ivannikov Memorial Workshop (IVMEM), 2020. P. 15–21.Filatov A., Mikheev V. Evaluation of thread-local garbage collection. Proc. 2020 Ivannikov Memorial Workshop (IVMEM), 2020, pp. 15–21.Purdom P., Stigler S., Cheam T. Statistical investigation of three storage allocation algorithms // BIT Numerical Mathematics. 1971. P. 187–195.Purdom P., Stigler S., Cheam T. Statistical investigation of three storage allocation algorithms. BIT Numerical Mathematics. 1971, pp. 187–195.Sleator D., Tarjan R. Self-adjusting binary search trees // J. ACM. 1985. P. 682–686.Sleator D., Tarjan R. Self-adjusting binary search trees. J. ACM. 1985, pp. 682–686.Agesen O. Gc points in a threaded environment // Technical report, SMLI TR-98-70. 1998.Agesen O. Gc points in a threaded environment. Technical report, SMLI TR-98-70. 1998.Dean J., Ghemawat S. Mapreduce: Simplified data processing on large clusters // Commun. ACM, 2008. P. 107–113.Dean J., Ghemawat S. Mapreduce: Simplified data processing on large clusters.Zaharia M., Chowdhury M., Franklin M., Shenker S., Stoica I. Spark: Cluster computing with working sets // Proc. 2nd USENIX Conference on Hot Topics in Cloud Computing, 2010. P. 10.Commun. ACM. 2008, pp. 107–113.White T. Hadoop: The Definitive Guide // O’Reilly Media, Inc., 4th ed., 2015.Zaharia M., Chowdhury M., Franklin M., Shenker S., Stoica I. Spark: Cluster computing with working sets. Proc. 2nd USENIX Conference on Hot Topics in Cloud Computing, 2010, p. 10.HiBench source repository [Электронный ресурс] // GitHub: [сайт]. URL: https://github.com/Intel-bigdata/HiBench (дата обращения: 01.01.2022).White T. Hadoop: The Definitive Guide. O’Reilly Media, Inc., 4th ed., 2015.MacQueen J. Some methods for classification and analysis of multivariate observations // Berkeley Symposium on Mathematical Statistics and Probability, 1967. P. 281–297.HiBench source repository [Electronic resource]. GitHub: website URL https://github.com/Intel-bigdata/HiBench (access date: 01.01.2022).Jones S., Launchbury J., Peyton Jones S. Unboxed values as first class citizens // Proc. ACM Conference on Functional Programming and Computer Architecture, 1991. P. 636–666.MacQueen J. Some methods for classification and analysis of multivariate observations. Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 281–297.O’Malley O. TeraByte Sort on Apache Hadoop [Электронный ресурс] // TeraByte Sort: [сайт]. URL: http://sortbenchmark.org/YahooHadoop.pdf (дата обращения: 01.01.2022).Jones S., Launchbury J., Peyton Jones S. Unboxed values as first class citizens. Proc. ACM Conference on Functional Programming and Computer Architecture, 1991, pp. 636–666.Berger E., McKinley K., Blumofe R., Wilson P. Hoard: A scalable memory allocator for multithreaded applications // SIGARCH Comput. Archit. News. 2000. P. 117–128.O’Malley O. TeraByte Sort on Apache Hadoop [Electronic resource]. TeraByte Sort: [website]. URL: http://sortbenchmark.org/YahooHadoop.pdf (access date: 01.01.2022).Vee V.-Y., Hsu W.-J. A scalable and efficient storage allocator on shared memory multiprocessors // Proc. 1999 International Symposium on Parallel Architectures, Algorithms and Networks, 1999. P. 230.Berger E., McKinley K., Blumofe R., Wilson P. Hoard: A scalable memory allocator for multithreaded applications. SIGARCH Comput. Archit. News. 2000, pp. 117–128.Kirsch C., Payer H., Röck H. Hierarchical plabs, clabs, tlabs in hotspot // Proc. International Conference on Systems (ICONS), 2012.Vee V.-Y., Hsu W.-J. A scalable and efficient storage allocator on shared memory multiprocessors. Proc. 1999 International Symposium on Parallel Architectures, Algorithms and Networks, 1999, p. 230.Anderson A. Optimizations in a private nursery-based garbage collector // Proc. 2010 International Symposium on Memory Management, 2010. P. 21–30.Kirsch C., Payer H., Röck H. Hierarchical plabs, clabs, tlabs in hotspot. Proc. International Conference on Systems (ICONS), 2012.Mole M., Jones R., Kalibera T. A study of sharing definitions in thread-local heaps // ICOOOLPS, 2012.Anderson A. Optimizations in a private nursery-based garbage collector. Proc. 2010 International Symposium on Memory Management, 2010, pp. 21–30.Mole M. A study of thread-local garbage collection for multi-core systems // PhD thesis, University of Kent, 2015.Mole M., Jones R., Kalibera T. A study of sharing definitions in thread-local heaps. ICOOOLPS, 2012, 21.Mole M. A study of thread-local garbage collection for multi-core systems. PhD thesis, University of Kent, 2015.Mole M. A study of thread-local garbage collection for multi-core systems. PhD thesis, University of Kent, 2015.

The authors declare that there are no conflicts of interest present.