Review of Software Quality Verification and Evaluation Methods
https://doi.org/10.55648/1998-6920-2023-17-1-92-106
Abstract
The basic concepts in the field of software quality testing and evaluation are defined. In particular, the concepts of verification, validation and software lifecycle and their role in the testing process are considered. The concept of an artifact in the software lifecycle is considered. The Russian and international standards in the field of software quality assessment are taken into account. The existing methods of software verification are regarded: expertise, static analysis, formal methods, dynamic and synthetic methods.
An overview of software quality assessment methods used in software development and testing cycles is presented. Functional and administrative factors affecting software quality are also considered. The article lists modern testing automation tools that can be used today. The review material of the article will be useful for software developers, novice testers and related specialists.
About the Authors
E. Y. Merzlyakova
Siberian State University of Telecommunications and Information Science (SibSUTIS)
Russian Federation
Russian Federation
Ekaterina Y.Merzlyakova, PhD (Engineering), Docent of the Department of Applied Mathematics and Cybernetics
630102, Novosibirsk, Kirova str., 86
tel. +7 383 2698 272
E. V. Yanchenko
Siberian State University of Telecommunications and Information Science (SibSUTIS)
Russian Federation
Russian Federation
Elena V.Yanchenko, PhD (Engineering), Docent of the Department of Applied Mathematics and Cybernetics
630102, Novosibirsk, Kirova str., 86
tel. +7 383 2698 272
References
1. Kulyamin V. V. Metody verifikatsii programmnogo obespecheniya [Software verification methods]. Moscow, Institut sistemnogo programmirovaniya, 2008, 111 p.
2. Semakhin A. M. Metody verifikatsii otsenki kachestva programmnogo obespecheniya [Software quality assessment verification methods]. Kurganskii gosudarstvennyi universitet, Kurgan, 2018. 150 p.
3. IEEE 1012-2004 Standard for Software Verification and Validation. IEEE, 2005.
4. GOST 28806-90 Kachestvo programmnykh sredstv. Terminy i opredeleniya [Quality of software. Terms and Definitions]. Gosudarstvennyy komitet SSSR po vychislitel'noy tekhnike i informatike, 1992.
5. ISO/IEC JTC 1/ SC7 Systems and software engineering - Systems and software Quality Rquirements and Evaluation (SQuaRE) - Guide to SQuaRE - Part 2: Software and systems engineering Geneva, Switzerland: ISO, 2014.
6. Informatsionnyye tekhnologii. Sistemnaya i programmnaya inzheneriya. Trebovaniya i otsenka kachestva sistem i programmnogo obespecheniya (SQuaRE). Trebovaniya k kachestvu gotovogo k ispol'zovaniyu programmnogo produkta (RUSP) i instruktsii po testirovaniyu [Information technology. System and software engineering. Requirements and quality assessment of systems and software (SQuaRE). Quality requirements for a ready-to-use software product (RUSP) and testing instructions]. Federal'noye agentstvo po tekhnicheskomu regulirovaniyu i metrologii, Standartinform, 2019.
7. Informatsionnyye tekhnologii. Sistemnaya i programmnaya inzheneriya. Trebovaniya i otsenka kachestva sistem i programmnogo obespecheniya (SQuaRE). Modeli kachestva sistem i programmnykh produktov [Information technology. System and software engineering. Requirements and quality assessment of systems and software (SQuaRE). Quality models of systems and software products]. Federal'noye agentstvo po tekhnicheskomu regulirovaniyu i metrologii, Standartinform, 2019.
8. IEEE 1028 Standard for Software Reviews. New York: IEEE, 1998.
9. Fagan M. E. Design and Code Inspections to Reduce Errors in Program Development. IBM Systems Journal, 1976, vol. 15, no. 3, pp. 182-211.
10. Kazman R., Bass L., Abowd G., Webb M. SAAM: A Method for Analyzing the Properties of Software Architectures. Proc. of 16-th International Conference on Software Engineering, 1994, pp. 81-90.
11. Plum Hall Inc, available at: http://www.plumhall.com/ (accessed 31.01.2023).
12. LDRA, available at: https://ldra.com/ (accessed 3.02.2023).
13. Jtest, available at: https://www.parasoft.com/ (accessed 3.02.2023).
14. Shleye M. Qt 5.10. Professional'noye programmirovaniye na C++ [Qt 5.10. Professional programming in C++]. BKHV-Peterburg, 2018.
15. PVS Studio – staticheskiy analizator na strazhe kachestva, zashchishchonnosti (SAST) i bezopasnosti koda [PVS Studio - a static analyzer guarding the quality, security (SAST) and security of the code], available at: https://pvs-studio.com/ru/pvs-studio/ (accessed 10.05.2022).
16. Kamkin A. S. Vvedeniye v formal'nyye metody verifikatsii programm [Introduction to formal methods of program verification]. Moskva, MGU imeni M.V. Lomonosov, 2018.
17. AVISPA - Automated Validation of Internet Security Protocols and Applications, available at: https://www.avispa-project.org/ (accessed 3.02.2023).
18. Blanshe B. Modelirovaniye i proverka protokolov bezopasnosti s pomoshch'yu prikladnogo ischisleniya Pi i proverki [Modeling and verification of security protocols using applied calculus Pi and verification] Osnovy i tendentsii v oblasti konfidentsial'nosti i bez-opasnosti. Oktyabr' 2016, no.1(1-2). pp.1-135.
19. Perevyshina, Ye. A., Babenko L. K. Analiz stoykosti k atakam kriptograficheskikh protokolov s ispol'zovaniyem formal'nogo verifikatora SPIN [Analysis of resistance to attacks of cryptographic protocols using the formal verifier SPIN]. Izvestiya YUFU. Tekhnicheskiye nauki. 2019, no. 5(207). pp. 58-68.
20. Gurin R.Ye., Rudakov I.V., Rebrikov A.V. Metody verifikatsii programmnogo obespecheniya [Software verification methods]. Setevoye nauchnoye izdaniye. Nauka i obrazovaniye. MGTU im. N.E. Baumena. Elektronnyy zhurnal no. 10, 2015. pp.235-251.
21. Perotskaya V. N., Gradusov D. A. Osnovy testirovaniya programmnogo obespecheniya [Fundamentals of software testing]. Vladimir, Izdatel'stvo VLGU, 2017.
22. Top 10 besplatnykh instrumentov dlya avtomatizirovannogo testirovaniya [Top 10 Free Automated Testing Tools], available at: https://medium.com/nuances-of-programming/топ-10-бесплатных-автоматизированных-инструментов-для-тестирования-32e461a78298 (accessed 3.02.2023).
23. Utting M., Pretschner A., Legeard B. A Taxonomy of Model-Based Testing. Technical Report, Department of Computer Science, The University of Waikato, New Zealand, 2006.
24. Delgado N., Gates A. Q., Roach S. A Taxonomy and Catalog of Runtime Software-Fault Monitoring Tools. IEEE Transactions on Software Engineering, 2004, vol. 30, no. 12, pp. 859-872.
25. Broy M., Jonsson B., Katoen J.-P., Leucker M., Pretschner A. (eds.). Model Based Testing of Reactive Systems. LNCS 3472, Springer, 2005.
26. Utting M., Legeard B. Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann, 2007.
27. Ambert F., Bouquet F., Chemin S., Guenaud S., Legeard B., Peureux F., Vacelet N., Utting M. BZ-TT: A tool-set for test generation from Z and B using constraint logic programming. Proc. of FATES’2002, 2002, pp. 105-119.
28. Ammann P., Black P. E. Abstracting formal specifications to generate software tests via model checking. Proc. of 18-th Digital Avionics Systems Conference, 1999, vol. 2, p. 10.
29. Engel C., Hahnle R. Generating unit tests from formal proofs. Y. Gurevich, B. Meyer, eds. Proc. of TAP 2007. LNCS 4454, Springer-Verlag, 2007. pp. 169–188.
30. Sen K., Agha G. CUTE and jCUTE: Concolic unit testing and explicit path model-checking tools. Proc. of Computer Aided Verification, 2006, pp.419-423.
31. Lee I., Kannan S., Kim M., Sokolsky O., Viswanathan M. Runtime Assurance Based On Formal Specifications. Proc. of International Conference on Parallel and Distributed Processing Techniques and Applications PDPTA’1999, 1999, pp. 279-287.
32. Havelund K., Rosu G. Monitoring Java Programs with Java PathExplorer. Electronic Notes in Theoretical Computer Science, 2004, vol. 55, no. 2, pp. 1-18.
33. Cavalli A., Gervy C., Prokopenko S. New approaches for passive testing using an Extended Finite State Machine specification. Information and Software Technology, 2003, vol. 45, no. 12, pp. 837-852.
34. Chen D., Wu J. Chi H. Passive testing on TCP. Proc. of International Conference on Communication Technology ICCT 2003, 2003, vol. 1, pp. 182-186.
35. Cheon Y., Leavens G. T. A runtime assertion checker for the Java Modeling Language (JML). Proc. of International Conference on Software Engineering Research and Practice (SERP’02), 2002, pp. 322-328.
36. Schoeller B. Strengthening Eiffel Contracts using Models. Proc. of Workshop on Formal Aspects of Component Software FACS’03, 2003, pp. 143-158.
37. Barnett M., Schulte W. Runtime verification of .NET contracts. Journal of Systems and Software, 2003, vol. 65, no. 3, pp. 199-208.
38. Clark E.M., Humburg O., Peled D. Verifikatsiya modelei programm: Model Checking [Verification of program models: Model Checking]. Moscow center for continuous mathematical education, Moscow, 2002. 416 p.
39. Rudakov I.V. Razrabotka i issledovanie sinteticheskogo metoda verifikatsii programmy s pomoshch'yu SMT-reshatelei [Development and research of a synthetic method for program verification using SMT solvers], Vestnik MGTU im. N.E. Baumana, Seriya Priborostroyeniye, 2016, no. 4(109), pp. 49-64.
40. Chernikov B.V. Upravlenie kachestvom programmnogo obespecheniya [Software quality management]. Moscow, Forum, 2012. 240 p.
41. Chernikov B.V., Poklonov B.E. Otsenka kachestva programmnogo obespecheniya [Software quality assessment]. Moscow, Forum:Infra-M, 2012. 400 p.
42. Sandler K., Budget T., Mayers G. Iskusstvo testirovaniya program [The art of software testing]. Moscow, Williams, 2015. 272 p.
Review
For citations:
Merzlyakova E.Y., Yanchenko E.V. Review of Software Quality Verification and Evaluation Methods. The Herald of the Siberian State University of Telecommunications and Information Science. 2023;17(1):92-106. (In Russ.) https://doi.org/10.55648/1998-6920-2023-17-1-92-106
Views:
1329
Email this article 
