Research of Video Stream Intensity Limits in UAV FPV Control in Frame Prediction Mode. Part I: Models and Methods
https://doi.org/10.55648/1998-6920-2024-18-3-115-139
Abstract
Currently, unmanned aerial vehicles have found wide application in various spheres of the national economy. FPV control refers to a method of controlling unmanned aerial vehicles in which the video stream from the unmanned aircraft is transmitted to the remote pilot station in real time. Due to failures in the communication network, packets with video stream data may be lost or delayed and be delivered late. One of the ways to compensate for the drop in FPS in case of frame loss or delay at the remote pilot station is to predict intermediate frames of the video stream. This paper presents a scheme for predicting intermediate frames of a video stream and presents the results of experiments to establish a realistically possible increase in FPS in a neural network codec, in which various autoencoders such as VQ-f16 are used for video compression as well as the lossless latent feature space compression algorithm DEFLATE, while the dynamic multiscale voxel flow neural network DMVFN is used for prediction. A regression model was developed to predict the prediction time. The task of analyzing FPS for various configurations of the neural network decoder on the side of the external pilot is formalized.
About the Authors
A. A. Berezkin
Bonch-Bruevich State university of telecommunications (SPbSUT)
Russian Federation
Russian Federation
Alexander A. Berezkin - PhD, associate professor of the Program Engineering and Computer Science Department, Bonch-Bruevich State university of telecommunications SPbSUT, director of the Center of Perspective Projects and Developments.
193232, Saint-Petersburg, Bolshevikov Avenue, 22-1
A. A. Chenskiy
Bonch-Bruevich State university of telecommunications (SPbSUT)
Russian Federation
Russian Federation
Alexander A. Chenskiy - Master's student of the Program Engineering and Computer Science Department, engineer of Center of Perspective Projects and Developments, SPbSUT.
193232, Saint-Petersburg, Bolshevikov Avenue, 22-1
R. V. Kirichek
https://sut.ru
Bonch-Bruevich State university of telecommunications (SPbSUT)
Russian Federation
Bonch-Bruevich State university of telecommunications (SPbSUT)
Russian Federation
Ruslan V. Kirichek - Doctor of technical science, rector, professor of the Program Engineering and Computer Science Department, SPbSUT.
193232, Saint-Petersburg, Bolshevikov Avenue, 22-1
Phone. +7 812 3051 200
References
1. Chmelev B. S., Kalyuka V. I., Dmitrenko M. E. Obzor sistem upravlenija bespilotnyh letatel'nyh apparatov obshhego pol'zovanija [Overview of general purpose unmanned aerial vehicle control systems]. Tehnologii. Innovacii. Svjaz': Sbornik materialov nauchno-prakticheskoj konferencii, Sankt-Peterburg, 19 April, 2021. Sankt-Peterburg, Federal'noe gosudarstvennoe kazjonnoe voennoe uchrezhdenie vysshego obrazovanija "Voennaja akademija svjazi imeni marshala Sovetskogo Sojuza S. M. Budjonnogo" Ministerstva oborony Rossijskoj Federacii, 2022. pp. 279-286.
2. Strategija razvitija otrasli svjazi Rossijskoj Federacii na period do 2035 goda [The strategy for the development of the telecommunications industry of the Russian Federation for the period until 2035], available at: http://government.ru/news/50304/ (accessed: 21.05.2024).
3. Jurij Borisov vystupil na strategicheskoj sessii, posvjashhjonnoj serijnomu proizvodstvu sputnikov [Yura Borisov made a presentation at the strategic session on serial production of satellites], available at: https://www.roscosmos.ru/40096/ (accessed: 21.05.2024).
4. Chislo uchtjonnyh v Rossii dronov vyroslo na tret' [The number of drones registered in Russia increased by a third], available at: https://favt.gov.ru/novosti-novosti/?id=10676 (accessed: 21.05.2024).
5. Berezkin A. A., Parshin A. A., Lazarev A. A. Adaptivnyj kontrol' intensivnosti videopotoka pri peredache FPV-trafika bespilotnyh sistem [Adaptive control of video stream intensity in FPV traffic transmission of unmanned systems]. 79-ja nauchno-tehnicheskaja konferencija SPb NTO RES im. A.S. Popova, posvjashhennaja Dnju radio: sbornik dokladov regional'noj konferencii, Saint Petersburg, 22-26 April, 2024, pp. 162-165.
6. Berezkin A. A., Savelov D. Ju., Suhodoeva A. V., Tumanov I. A., Kirichek R. V. Issledovanie nejrosetevyh modelej predskazanija videopotoka pri upravlenii bespilotnymi sistemami ot pervogo lica [Research of neural network models for video stream prediction in first-person control in unmanned systems]. Trudy Nauchno-issledovatel'skogo instituta radio, 202, no. 3-4, pp. 40-56.
7. Goncharov V. G., Berezkin A. A. Analiz modelej predskazanija kadrov FPV-videopotoka v kanalah informacionnogo obmena bespilotnyh sistem [Analysis of FPV video stream frame prediction models in information exchange links of unmanned systems]. 79-ja nauchno-tehnicheskaja konferencija SPb NTO RES im. A.S. Popova, posvjashhennaja Dnju radio: sbornik dokladov regional'noj konferencii, Saint-Petersburg, 22-26 April, 2024, pp. 194-197.
8. Berezkin A. A., Vivchar' R. M., Kirichek R. V., Zaharov A. A. Metod dekompressii FPV-videopotoka ot bespilotnyh sistem na osnove latentnoj diffuzionnoj nejrosetevoj modeli [Method of decompression of FPV video stream from unmanned systems based on latent diffusion neural network model]. Electrosvjaz', 2024, №1, pp. 42-53.
9. Rombach R., Blattmann A., Lorenz D., Esser P., Ommer B. High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10684-10695.
10. Hu X., Huang Z., Huang A., Xu J., Zhou S. A dynamic multi-scale voxel flow network for video prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6121-6131.
11. Berezkin A. A., Vivchar' R. M., Slepnev A. V., Kirichek R. V., Zaharov A. A. Metod szhatija videopotoka pri upravlenii bespilotnymi sistemami v gibridnyh orbital'no-nazemnyh setjah svjazi [Video compression method for controlling unmanned systems in hybrid orbital-terrestrial communication networks]. Electrosvjaz', 2023, no. 10, pp. 48-56.
12. Bardasov S. A. Ekonometrika: uchebnoe posobie. 2 izdanie, pererabotannoe i dopolnennoe [Econometrics: textbook. 2nd edition, revised and expanded]. Tjumen', Izdatel'stvo Tjumenskogo gosudarstvennogo universiteta, 2010. 264 p.
13. Berezkin A. A., Phuc H. Do., Tran D. Le., Kirichek R. V. A comparative analysis of diffusion-based super-resolution techniques in a video stream compression system in FPV control of unmanned systems. Trudy Nauchno-issledovatel'skogo instituta radio, 2023, no. 3-4, pp. 26-39.
14. Kozlova A. I., Oblakov N. A., Berezkin A. A. Analiz modelej predskazanija kadrov FPV-videopotoka v kanalah informacionnogo obmena bespilotnyh sistem [Analysis of FPV video stream frame prediction models in information exchange channels of unmanned systems]. 79-ja nauchno-tehnicheskaja konferencija SPb NTO RES im. A.S. Popova, posvjashhennaja Dnju radio: sbornik dokladov regional'noj konferencii, Saint-Petersburg, 22-26 April, 2024, pp. 170-173.
Supplementary files
Review
For citations:
Berezkin A.A., Chenskiy A.A., Kirichek R.V. Research of Video Stream Intensity Limits in UAV FPV Control in Frame Prediction Mode. Part I: Models and Methods. The Herald of the Siberian State University of Telecommunications and Information Science. 2024;18(3):115-139. (In Russ.) https://doi.org/10.55648/1998-6920-2024-18-3-115-139
Views:
161
Email this article 