Research of Video Stream Intensity Limits in UAV FPV Control in Frame Prediction Mode. Part II: adaptive system of video stream latency and intensity control

In recent years, unmanned aircraft have been receiving active use. In a number of applications, first-person control of unmanned aircraft is used. First-person control requires the transmission of a video stream from an unmanned aircraft to an external pilot station. However, during network transmission, delay and frame loss of the video stream occurs. This results in
lower intensity and higher latency of the video stream on the monitor of the external pilot station. In the previous part of the research, the method of increasing the intensity of video stream was presented, its mathematical models were built. In related works: the possibility of reducing the time of first-person control desynchronization was researched; the structure of unmanned systems first person control information exchange system was presented. The second part of the present research summarizes the results obtained in earlier works and presents an adaptive system of video stream latency and intensity control. The system consists from three blocks: neural codec mode controller, information exchange links control unit and video intensity control unit in two modes: intensity enhancement and delay reduction.

1. Berezkin A. A., Chenskiy A. A., Kirichek R.V. Reduction of FPV video stream latency with future video stream frame prediction [Umen’shenie zaderzhki peredachi FPV-videopotoka cherez sputnikovye kanaly sv’azi metodom predskazanija budushhih kadrov]. Elektrosvjaz’. 2025. No. 4 (In Russ.)

2. Berezkin A. A., Chenskiy A. A., Kirichek R. V. Research of video stream intensity limits in UAV FPV control in frame prediction mode. Part I: models and methods [Issledovanie granic intensivnosti videopotoka pri FPV-upravlenii BPLA v rezhime predskazanija kadrov. Chast’ I: modeli i metody]. Vestnik SibGUTI, 2024, No. 18(3), pp. 115-139.

3. Hu X., Huang Z., Huang A., Xu J., Zhou S. A dynamic multi-scale voxel flow network for video prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6121-6131.

4. Choquette J., Gandhi W. Nvidia A100 GPU: Performance & innovation for GPU computing. 2020 IEEE Hot Chips 32 Symposium (HCS), IEEE Computer Society, 2020, pp. 1-43.

5. Choquette J., Gandhi W., Giroux O., Stam N., Krashinsky R. Nvidia A100 tensor core GPU: Performance and innovation. IEEE Micro, 2021, No. 41(2), pp. 29-35.

6. Giambene G., Kota S., Pillai P. Satellite-5G integration: A network perspective. IEEE Network, 2018, No. 32(5), pp. 25-31.

7. Berezkin A. A., Parshin A. A., Lazarev A. A. Adaptivnyj kontrol’ intensivnosti videopotoka pri peredache FPV-trafika bespilotnyh sistem [Adaptive control of video stream intensity in FPV traffic transmission of unmanned systems]. 79-ja nauchno-tehnicheskaja konferencija SPb NTO RES im. A.S. Popova, posvjashhennaja Dnju radio: sbornik dokladov regional’noj konferencii, Saint Petersburg, 22-26 April, 2024, pp. 162-165.

8. Strategy for the development of the telecommunications industry of the Russian Federation for the period until 2035 [Strategija razvitija otrasli svjazi Rossijskoj Federacii na period do 2035 goda]. Government of Russia. URL: texttthttp://government.ru/news/50304/ (accessed: 02.01.2025).

9. Berezkin A. A., Chenskiy A. A., Kirichek R. V., Zaharov A. A. Research of neural codecs configurations for adaptive unmanned systems FPV-control video stream frame compression system. Part I. Methodics [Issledovanie konfiguracij nejrosetevyh kodekov dlja adaptivnoj sistemy szhatija kadrov FPV-videopotoka pri upravlenii bespilotnymi sistemami. Chast’ I. Metodika]. Elektrosvjaz’, Sankt-Peterburg, 2024, No. 9, pp. 42-51.

10. Berezkin A. A., Chenskiy A. A., Kirichek R. V., Zaharov A. A. Research of neural codecs configurations for adaptive unmanned systems FPV-control video stream frame compression system. Part II. Experiment [Issledovanie konfiguracij nejrosetevyh kodekov dlja adaptivnoj sistemy szhatija kadrov FPV-videopotoka pri upravlenii bespilotnymi sistemami. Chast’ II. Jeksperiment]. Elektrosvjaz’, Sankt-Peterburg, 2024, No. 10, pp. 59-69.

11. Zhou D., Wang W., Yan H., Lv W., Zhu Y., Feng J. MagicVideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022.

12. Saharia C., Ho J., Chan W., Salimans T., Fleet D. J., Norouzi M. Image super-resolution via iterative refinement. IEEE transactions on pattern analysis and machine intelligence, 2022, No. 45(4), pp. 4713-4726.

13. Liu Z., Lin Y., Cao Y., Hu H., Wei Y., Zhang Z., Lin S., Guo B. SWIN transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10012-10022.

14. Liang J., Cao J., Sun G., Zhang K., Van Gool L., Timofte R. SwinIR: Image restoration using swin transformer. Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 1833-1844.

15. Wang X., Yu K., Wu S., Gu J., Liu Y., Dong C., Qiao Y., Loy C. C. ESRGAN: Enhanced super-resolution generative adversarial networks. Proceedings of the European conference on computer vision (ECCV) workshops, 2018.

16. Kozlova A. I., Oblakov N. A., Berezkin A. A. Comparative analysis of superresolution methods in UAV FPV-control video stream compression system [Sravnitel’nyj analiz metodov sverhrazreshenija v sisteme szhatija videopotoka pri FPV upravlenii bespilotnymi sistemami] // Materialy 79-j nauchno-tehnicheskoj konferencii SPb NTO RJeS im. A. S. Popova, posvjashhennoj Dnju radio, Saint Petersburg, 22–26 April 2024, pp. 166–169.

17. Berezkin A. A., Vivchar’ R. M., Slepnev A. V., Kirichek R. V., Zaharov A. A. Video stream compression method for unmanned systems first person view controll in hybrid orbital-terrestrial networks [Metod szhatija videopotoka pri upravlenii bespilotnymi sistemami v gibridnyh orbital’no-nazemnyh setjah svjazi]. Jelektrosvjaz’, 2023, №. 10, pp. 48-56.

18. Berezkin A. A., Vivchar’ R. M., Kirichek R. V., Zaharov A. A. Method of FPV-video stream from unmanned systems decompression with diffusional neural model [Metod dekompressii FPV-videopotoka ot bespilotnyh sistem na osnove latentnoj diffuzionnoj nejrosetevoj modeli]. Elektrosvjaz’, 2024, №. 1, pp. 42-53.

19. Berezkin A. A., Chenskiy A. A., Kirichek R. V., Zaharov A. A. Masking of UAV FPV-control video stream. Part I: Models and methodics [Maskirovanie kodirovannogo FPV videopotoka pri upravlenii BPLA. Chast’ I. Modeli i metodika]. Elektrosvjaz’, 2024, No. 12-2, pp. 52-65.

20. Berezkin A. A., Chenskiy A. A., Kirichek R. V. Video stream frames artifacts suppression in unmanned systems FPV-control [Nivelirovanie artefaktov kadrov videopotoka pri FPV-upravlenii bespilotnymi sistemami]. Infkommunikacionnye tehnologii, 2024, № 22(85), pp. 7-17.

21. Berezkin A. A., Oblakov N. A., Chenskiy A. A., Kirichek R. V. Latent video stream correlational analysis in UAV first person view control [Korreljacionnyj analiz latentnogo videopotoka pri upravlenii BAS otpervogo lica]. Elektrosvjaz’, 2024, № 11, pp. 25-31.

22. Larzon L. A., Degermark M., Pink S. UDP lite for real time multimedia applications. Hewlett-Packard Laboratories, 1999.

23. Allman M., Falk A. On the effective evaluation of TCP. ACM SIGCOMM Computer Communication Review, 1999, №. 29(5), pp. 59-70.

24. Langley A., Riddoch A., Wilk A., Vicente A., Krasic C., Zhang D., Shi Z. The QUIC transport protocol: Design and internet-scale deployment. Proceedings of the conference of the ACM special interest group on

25. data communication, 2017, pp. 183-196.

26. Cui Y., Li T., Liu C., Wang X., Kühlewind M. Innovating transport with QUIC: Design approaches and research challenges. IEEE Internet Computing, 2017, №. 21(2), pp. 72-76.

27. Berezkin A. A., Matveev D. V., Savelov D. Ju. Video prediction model optimization with Tensorrt to decrease latency in UAV control [Optimizacija modeli predskazanija video s pomoshh’ju tensorrt dlja umen’shenija zaderzhek pri upravlenii BPLA]. Podgotovka professional’nyh kadrov v magistrature dlja cifrovoj jekonomiki (PKM-2023) : Vserossijskaja nauchno-tehnicheskaja i nauchno-metodicheskaja konferencija magistrantov i ih rukovoditelej; Sbornik luchshih dokladov: v 2 tomah, Saint Petersburg,

28. –07 december 2023, Saint Petersburg: Bonch-Bruevich State University of Telecommunications, 2023, pp. 328-334.

29. Hore A., Ziou D. Image quality metrics: PSNR vs. SSIM. 2010 20th international conference on pattern recognition, IEEE, 2010, pp. 2366-2369.

30. Sara U., Akter M., Uddin M. S. Image quality assessment through FSIM, SSIM, MSE and PSNR - a comparative study. Journal of Computer and Communications, 2019, No. 7(3), pp. 8-18.

31. Bardasov S. A. Econometrics: textbook. 2nd edition, revised and expanded [Jekonometrika: uchebnoe posobie. 2-e izdanie, pererabotannoe i dopolnennoe]. Tjumen’: Izdatel’stvo Tjumenskogo gosudarstvennogo universiteta, 2010, 264 P.

32. Chenskiy A. A., Berezkin A. A. Simple dynamic models for variational autoencoders inference time prediction [Prostye dinamicheskie modeli predskazanija vremeni raboty variacionnyh avtokodirovshhikov] // Materialy 80-j nauchno-tehnicheskoj konferencii SPb NTO RJeS im. A. S. Popova, posvjashhennoj Dnju radio, Saint Petersburg, 21–25 April 2025, pp.

33. Roerich. Github. URL: https://github.com/HSE-LAMBDA/roerich (accessed: 05.01.2025).

34. Ikatser I. D., Kozitsin V. O. Skoltech anomaly benchmark (SKAB). Kaggle. URL: https://www.kaggle.com/dsv/1693952 (accessed: 05.01.2025).

35. Shewhart W. A. Economic Control of Manufactured Product. New York: D.Van Norsrand, Co, 1931, 501 P.

36.