A methodology for evaluating the effectiveness of an automated phishing detection pipeline based on FastText within the framework of a specialized AutoML library for analyzing barriers to the introduction of artificial intelligence into modern intrusion detection systems

The article presents an approach to phishing attack detection based on the construction
and automatic optimization of machine learning pipelines using a specialized library (PhishAutoML). The urgency of the problem is due to the evolution of phishing attacks that use social
engineering methods and lexical tricks, which makes traditional static protection methods ineffective. The theoretical foundations of text vectorization using the FastText model and its application within an AutoML approach based on Bayesian optimization, which allows for the automatic selection of hyperparameters for the entire pipeline, are described. The proposed
PhishAutoML concept is used to build models capable of detecting phishing based on semantic
analysis and flexibly configuring the trade-off between quality and performance. The results of
computational experiments are presented: final metrics of quality and performance, as well as a
comparative analysis with classical (TF-IDF) and modern (DistilBERT) approaches. The conclusions confirm the effectiveness of the proposed solution (achieving a phishing detection recall of 95%, which is several times higher than alternative methods) and outline directions for
its further development. However, integrating AI into traditional intrusion detection and prevention systems poses significant risks and challenges. This article explores the key technical, organizational, and ethical barriers that hinder the widespread adoption of AI-powered solutions and
suggests potential solutions to overcome them.

1. Petrenko S.A. Nacional'naya sistema rannego preduprezhdeniya o komp'yuternom napadenii: nauchnaya monografiya / Petrenko S.A., Stupin D.D. / pod redakciej S.F. Boeva. – 2-e izd. – Universi-tet Innopolis. – Innopolis: «Izdatel'skij Dom «Afina», 2018. -448 s.

2. Lavrova D.S. Metodologicheskoe i matematicheskoe obespechenie dlya SIEM-sistem v internete veshchej: dissertaciya na soiskanie uchenoj stepeni kandidata tekhnicheskih nauk: 05.13.19 / Lavrova Dar'ya Sergeevna; [Mesto zashchity: Peterb. gos. un-t putej soobshch.]. - Sankt-Peterburg, 2016. - 179 s.: il.

3. Uorr Keti. Nadezhnost' nejronnyh setej: ukreplyaem ustojchivost' II k obmanu. — SPb.: Piter, 2021. — 272 s.: il. — (Seriya «Bestsellery O’Reilly»)

4. Kornienko A.A. Modelirovanie komp'yuternyh atak na raspredelennuyu informa-cionnuyu sistemu / Kornienko A.A., Nikitin A.B., Diasamidze S.V., Kuz'menkova E.Yu. // Izvestiya Peterburgskogo uni-versiteta putej soobshcheniya. 2018. T. 15. № 4. S. 613-628.

5. Petrenko, S. A. O razrabotke bezopasnogo chastnogo oblaka / S. A. Petrenko // Mezhdunarodnyj forum KAZAN DIGITAL WEEK - 2020 : Sbornik materialov mezhduna-rodnogo foruma, Kazan', 21–24 sentyabrya 2020 goda / Akademiya nauk Respubliki Tatarstan. Tom Chast' 2. – Kazan': Nauchnyj centr bezopasnosti zhiznedeyatel'nosti, 2020. – S. 63-70.

6. Lavrova D.S. Modelirovanie setevoj infrastruktury slozhnyh ob"ektov dlya re-sheniya zadachi pro-tivodejstviya kiberatakam / Lavrova D.S., Zegzhda D.P., Zajceva E.A // Vo-prosy kiberbezopasnosti. 2019. №2(30) S.13-20

7. Zegzhda D. P. Upravlenie dinamicheskoj infrastrukturoj slozhnyh sistem v uslo-viyah celenaprav-lennyh kiberatak / Zegzhda D. P., Lavrova D. S., Pavlenko E. Yu. // Izvestiya RAN. Teoriya i sistemy upravleniya, 2020, № 3, s. 50–63

8. Kotenko I.V. Model' sistemy upravleniya informaciej i sobytiyami bezopasno-sti / Kotenko I.V., Parashchuk I.B. // Vestnik Astrahanskogo gosudarstvennogo tekhnicheskogo universiteta. Seriya: Up-ravlenie, vychislitel'naya tekhnika i informatika. 2020. № 2. S. 84-94.

9. Kornienko A.A. Metodika detalizirovannogo ocenivaniya znachimosti sistem orga-nizacii dvizheni-ya poezdov kak ob"ektov kriticheskoj informacionnoj infrastruktury / Kornienko A.A., Kornienko S.V., Karpov D.Yu. // Byulleten' rezul'tatov nauchnyh issledo-vanij. 2020. № 2. S. 5-19.

10. Danilin G.V. Mul'tiservisnye seti: metody povysheniya zashchishchennosti dannyh v usloviyah setevyh atak / Danilin G.V., Sokolov S.S., Nyrkov A.P., Knysh T.P. // XXI vek: itogi proshlogo i prob-lemy nastoyashchego plyus. 2020. T. 9. № 2 (50). S. 158-163.

11. Saenko I.B. Primenenie iskusstvennyh nejronnyh setej dlya vyyavleniya ano-mal'nogo povedeni-ya pol'zovatelej centrov obrabotki dannyh / Saenko I.B., Kotenko I.V., Al'-Barri M.H. // Voprosy kiberbezopasnosti. 2022. № 2 (48). S. 87-97.

12. Zelenin D.D. Razrabotka modeli upravleniya riskami dlya korporativnoj seti, ata-kuemoj vredonosnym programmnym obespecheniem tipa setevoj cherv' // Vestnik Voronezh-skogo instituta vysokih tekhnologij. 2020. № 2 (33). S. 137-142.

13. Semykina N.A. Matematicheskaya model' zashchity v informacionnoj sisteme ierar-hicheskoj struktury // Yuzhno-Sibirskij nauchnyj vestnik. 2021. № 4 (38). S. 75-80.

14. Gryzunov V. V. Adaptivnoe upravlenie dostupnost'yu resursov geoinformacion-noj sistemy kriticheskogo primeneniya v usloviyah destruktivnyh vozdejstvij : disserta-ciya ... doktora tekhnich-eskih nauk : 2.3.6. / Gryzunov Vitalij Vladimirovich; [Mesto zashchity: FGBOU VO «Peterburgskij gosudarstvennyj universitet putej soobshcheniya Imperatora Aleksandra I» ; Dissovet 44.2.004.02 (D 218.008.06)]. - Sankt-Peterburg, 2022. - 395 s. : il.

15. Lipatnikov V.A. Raspoznavanie vtorzhenij narushitelya pri upravlenii kiber-bezopasnost'yu infra-struktury integrirovannoj organizacii na osnove nejro-nechetkih setej i kognitivnogo modelirovaniya / Lipatnikov V.A., Tihonov V.A., // \"Aktual'nye pro-blemy infotelekommunikacij v nauke i obra-zovanii\" VIII Mezhdunarodnoj nauchno-tekhnicheskoj i nauchno-metodicheskoj konferencii sb. nauch. st. SPb. : SPbGUT, 2018.

16. Aho A., Kompilyatory. / Aho A., Seti R., Ul'man D. // Principy, tekhnologii, in-strumenty. — M. i dr.: Vil'yams, 2001. — 1184 c.

17. Vitkova, L. A. Model' vredonosnoj informacii i ee rasprostranitelya v soci-al'nyh setyah / L. A. Vitkova, D. V. Saharov, D. R. Goluzina // Zashchita informacii. Insajd. – 2020. – № 3(93). – S. 66-72.

18. Krasov A.V. Obespechenie bezopasnosti peredachi multicast-trafika v IP-setyah / A. V. Krasov, D. V. Saharov, I. A. Ushakov, E. P. Losin // Zashchita informacii. Insajd. – 2017. – № 3(75). – S. 34-42.

19. Saharov D.V. Ispol'zovanie matematicheskih metodov prognozirovaniya dlya ocen-ki nagruzki na vychislitel'nuyu moshchnost' IoT-seti / D. V. Saharov, A. M. Gel'fand, A. A. Kazancev, I. E. Pestov // Nauchno-analiticheskij zhurnal \"Vestnik Sankt-Peterburgskogo universiteta Gosudarstvennoj pro-tivopozharnoj sluzhby MChS Rossii\". – 2020. – № 2. – S. 86-94.

20. Abramova T. V. Obnaruzhenie anomalij i nejtralizaciya ugroz v raspredelennyh avtomatiziro-vannyh sistemah upravleniya na osnove monitoringa setevyh informacion-nyh potokov : dissertaciya ... kandidata tekhnicheskih nauk : 2.3.6. / Abramova Taisiya Vyache-slavovna; [Mesto zashchity: Ufimskij universitet nauki i tekhnologij ; Dissovet 24.2.479.07 (24.2.479.07)]. - Orenburg, 2024. - 235 s. : il.

21. Sheluhin O. I. Obnaruzhenie anomal'nyh vybrosov telekommunikacionnogo tra-fika metodami diskretnogo vejvlet-analiza / Sheluhin O. I., Garmashev A. V.//Elektromagnitnye volny i elektronnye sistemy. – 2012. – T. 17. – №. 2. – S. 15-26.

22. Sheluhin O.I. Intellektual'nye tekhnologii informacionnoj bezopasnosti. Uchebnoe posobie dlya vuzov / O.I. Sheluhin, D.P. ZEgzhda, D.I. Rakovskij; Pod. Red. Doktora tekhn. Nauk, professora O.I. Sheluhina. – M.: Goryachaya liniya – Telekom, 2023. – 384 s.: il.

23. Zegzhda D.P. Kiberbezopasnost' cifrovoj industrii. Teoriya i praktika funkci-onal'noj ustojchivosti k kiberatakam / D.P. Zegzhda, E.B. Aleksandrova, M.O. Kalinin, A.S. Markov, I.Yu. Zhukov, D.V. Ivanov, A.S. Konoplev, D.S. Lavrova, D.A. Moskvin, E.Yu. Pavlenko, M.A. Poltavceva, N.N. Shenec, A.D. Dahnovich, V.M. Krundyshev. // Pod redakci-ej professora RAN, doktora tekhnicheskih nauk D.P. zegzhdy. – M.: Goryachaya liniya – Telekom, 2022. – 560 s.: il.