Contextual regularization of the feature space of weakly structured data for analyzing the risk topology of complex technical systems

System Analysis and Control
Authors:
Abstract:

The paper addresses the problem of eliminating sparsity and «false orthogonality» in short, weakly structured technical messages that hinder systematic analysis and modeling of the risk topology of complex technical systems. A method of contextual regularization of the feature space is proposed, which treats the enrichment of vector representations as a controlled diffusion process on a graph of joint occurrence of lemmas. The context topology is specified by a weighted adjacency matrix based on positive pointwise mutual information, and the recursive diffuser performs iterative feature propagation with depth attenuation and adaptive IDF gating, which suppresses noisy connections and amplifies diagnostically significant terms. The regularization parameter tuning is formalized as a task of maximizing the target quality functional, combining metrics of structural separability and semantic completeness with a threshold penalty for separability degradation. A priori, the limited nature of the diffusion process is demonstrated, and the elimination of orthogonality of terminologically heterogeneous descriptions in the presence of a contextual «bridge» in the graph is proven. Experimental testing on the NRC operational message corpus demonstrates a significant increase in the semantic coherence of topics while maintaining the geometric separability of clusters. The resulting regularized space improves the interpretability of the thematic structure of incidents and creates a basis for the subsequent self-organization of the risk event taxonomy and the construction of verifiable decision support contours.

Funding:

The research was financially supported by the Ministry of Science and Higher Education of the Russian Federation within the framework of the state assignment “Development of methodology for the formation of a tool base for analysis and modeling of spatial socio-economic development of systems in the conditions of digitalization with reliance on internal reserves” (FSEG-2023-0008).

  • References

    1. Rodionov D.G., Konnikov E.A., Mugutdinov R.M. Sistemnyi analiz konkurentosposobnosti tsifrovogo predpriiatiia v ramkakh informatsionnoi sredy [System analysis of the competitiveness of a digital enterprise within the information environment]. Economic Sciences, 2020, Vol. 193, No. 12, Pp. 394–401. DOI: 10.14451/1.193.394

    2. Makarova E.A. Processing of semi-structured text data for use in data analysis models. Information and mathematical technologies in science and management, 2023, Vol. 29, No. 1, Pp. 178–189. DOI: 10.25729/ESI.2023.29.1.015

    3. Salganik M.J. Bit by Bit: Social Research in the Digital Age. Princeton: Princeton University Press, 2017.

    4. Postiglione A., Monteleone M. Predictive maintenance with linguistic text mining. Mathematics, 2024, Vol. 12, No. 7, Art. no. 1089. DOI: 10.3390/math12071089

    5. Boyd R.L., Schwartz H.A. Natural language analysis and the psychology of verbal behavior: The past, present, and future states of the field. Journal of Language and Social Psychology, 2021, Vol. 40, No. 1, Pp. 21–41. DOI: 10.1177/0261927X20967028

    6. Wang Y., Chung S.-H. Artificial intelligence in safety-critical systems: a systematic review. Industrial Management & Data Systems, 2022, Vol. 122, No. 2, Pp. 442–470. DOI: 10.1108/IMDS-07-2021-0419

    7. Ignatow G., Mihalcea R.F. Text Mining: A Guidebook for the Social Sciences. Los Angeles: SAGE Publications, 2017.

    8. Konnikov E.A., Kryzhko D.A. Two-stage semantic clustering of embeddings as an alternative to LDA for infometric analysis of industry news. Software Systems and Computational Methods, 2025, Vol. 3, Pp. 10–19. DOI: 10.7256/2454-0714.2025.3.75348

    9. Macanovic A., Przepiorka W. A systematic evaluation of text mining methods for short texts: Mapping individuals’ internal states from online posts. Behavior Research Methods, 2024, Vol. 56, Pp. 2782–2803. DOI: 10.3758/s13428-024-02381-9

    10. Sazonov G.V., Lukyanov K.S., Boyarsky S.K., Makarov I.A. Is AI interpretability safe: the relationship between interpretability and security of machine learning models. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS), 2024, Vol. 36, No. 5, Pp. 127–142. DOI: 10.15514/ISPRAS-2024-36(5)-9

    11. Rodionov D.G., Karpenko P.A., Konnikov E.A. Metodika kvantifikatsii sostoianiia trudovykh resursov v kontekste upravleniia razvitiem regional'noi sotsial'no-ekonomicheskoi sistemoi [Methodology for quantifying the state of labor resources in the context of managing the development of the regional socio-economic system]. Economic Sciences, 2021, Vol. 197, No. 4, Pp. 171–179. DOI: 10.14451/1.197.171

    12. Probierz B., Hrabia A., Kozak J. A new method for graph-based representation of text in natural language processing. Electronics, 2023, Vol. 12, No. 13, Art. no. 2846. DOI: 10.3390/electronics12132846

    13. Krasnov F.V., Baskakova E.N., Smaznevich I.S. Assessment of the applied quality of topic models for clustering problems. Tomsk State University Journal of Control and Computer Science, 2021, No. 56, Pp. 100–111. DOI: 10.17223/19988605/56/11

    14. Irkhin I.A., Bulatov V.G., Vorontsov K.V. Additive regularization of topic models with fast text vectorization. Computer Research and Modeling, 2020, Vol. 12, No. 6, Pp. 1515–1528. DOI: 10.20537/2076-7633-2020-12-6-1515-1528

    15. Li P., Fu X., Chen J., Hu J. CoGraphNet for enhanced text classification using word-sentence heterogeneous graph representations and improved interpretability. Scientific Reports, 2025, Vol. 15, Art. no. 356. DOI: 10.1038/s41598-024-83535-9

    16. Hsu M.-F., Chang C., Zeng J.-H. Automated text mining process for corporate risk analysis and management. Risk Management, 2022, Vol. 24, Pp. 386–419. DOI: 10.1057/s41283-022-00099-6

    17. Gelastopoulos G., Keramydas C. A systematic review of text mining analytics for supply chain risk management using online data. Supply Chain Analytics, 2025, Vol. 12, Art. no. 100167. DOI: 10.1016/j.sca.2025.100167

    18. Troxler A., Schelldorfer J. Actuarial applications of natural language processing using transformers: Case studies for using text features in an actuarial context. British Actuarial Journal, 2024, Vol. 29, Art. no. 4. DOI: 10.1017/S1357321724000023

    19. Murshed B.A.H., Mallappa S., Abawajy J., Saif M.A.N., Al-ariki H.D.E., Abdulwahab H.M. Short text topic modeling approaches in the context of big data: taxonomy, survey, and analysis. Artificial Intelligence Review, 2023, Vol. 56, Pp. 5133–5260. DOI: 10.1007/s10462-022-10254-w

    20. Vashchenko V.A. Topic modeling for short texts: comparative analysis of algorithms. Sociology: Methodology, Methods, Mathematical Modeling (Sociology: 4M), 2024, Vol. 56, Pp. 69–112. DOI: 10.19181/4m.2023.32.1.2

    21. Mozaidze E.S. Topic modeling in the stream of short messages in Russian. Russian Technological Journal, 2025, Vol. 13, No. 1, Pp. 38–48. DOI: 10.32362/2500-316X-2025-13-1-38-48

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Previous articleNext article