Research Article

Enhanlcing the accuracy of intrusion detection systems by reducing the rates of false negatives through using XG-boost and optimization algorithm

¹ Department of Information and Communication Technologies, National School of Engineers of Tunis (ENIT), University of Tunis El-Manar, Tunis, 1008, Tunisia
² Department of Cyber Security, College of Computer Science and Mathematics, Tikrit University, Tikrit, 3400, Iraq
³ Department of Computer Science and Mathematics, National Institute of Applied Sciences and Technology, Tunis, 1080, Tunisia
⁴ Department of Petroleum Project Management, College of Industrial Management of Oil and Gas, Basrah University for Oil and Gas, AI-Basrah, 61004, Iraq

^* Corresponding authors (email: This email address is being protected from spambots. You need JavaScript enabled to view it. )

Received: 26 July 2024
Revised: 5 November 2024
Accepted: 31 December 2024

Abstract

Intrusion Detection Systems (IDSs) are critical for network security, detecting and mitigating malicious activities. A key challenge in IDS implementation is the high rate of false negatives, where attacks go undetected, posing significant security risks. This study proposes an enhanced IDS model that integrates XG-boost, a robust gradient boosting algorithm, with Cat Swarm Optimization (CSO) to reduce false negatives and improve detection accuracy. XG-boost’s scalability and performance make it ideal for managing complex network traffic data, while CSO optimizes XG-boost’s hyperparameters by mimicking natural cat behaviors, ensuring optimal model performance. The proposed approach was evaluated using a benchmark dataset, demonstrating a notable reduction in false negatives compared to traditional IDS methods. The upgraded IDS also improve detection accuracy across various types of cyberattacks while maintaining a low false positive rate, crucial for minimizing disruptions to regular network operations. The optimized XG-boost model achieved an accuracy of 98%, with precision of 97.8% and an F1-score of 97.7%, significantly outperforming the non-optimized model (accuracy: 84.1%, precision: 86.5%, F1-score: 84.1%). These results highlight the effectiveness of the proposed method in real-world IDS deployment, where both security and operational efficiency are critical.