[Submitted on 9 Jun 2026 (v1), last revised 24 Jul 2026 (this version, v2)]

Authors:Zach Moczkodan (1), Hany Ragab (1) ((1) Royal Military College of Canada, Kingston, Canada)

View PDF HTML (experimental)

Abstract:Recent deep learning approaches for network intrusion detection increasingly incorporate temporal architectures such as recurrent networks and Transformers, often reporting near-perfect performance on CIC-IDS2017. However, many existing studies neither supply their temporal modules with genuine sequence inputs nor evaluate under realistic, leakage-free conditions, making it unclear whether reported gains arise from true sequence-modelling capability. In this work, we reformulate CIC-IDS2017 as a temporal intrusion-detection task by constructing ordered flow sequences from network conversations and benchmarking nine classical and deep learning architectures under a random split, two leakage-free splits, and a padding-scheme ablation. The central finding is that padding convention, not architecture, determines the Transformer's performance: on genuinely sequential (non-padded) windows the Transformer achieves the highest macro-F1 of any model in the experiment ($0.89$), yet under zero-pad+mask evaluation it drops markedly while LSTM, GRU, and 1D-CNN remain stable and a class-balanced Random Forest is the most robust model overall. The same repeat-last padding cue masks a $67\times$ increase in the Transformer's false-alarm rate that only leakage-free evaluation reveals. Evaluation methodology -- specifically padding convention and split protocol -- thus has a larger effect on reported performance than architectural choice. We advocate for leakage-free splits, explicit padding disclosure, and sequence-aware benchmarking as standard practice in future IDS research. Our code and implementation details are available here: this https URL.

Submission history

From: Hany Ragab [view email]
[v1] Tue, 9 Jun 2026 16:57:10 UTC (222 KB)
[v2] Fri, 24 Jul 2026 13:00:31 UTC (405 KB)