DeepGray: A Novel Approach to Malware Classification Using Grayscale Images with Deep Learning




Polsani, Harshitha
Jiang, Haodi

Journal Title

Journal ISSN

Volume Title


Institute for Homeland Security


In the ever-evolving landscape of cybersecurity, the threat posed by malware continues to loom large, necessitating innovative and robust approaches for its effective detection and classification. In this paper, we introduce a novel method, DeepGray, for multi-class malware classification utilizing grayscale images and the power of deep learning. Our dataset combines the malware sample from the BODMAS dataset and the benign sample from the DikeDataset. Our approach involves transforming executable files into a format suitable for deep learning by converting them into grayscale images while retaining the essentialdata characteristics. During the data preprocessing step, applied Principal Component Analysis (PCA) was applied to distill the most significant features. To achieve state-of-the-art results in multi-class malware classification, we harnessed the power of deep learning and transfer learning, employing well-established neural network architectures such as a customized Convolutional Neural (CNN), VGG16, EfficientNet, and Vision Transformers (ViT). The models were meticulously trained and rigorously evaluated using a 5-fold cross-validation methodology. Notably, our approach yielded remarkable results, with ViT achieved an impressive accuracy of 0.95. This research underscores the potential of grayscale image analysis and deep learning within the domain of multi-class malware classification. The insights derived from this study contribute significantly to the field of cybersecurity and pave the way for further advancements in the realm of malware detection and classification.



Malware Images, Transfer Learning, Convolutional Neural Networks


Polsani, H., Jiang, H. (2023) DeepGray: A Novel Approach to Malware Classification Using Grayscale Images with Deep Learning. (Report No. IHS/CR-2023-1014). The Sam Houston State University Institute for Homeland Security.