Windows Malware Detection Techniques Using Static and Behavioural-based Features

Abstract

The advancement in Internet-based communication technology has enabled malware to achieve its intent without the user’s consent. It penetrates or harms a computer system’s integrity, availability, and confidentiality. Forbye, a modern malware, is equipped with obfuscation techniques that maximize its capability to defeat antimalware detection systems and evade detection. The conventional anti-malware detection techniques exhibit inherent delayed effectiveness due to their signature-based detection and are inadequate to ascertain advanced malware. Therefore, there is need for a proficient malware detection technique, which can precisely identify it. The traditional Windows malware detection techniques can analyze malware without executing them. These techniques discern the malware by analyzing the static features of the Portable Executable (PE) files. However, they are incompetent against the emerging advanced malware attacks. To address this, behavioural-based malware detection technique emerges as an essential complement to defend against such sophisticated malware. The behavioural-based detection technique monitors and captures the activities of the malware during its runtime. It executes the input file (PE) in an isolated environment and records its behaviours during execution. However, in real-life scenario, it is tedious to examine all the recorded features. Hence, identifying significant features from the original features set is the primary challenging task in this technique. Several issues remain open in the development of an intricate malware detection system that can resist the attacks caused by the malware. Many examinations illustrate that the current malware detection systems are easily compromised by sophisticated malware. There are various solutions proposed in literature to uncover malware. However, each detection approach has its own limitation(s). The present research work aims to propose a classic approach to detect and classify Windows malware by extracting static features or behavioural features or a combination of both (hybrid features) of the PE files. In this regard, initially, the Malware Detection System (MDS) was designed based on the information extracted related to Portable Executable Optional Header Fields (PEOHF) as static features. In addition, to identify the malicious activities of the malware, behaviour analysis of the PE files was also performed by considering Application Programming Interface (API) or API with their corresponding category (CAT-API) or System calls invoked by the input PE file during execution. Concurrently, for precise classification operation, preserving the informative features is highly necessary to detect and distinguish the unknown PE files as malwareor benign. With this in view, the performance of the Feature Selection Techniques (FSTs) in recommending the best features is crucial for classifiers in discriminating between benign and malware PEs was evaluated. Subsequently, a malware detection technique based on visualization images was proposed where the images were generated using behavioural features suggested by the FST. Moreover, the effectiveness of the hybrid features in the detection of malware was examined based on the significant features recommended by the FSTs. Several sets of experiments were carried out to evaluate and demonstrate the potency of the proposed approaches. The efficiency of all the proposed approaches was assessed using real-world malware samples with 10-fold cross-validation tests. Different evaluation metrics such as True Positive Rate (TPR), False Positive Rate (FPR), Precision, Recall, F-Measure, and Accuracy were used to evaluate the proposed approaches. Based on the obtained experimental results, it was observed that the proposed approaches are effective in the detection and classification of the Windows malware.