Trusted system-calls analysis methodology aimed at detection of compromised virtual machines using sequential mining

Nir Nissim, Yuval Lapidot, Aviad Cohen, Yuval Elovici

Knowledge-Based Systems 153, 147-175, 2018

Most organizations today employ cloud-computing environments and virtualization technology; Due to their prevalence and importance in providing services to the entire organization, virtual-servers are constantly targeted by cyber-attacks, and specifically by malware. Existing solutions, consisting of the widely-used antivirus (AV) software, fail to detect newly created and unknown-malware; moreover, by the time the AV is updated, the organization has already been attacked. In this paper, we present a during run-time analysis methodology for a trusted detection of unknown malware on virtual machines (VMs). We conducted trusted analysis of volatile memory dumps taken from a VM and focused on analyzing their system-calls using a sequential-mining-method. We leveraged the most informative system-calls by machine-learning algorithms for the efficient detection of malware in widely used VMs within …