Protecting VNF services with smart online behavior anomaly detection method

https://doi.org/10.1016/j.future.2018.12.058Get rights and content

Highlights

  • Propose a robust multivariate HMM model to profile normal VNF behavior patterns.

  • Build a general and efficient anomaly detection framework in the NFV Infrastructure layer.

  • Train two types of VNF behavior models, the virtual router and virtual firewall are trained using real life normal network traffics.

Abstract

Network Function Virtualization (NFV) is an emerging technology that allows network operators to deploy their Virtualized Network Functions (VNFs) on low-cost commodity servers in the cloud data center. The VNFs, such as virtual routers, firewalls etc., that typically control and transmit critical network packages, require strong security guarantees. However, detecting malicious or malfunctioning VNFs are challenging, as the behaviors of VNFs are dynamic and complex due to the changing network traffics in the cloud. In this paper, we propose a smart and efficient Hidden Markov Model based anomaly detection system (named vGuard) to protect online VNF services in the cloud. A general multivariate HMM model is proposed to profile the normal VNF behavior patterns. Using the VNF behavior model trained with normal observation sequences, vGuard can effectively detect abnormal behaviors online. vGuard is a general framework that can train different types of VNF behavior models. We implement the vGuard prototype in the OpenStack platform. Two types of VNF models, virtual router and virtual firewall, are trained using real normal network traffics in our experiment evaluation. A collection of abnormal attack cases are tested on the VNFs that showed the effectiveness of vGuard in detecting VNF behavior anomalies.

Introduction

Network Function Virtualization (NFV) is an emerging technology that allows telecom and enterprise network operators to control their networking functions (physical, virtual and functional domains) using low-cost off-the-shelf hardware and open source software for management and orchestration [1]. NFV brings the inherent benefits of cloud computing, which aims at addressing the pain points encountered by the communication industry: the traditional networking infrastructures are expensive, proprietary, and inflexible.

NFV enables the Virtual Network Functions (VNFs), such as virtual routers, firewalls, load balancers, Intrusion Detection Systems (IDS), etc., run inside virtual machines on top of standard physical servers, switches and storage devices (known as cloud infrastructure), instead of using traditional proprietary physical appliances for each network function. The VNFs are usually placed in the key nodes of service chains, which control and transmit critical network packages. Therefore, the security guarantee of VNFs is a top priority.

Compared with the traditional physically isolated network appliances, the VNFs are more vulnerable to security threats due to the more complex virtualized computing platform and the shared nature of NFV deployment [2]. In order to securely apply NFV in the large-scale production environment, the network operators require the NFV infrastructure to provide security techniques to detect and prevent malicious or malfunctioning VNFs. How to effectively protect VNFs from security threats is one of the key challenges for the widely adoption of NFV in the industry.

Anomaly detection is a basic security mechanism to detect malicious activities in cyber–physical systems. The accurate detection makes it possible to timely prevent an attack or correct malfunctioning services. There are a range of anomaly detection techniques proposed in different application domains [3], which include classification based, clustering based, and other statistical methods. The most relevant techniques used in this paper is the anomaly detection for discrete sequences [4]. Although some sequential Hidden Markov Model (HMM) based anomaly detection methods [5], [6], [7], [8], [9] are proposed in previous researches, they mainly focused on a certain aspect of anomalies, such as network flows [5], system payload [8] or user behavior [9] etc., and are applied in the traditional host-based as well as other application domains that could not be directly applied in the more dynamic and flexible NFV environment. To the best of our knowledge, the general online VNF behavior anomaly detection in the practical NFV cloud environment is not well studied yet and needs further investigation.

Anomaly detection for VNF behaviors faces three major challenges: (1) Many types of VNFs, such as virtual routers, firewalls, and load balancers etc., have their specific behaviors that are different from each other. This requires a general anomaly detection method to detect anomalies for different VNFs on top of the NFV Infrastructure (NFVI) layer. (2) The normal behavior patterns of a VNF may change due to the network traffic variations or the heterogeneous interactions between VNFs and the underlying physical servers. Therefore, a robust statistical model is required to profile VNF normal behaviors. (3) VNFs are typically low latency and performance critical applications, the online anomaly detection should be efficient and cannot hurt the performance of original VNFs.

In this paper, we propose a smart and efficient Hidden Markov Model based anomaly detection system (named vGuard) to guard online VNF behaviors. As VNFs run inside virtual machines (VMs), the observable VNF behaviors in the NFVI layer are performance feature events related to compute side resource usage and network side traffic flows. These performance features are universal to different types of VNFs so that a general detection framework can be applied. To capture the dynamic changes of hidden states underlying the observable VNF behaviors, a multivariate Hidden Markov Model is derived to profile the pattern of normal VNF behaviors. vGuard leverages the performance monitoring capabilities of OpenStack platform to periodically extract the observable VNF behavior features. A total of 23 performance events that include both compute and network features are collected at a time to form one observation vector point. A long sequence of normal behavior observation vectors for each VNF is formed during the training period. To improve the training and detection efficiency, the EM clustering algorithm is first applied to the observation sequence to form a condensed two-dimensional sequence. In the online anomaly detection phase, vGuard continually extracts online VNF observation sequences within a sliding window. If the new observation sequence analyzed by the trained model exhibits sufficiently low probability, vGuard raises an anomaly alert to reveal a possible attack or malfunctioning behavior is detected.

In summary, the main contributions of this paper are described as follows:

(1) A general and efficient anomaly detection framework is proposed in the NFV Infrastructure layer. The anomaly detection framework can be deployed in distributed local nodes to train different types of VNF behavior models respectively and detect anomalies using the trained models.

(2) A robust multivariate Hidden Markov Model is proposed to quantify the normal VNF behavior patterns. A comprehensive set of performance features are clustered into a two-dimensional observation sequence. The observation sequence, which represents both compute resource usages and network traffic flows of a normal VNF, is used to train the behavior model.

(3) The proposed vGuard system is designed and implemented in the OpenStack cloud platform. Two types of VNF behavior models, the virtual router and virtual firewall, are trained using real life normal network traffic. These models are tested in the OpenStack platform to show their effectiveness of detecting abnormal behaviors.

The rest of this paper is organized as follows. Section 2 described the background and motivation. Sections 3 VNF behavior anomaly detection, 4 System implementation describe the vGuard system design and implementation. Section 5 presents performance evaluation. Section 6 describes the related work. And finally, Section 7 concludes our work.

Section snippets

Network function virtualization

The goal of vGuard is to help the NFV operators (serviceproviders) timely detect the malicious or malfunction VNFs. vGuard is a general VNF behavior anomaly detection system. Therefore, vGuard is designed and implemented in the NFV Infrastructure layer to monitor the VNF behaviors on top of NFVI.

Fig. 1 shows a brief overview of Network Function Virtualization Infrastructure (NFVI) based on the OpenStack cloud platform. OpenStack is a reference NFV platform to act as the Virtualized

vGuard system overview

The proposed vGuard system provides the security services of guarding NFV behaviors and detecting anomalies online. Fig. 2 shows the vGuard system integrated with the multi-layer NFV architecture. The physical layer provides the basic compute and network hardware resources to the VNFs. On top of the physical layer, virtualization techniques are deployed to form the network function virtualization layer. vGuard resides on the virtualization layer and continually monitors different types of VNFs.

System implementation

The NFV Infrastructure deployment is currently based on the OpenStack cloud platform. We design and implement the vGuard system that is integrated with the OpenStack Liberty version. Fig. 6 shows an overview of vGuard system architecture. The OpenStack platform typically has three types of server nodes: (i) the compute node for hosting VMs, (ii) the network node for virtualized network capabilities, and (iii) the controller node for overall resource and identity management. The VNFs running on

Performance evaluation

In this section, we evaluate the proposed vGuard system based on the OpenStack Liberty cloud platform. The three types of OpenStack server nodes have the same physical configurations, which are equipped with two Intel Xeon E5560 processors, 64 GB DDR3 RAM and 1 TB disk. Each server is installed with the Ubuntu 14.04.5 64-bit OS, the Linux kernel version is 4.7.0. The QEMU-KVM 1.2.0 is used as the hypervisor. In the experiment evaluation, we mainly test the performance of vGuard in the compute

Related work

There are many previous researches focusing on the Intrusion Detection Systems (IDSs). The detection methods of IDSs [25] can be mainly categorized as: (i) Signature-based detection, (ii) Anomaly-based detection, and (iii) Hybrid detection. From the deployment scenarios perspective [26], the IDSs can be generally classified as: (i) Host-based IDS, (ii) Network-based IDS, and (iii) Hypervisor-based IDS. Most traditional IDSs are designed and deployed in the classic host-based and network-based

Conclusion

In this paper, we proposed the VNF online behavior anomaly detection system in the NFV cloud environment. The hidden Markov model is applied to profile the normal behavior patterns of VNFs. The online performance feature monitor is implemented to periodically collect VNF performance features that forms the observation sequence used in the model. Two types of VNF behavior models are trained using real-life ISP network traffic. The online anomaly detection method is proposed based on the trained

Acknowledgments

This work was funded by Huawei HIRP, China and NSFC, China (Grant number 61802080).

Yuxia Cheng received the Ph.D. degree in computer science and technology from Zhejiang University, Hangzhou, China, in 2015. He is currently an associate Professor at the School of Computer Science and Technology, Hangzhou Dianzi University. He was an associate research fellow at School of Information Technology, Deakin University. His current research interests include multicore architecture, operating systems, virtualization and system security.

References (50)

  • AriuD. et al.

    HMMPayl: An intrusion detection system based on Hidden Markov Models

    Comput. Secur.

    (2011)
  • ShonT. et al.

    A hybrid machine learning approach to network anomaly detection

    Inform. Sci.

    (2007)
  • WangS.-S. et al.

    An integrated intrusion detection system for cluster-based wireless sensor networks

    Expert Syst. Appl.

    (2011)
  • ChungY.Y. et al.

    A hybrid network intrusion detection system using simplified swarm optimization (SSO)

    Appl. Soft Comput.

    (2012)
  • O.F. Report, Accelerating NFV delivery with OpenStack, Tech. rep....
  • MijumbiR. et al.

    Network function virtualization: State-of-the-art and research challenges

    IEEE Commun. Surv. Tutor.

    (2016)
  • ChandolaV. et al.

    Anomaly detection: A survey

    ACM Comput. Surv.

    (2009)
  • ChandolaV. et al.

    Anomaly detection for discrete sequences: A survey

    IEEE Trans. Knowl. Data Eng.

    (2012)
  • XieY. et al.

    A general collaborative framework for modeling and perceiving distributed network behavior

    IEEE/ACM Trans. Netw.

    (2016)
  • HuJ. et al.

    A simple and efficient hidden Markov model scheme for host-based anomaly intrusion detection

    IEEE Netw.

    (2009)
  • SrivastavaA. et al.

    Credit card fraud detection using Hidden Markov model

    IEEE Trans. Dependable Secure Comput.

    (2008)
  • XieY. et al.

    Modeling oscillation behavior of network traffic by nested Hidden Markov model with variable state-duration

    IEEE Trans. Parallel Distrib. Syst.

    (2013)
  • BaumL.E. et al.

    Statistical inference for probabilistic functions of finite state markov chains

    Ann. Math. Stat.

    (1966)
  • BaumL.E. et al.

    An inequality with applications to statistical estimation for probabilistic functions of markov processes and to a model for ecology

    Bull. Amer. Math. Soc.

    (1967)
  • RabinerL.R.

    A tutorial on hidden Markov models and selected applications in speech recognition

    Proc. IEEE

    (1989)
  • Cited by (0)

    Yuxia Cheng received the Ph.D. degree in computer science and technology from Zhejiang University, Hangzhou, China, in 2015. He is currently an associate Professor at the School of Computer Science and Technology, Hangzhou Dianzi University. He was an associate research fellow at School of Information Technology, Deakin University. His current research interests include multicore architecture, operating systems, virtualization and system security.

    Huijuan Yao received the Ph.D. degree in communication and information system from Beijing University of Posts and Telecommunications, she works as a senior researcher at the Shield Lab in Huawei technologies Co., LTD, China. Her research interests include AI security, NFV security, network security and mobile networks.

    Yu Wang received the Ph.D. degree in computer science from Deakin University, Victoria, Australia. He is currently with the School of Information Technology, Deakin University. His research interests include network traffic modeling and classification, social networks, mobile networks, and network security.

    Yang Xiang received his Ph.D. in Computer Science from Deakin University, Australia. He is currently a Full Professor at the School of Information Technology, Deakin University. He is the Director of the Network Security and Computing Lab (NSCLab) and the Associate Head of School (Industry Engagement). His research interests include network and system security, distributed systems, and networking. In particular, he is currently leading his team developing active defense systems against largescale distributed network attacks. He is the Chief Investigator of several projects in network and system security, funded by the Australian Research Council (ARC). He has published more than 180 research papers in many international journals and conferences, such as IEEE Transactions on Computers, IEEE Transactions on Parallel and Distributed Systems, IEEE Transactions on Information Security and Forensics, and IEEE Journal on Selected Areas in Communications.

    Hongpei Li was born in 1970, received his Ph.D. degree in Communication and Information System from Xidian University, Xi’an, China. He is now a principal scientist on cloud&AI security, at the Shield Lab, Huawei Technologies Co.,Ltd. His areas of research include cloud security, network security and industrial network security.

    View full text