Responsible partners:
OLISTIC is an innovative Enterprise Risk Management Platform offered by UBITECH that enables the management of risks across all operational domains of an organization. It builds on top of an asset representation model and uses graphs to define all applicable asset interconnections, which it then uses to calculate the risk, while it includes social intelligence and network intrusion detection and protection functionalities. In the scope of pilot #3, these compute intensive capabilities are transformed to use High-Performance Data Analytics (HPDA) in order to allow the efficient scaling of OLISTIC to large scale deployments. Giannis Ledakis from UBITECH explains what challenges his team and he face and how REGALE helps to overcome them.
Can you please summarize what pilot #3 is about?
OLISTIC uses assets and nodes representation and can be used to calculate risk per asset in a cumulative way. In addition OLISTIC includes Security Operation Center (SOC) functionalities, based on an open intelligence module that gathers security related information from different online sources (social media, RSS feeds, blogs, etc.), a honeypot-based, Threat Intelligent Network that is used for detecting Advanced Persistent Threats (a.k.a. APTs) and security agents deployed across the network. Advanced analytics that target APTs cannot rely on collected logs but must be performed using low-level traffic analysis; an analytics intensive process that can be very demanding when the magnitude of the input data scales. Therefore, HPC in combination with Artificial Intelligence (AI) and Machine Learning (ML) techniques can be used to detect strange systems behaviour, very early cyber-attack patterns, or potential misuse of systems.
What is the research field?
High-Performance Data Analytics (HPDA), meaning the use of HPC to analyze large data sets for patterns and insights.
What are the main challenges in your pilot?
There are three main challenges raised by the advanced analytics that target APTs; a) the order of magnitude of the input data; b) the time constraints regarding the output of an analysis and c) the streaming mode of the network traffic (i.e. data are not permanently persisted). The order of magnitude is undoubtedly increasing. In the past years, administrators were monitoring only a small number of network devices or less than a thousand computers. The network bandwidth was maybe just less or 100 Mbps (Megabits per second). Currently, administrators have to deal with higher speed wired networks (more than 1Gbps (Gigabits per second)) and various networks such as ATM (Asynchronous Transfer Mode) networks and wireless networks. As it is inferred, the operation of only one 24-port gigabit router may generate, under full utilization, approximately 50TB of data on a daily basis. As the data generated is bound to the actual traffic, it is important to achieve scaling and the optimal allocation and usage of HPC resources. Moreover, the network is analysed at different levels i.e. at packet level and at flow level, while in some cases analysis is time-critical, such as when performing flow-classification.
What do you expect from REGALE?
Using HPC allows increasing the scalability of our product and providing a complete package that includes risk assessment with integrated SOC functionalities to large organizations, while there is an additional benefit of allowing us to target organizations that own HPC infrastructure and would prefer exploiting such resources to efficiently deploy OLISTIC.