We hear about cyber threats and attacks and breaches almost regularly. The looming question is why companies are not able to detect these threats in time to prevent the costly breaches ?
Well…this is not a simple task to handle if the right tools and platforms are not in play given the large amount of data that enterprises handle today.
Detecting cyber threats requires mind full mining through hundreds and thousands of threat indicators that gets triggered from various silos (such as ap plication and NW Firewalls, Policy Engines, Access Control Lists) and then researching through them in understanding the correlation between the threats to identify the attack signature before solutions can be engineered to mitigate them. In some cases attack signatures remain unknown due to lack of precedence, which makes it even harder to uncover and mitigate them.
In today’s world, simply put, it is not humanly possible to perform all of the above tasks in due time to keep companies secured and uninterrupted without using modern technologies that can do all of the above and protect against unknown attack signatures by looking at all things data.
Seceon’s OTM comprehensive cyber security platform empowered with intelligent cognitive capability was engineered to address these challenges.
The cognitive and intelligent threat detection function comes from the ML engine developed in-house by Seceon’s Data Science Team. We use unsupervised and semi-supervised learning principles to add cognitive ability to the ML models.
The Data Science team in Seceon is leap frogging the innovation in the cyber-security space with ML.
The key differentiator in our applied machine learning is how we feature engineer our models to look at threats by inspecting billions of bytes of data while providing actionable intelligence. T his is more than just identifying the right algorithms for the target use-case.
Mathematicians are not equipped to feature engineer and implement a solution that meets the performance and scale for real world use.
Most often what we require in a model is not available from the standard ML libraries. So we implement our algorithms directly from research papers while consulting with academicians and then feature engineer to build the models. In all our approaches we engineer to meet the scale and performance, tune ability and accuracy.
ML engine executes a suite of General Anomaly and Threat Specific cognitive models that constructs various dynamic blueprints of the target environment’s baseline behavior. These models are kept updated through continuous learning feedback that comes from live data while decaying very old data in parallel. This enables the platform to identify cyber breaches in real-time.
What type of techniques and approaches do you use in ML Engine?
We use a range of techniques and algorithms starting from simple statistical models, clustering and graph based classifiers, Bayesian probabilistic models to more complex Stochastic Optimization techniques build around complex surrogate functions. And in the heart of all of this, all our algorithms are engineered to maximize for scale, speed and accuracy. We use spark’s distributed computing framework, Basic Linear Algebra Subsystems (BLAS) libraries in a data-lake ecosystem.
Can you give us some insight on how your algorithms are designed to perform in real time?
Dimensionality reduction using PCP is widely used in ML applications, where the general idea is to reduce the large input vector space (M) to a lower dimensional subspace (L) such that the original input space M can be accurately recovered from L. This typically involves finding the lowest rank matrix L. The general affine rank minimization problem is NP-hard since it contains vector cardinality minimization as a special case. Most widely used approach to get to the minimum-rank solution is by solving a convex optimization problem, namely, the minimization of the nuclear norm over the given affine space. In Seceon’s ML engine we have taken a novel approach to compute the minimum rank solution using special mathematical algorithms. This enables us to model customer’s environment as L subspace in near real time so we can detect threats as they are happening. This is a significant differentiation when you compare it with competitive solutions that do not work on live data to reduce to L and as a result will not be able to identify subspace change in real time.
What is the difference between threat specific Algorithm vs. General Anomaly Detection Algorithms?
In general anomaly detection, we are using mathematical algorithms that use various types of entropy functions that model various traits to represent the target space as a whole.
In Threat Specific Algorithms we are building models that target specific types of attack such as DDOS, Insider Threat, Malware Beacon detection etc.
