It’s time for security teams to embrace security data lakes – TechCrunch



The typical company safety group spends $18 million yearly however is essentially ineffective at stopping breaches, IP theft and knowledge loss. Why? The fragmented method we’re presently utilizing within the safety operations heart (SOC) doesn’t work.

Right here’s a fast refresher on safety operations and the way we obtained the place we’re at the moment: A decade in the past, we protected our functions and web sites by monitoring occasion logs — digital information of each exercise that occurred in our cyber atmosphere, starting from logins to emails to configuration modifications. Logs had been audited, flags had been raised, suspicious actions had been investigated, and knowledge was saved for compliance functions.

The safety-driven knowledge saved in an information lake may be in its native format, structured or unstructured, and due to this fact dimensional, dynamic and heterogeneous, which provides knowledge lakes their distinction and benefit over knowledge warehouses.

As malicious actors and adversaries grew to become extra energetic, and their ways, methods and procedures (or TTP’s, in safety parlance) grew extra subtle, easy logging developed into an method referred to as “safety info and occasion administration” (SIEM), which entails utilizing software program to offer real-time evaluation of safety alerts generated by functions and community {hardware}. SIEM software program makes use of rule-driven correlation and analytics to show uncooked occasion knowledge into doubtlessly helpful intelligence.

Though it was no magic bullet (it’s difficult to implement and make every part work correctly), the flexibility to seek out the so-called “needle within the haystack” and establish assaults in progress was an enormous step ahead.

In the present day, SIEMs nonetheless exist, and the market is essentially led by Splunk and IBM QRadar. In fact, the know-how has superior considerably as a result of new use circumstances emerge continuously. Many corporations have lastly moved into cloud-native deployments and are leveraging machine studying and complicated behavioral analytics. Nonetheless, new enterprise SIEM deployments are fewer, prices are better, and — most significantly — the general wants of the CISO and the hard-working workforce within the SOC have modified.


New safety calls for are asking an excessive amount of of SIEM

First, knowledge has exploded and SIEM is just too narrowly centered. The mere assortment of safety occasions is now not adequate as a result of the aperture on this dataset is just too slender. Whereas there’s probably an enormous quantity of occasion knowledge to seize and course of out of your occasions, you might be lacking out on huge quantities of further info equivalent to OSINT (open-source intelligence info), consumable external-threat feeds, and helpful info equivalent to malware and IP repute databases, in addition to reviews from darkish internet exercise. There are countless sources of intelligence, far too many for the dated structure of a SIEM.

Moreover, knowledge exploded alongside prices. Information explosion + {hardware} + license prices = spiraling whole value of possession. With a lot infrastructure, each bodily and digital, the quantity of data being captured has exploded. Machine-generated knowledge has grown at 50x, whereas the typical safety funds grows 14% yr on yr.

The associated fee to retailer all of this info makes the SIEM cost-prohibitive. The typical value of a SIEM has skyrocketed to shut to $1 million yearly, which is just for license and {hardware} prices. The economics power groups within the SOC to seize and/or retain much less info in an try to hold prices in examine. This causes the effectiveness of the SIEM to turn out to be even additional decreased. I lately spoke with a SOC workforce who wished to question giant datasets trying to find proof of fraud, however doing so in Splunk was cost-prohibitive and a gradual, arduous course of, main the workforce to discover options.

The shortcomings of the SIEM method at the moment are harmful and terrifying. A latest survey by the Ponemon Institute surveyed nearly 600 IT safety leaders and located that, regardless of spending a median of $18.four million yearly and utilizing a median of 47 merchandise, a whopping 53% of IT safety leaders “didn’t know if their merchandise had been even working.” It’s clearly time for change.


Supply hyperlink