The yin and yang of SIEM

After more than 5 years working in SIEM projects, I've decided to move into the GRC space. In this last post, I'd like to summarized the most important lesson I've learned during my SIEM journey.

During this time I've been able to see from a first row seat the evolution from purely compliance-driven projects to more security focused initiatives. Despite compliance/regulations/internal revisions still are the most common driver here in Norway, more and more organizations understand that log management is necessary as underlying technology to be able to dig into all the information is generated in the different systems.

Many organizations have taken a step forward in their strategy and start to see at SIEM as the tool to support and centralize the company's security monitoring efforts.

There are 3 critical factors I've consistently seen affecting the outcome of SIEM initiatives:

1. Understand the components. SIEM is very different from other security technologies where the product is the key. Here three components needs to collaborate together:

• PEOPLE. The first component to understand is who is going to use the solution. What are their needs, what this technology can do for helping them?
• PROCESSES.  The second factor is define how this technology is going to be used. How is going to be handle a high level incident? And a monthly report?
• TECHNOLOGY. The last component is the technology used. It's critical that it enable the user to let their imagination run wild instead of been the limitation of what it can be done. And this lead directly to the next factor.

2. Top down initiatives works best. The most successful projects I've seen have taken an approach where they started defining the use cases instead of just defining the project in terms of integrate thousand of systems without a clear idea why they are needed or what they will be used for. Down-to-top approach is complex and has a bigger cost than top-to-down which produces a much faster return of investment.

   Top-down approach can be seen a sequential process where you start from the logical use case definition. It's as "high level" scenario where involved critical assets are identified. Over these critical assets a set of controls are defined. These are the control that need to be monitored. Then an incident response procedure is defined for resolving the incident and mitigate the associate risk if possible. This approach makes also easy to identify the last critical factor.

3. Logs contain enough information. It seems obvious, but sometime the information available is just not enough. Most of the times, the controls we defined have their specific set of information that can be provided. Be sure that the information you can get is enough to solve your use case or at least identify it as soon as possible to find an alternate workaround.

With these three pieces in place, your chances of having a successful SIEM project are maximal.

But remember that SIEM projects should be in continuous evolution. Not only to unsure that the use cases previously defined continue being relevant, but also because new use cases will appear to respond to the changing threat landscape.

Good luck!

Thanks for been following this blog. From now on I will continue writing on mnemonic's web site blog. 

See you soon,

/Alonso

A Darwinian Theory of Logs

Hi again,

In September I had the opportunity to participate in the ISF (Norwegian Information Security Forum) autumn conference with my talk "A Darwinian Theory of Logs". It was a great conference, well organized and with several interesting talks and networking possibilities.

Obviously it won't be the same to just see the slides than be there in person. The original slides had very little text and they where use only as a supporting mechanism for transmit the content of the talk. But anyway I'd like to share with you the presentation here in our blog.

Click here to download the presentation. 

Comments are welcome :)

Raw data is good, Normalized is better, Categorized is awesome!

Continuing my last post on why using normalized data is better than just using raw data and how it accelerates the analisys process resulting in a faster response and therefor money saving, I'd like to focus now in the data mining aspect.

Remember the scenario: you are the IT responsible for your company's custom developed transaction application and your boss ask you to send him a report with all the activity related to the account number 1234567890.

Of course you can give all the raw information to your boss, but I'd not sure he is gonna like the idea of receiving a 20 pages report with all the entries where that account has been involved...

Having the data in raw format is good, we need it, but data mining is very difficult on it.

I'd prefer to give him some data more easy to handle, maybe a excel file where the information is easy to visualize, filter, group etc....Maybe create some graphs...

Raw data is good, Normalized is better!

This week my company arranged a seminar on log management and I had the opportunity to make a demo of one of our products.

My goal was to show why using normalized data is better than just using raw data and how this accelerate the analisys process resulting in a faster response and therefor money saving.

When I talk of normalized data, I mean that the information contained in a event is split in different pairs of "field:value". Saying it in a different way, we understand the content of that event.

Imagine an scenario where you are the IT responsible for your company custom developed transaction application. This application manage all the economical transaction between the company's different locations and the central server. You, aware as you are of the importance of having the logs properly secured, have the logs of this application sent into your company log management system.

As you don't have any specific usage for this logs beyond possible future troubleshooting and there isn't any regulation requirements which specify anything else, you decided that having the logs directly in raw format is enough. When I say "raw" format, I mean storing the logs just how they are created in the application, with other words, without understanding its content. And this may look as a complete valid statement for some cases.

But imagine...

Automated open source intelligence utilities

In my last post I talked about how you can use open source intelligence information to prioritize your alerts. And I think it could be interesting to make a short comparison between the two utilities I mentioned: ArcOSI and EnigmaIndicators.

As said before, the general idea in both utilities is the same:

1. Scrapes different open sources intelligence sites for known malware information as IPs, domains, urls, MD5 files hashes, email addresses, etc.
2. For each entry, create an CEF event with the source and type of intelligence and the threat information.
3. Send it via Syslog to a defined destination.

Both utilities are designed for a easy integration with ArcSight, using CEF, so no parser is needed.

And in both you can defined your own sources of information and whitelist specific entries.

The main different i can see are shown in the table below:

	ArcOSI (http://code.google.com/p/arcosi/)	EnigmaIndicators http://enigmaindicators.codeplex.com/
Scripting language used	Python	Bash (dependencies - bash, cut, grep, zgrep, sed, awk, curl, wget, sort, perl and *nix /dev/udp and/or tcp socket)
Types / number of reputation sources	IP / 7 Domain / 7	IP / 49 Domain / 35 Web requested URL / 8 URL file name / 8 User agent string /  2 Email address sender / 1 Email subject / 1 Suspicious files /4 News feed / 1 MD5 file hash / 7
Entropy calculation	N/A	Enigma calculates entropy (measures the randomness of possible outcome) against the relevant data it parses for advance heuristics detection

Do you know of any other interesting open source intelligence utility?

Prioritizing alerts using automated open source intelligence

After a very busy 2011, I'm starting 2012 with a new year's resolution: "To write posts more often". And here is the first one...

Lately I've been working in how to enhance the data using open source intelligence information. And I'm amazed how much value you can get from it.

The idea is to use reputation information from public sources and correlate it with your internal events in order to prioritize alerts. For example, in IDS/IPS alerts, you can correlate the external IPs in the IDS signatures against a list of known Malware IPs and increase the priority of if you get a match. Of course you can extend this to domain names, urls, etc. and also to different log sources as firewalls, proxies and so on.

SmartSIEM iPhone app

work in progress.....