Consolidated Logging / Syslog Solution (27001)

DHR

DHR

Soldato
Joined
30 Apr 2003
Posts
3,434
I've come to the point where I need to begin looking at logging for ISO 27001 compliance.

Budgets are tight (aren't they always!) but I'm thinking the easiest way is to use some form of syslog server in our Windows domain based environment and regularly review the most frequent or critical events logged?

Has anyone done this recently? Ideally looking at open source, free solutions to make our compliance lives just that little bit easier?
 
Soldato
Joined
18 Oct 2002
Posts
8,162
Location
The Land of Roundabouts
Graylog, if your not huge its pretty straightforward out of the box with a little learning investment, a bit more complex once you have the need to spread out the components!

its linux but its web based and unless you go down the rabbit hole of its capability you will never see know otherwise.

Otherwise theres free splunk if your not expecting massive amounts of data
 

Ev0

Ev0

Soldato
Joined
18 Oct 2002
Posts
14,172
ELK comes to mind as mentioned when looking at 'free' solutions, Graylog looks like it could be useful too.

Splunk and QRadar have free tiers if data volumes are low enough, not sure how good these are if you then need to expand into paid product land.

When looking at paid solutions Alienvault is usualy a cheap one, but if you just want centralised log storage might be overkill.
 
Last edited:
Soldato
Joined
14 Apr 2014
Posts
2,587
Location
East Sussex
ELK everytime for me, it's cross platform, widely used, the free version is great and there's a paid for version available at a reasonable price if needed, and it can deal with most common log formats straight out of the box.
 
Soldato
Joined
18 Oct 2002
Posts
8,162
Location
The Land of Roundabouts
Not sure what put me of ELK, greylog looks like an ideal one to get up and running today. @LizardKing you collecting windows logs with it?

Yeah we log all and everything with it and use it for various alerts (ie security group changes, firewall logins etc), then use Grafana to produce pointless fancy outputs that execs love.

It handles the 5-6million messages a day we throw at it with ease on a single vm. (firewalls can be very chatty!)
 

Ev0

Ev0

Soldato
Joined
18 Oct 2002
Posts
14,172
It handles the 5-6million messages a day we throw at it with ease on a single vm. (firewalls can be very chatty!)

At those event rates I’d imagine/hope most systems could handle that on a single vm.

How much resources you need to give the vm though, that’s another story ;)
 
Soldato
Joined
18 Oct 2002
Posts
8,162
Location
The Land of Roundabouts
At those event rates I’d imagine/hope most systems could handle that on a single vm.

How much resources you need to give the vm though, that’s another story ;)

lol I know what your getting at :D its all about scaling out these days. however for ref I cant remember the specs but its not a lot, something along the lines of 4gb ram, 2 ancient xeon cores and raid10 local storage.

Works out at about 200-300mb an hour i think so no not huge amounts of data. The elastic search works pretty well considering the specs!

Id have gone for splunk given its the industry standard, but the beancounters wouldn't have gone for it as its primarily a box ticking project.
 

Ev0

Ev0

Soldato
Joined
18 Oct 2002
Posts
14,172
Yeah it’s not too bad spec wise, pretty light really which is great when you want something at low/no cost that’ll also run on relatively inexpensive hardware.

I work in this area so know more about the commercial systems of this ilk than the ‘free’ stuff, but it is impressive what can be done on this side of it all.

Got a few clients who do some cool stuff taking data out of their systems and using Kibana to create some great visualisations.
 

DHR

DHR

Soldato
OP
Joined
30 Apr 2003
Posts
3,434
Go my teeth stuck into graylog, bit of a learning curve, windows logs were a nightmare to get right but i'm there now, well with the logging, need to come up with some fancy dashboards.

Currently starting to pipe in ASA logs which is a joy :(
 
Associate
Joined
4 Oct 2009
Posts
1,007
Have you gone for a single server deployment?
I am also trying this at the moment, currently single server but will look at splitting it all out if we go for it
 

DHR

DHR

Soldato
OP
Joined
30 Apr 2003
Posts
3,434
Have you gone for a single server deployment?
I am also trying this at the moment, currently single server but will look at splitting it all out if we go for it

Yes single server, I'm hoping it'll be enough for our environment, I'm selectively logging only critical systems at the moment. Firewall logs are going to be a challenge.

One thing I've not gotten my head around yet is retention and disk space management, anyone have any input on that when you're not using the enterprise version?
 
Soldato
Joined
18 Oct 2002
Posts
8,162
Location
The Land of Roundabouts
the rotations depends on your requirements. Ie do you need them for 30 days or are you space limited, or do you need to archive them off.

though to keep it simple its probably easier to just set the index size to 2gb, rotation strategy to delete, then set the max number so it works out to ~10gb below your HD space. then you can figure out your burn rate. Running out of space is not an easy fix :)

so a 50gb hard drive set the max number to ~20.

You can be clever and pipe different log sources to different indices sets and have separate retention policy's on those.

Theres some good dashboard templates in the market place for AD stuff that will get you going pretty quickly!
 

DHR

DHR

Soldato
OP
Joined
30 Apr 2003
Posts
3,434
Have I made a mistake not going for version 3? I've been working with 2.5 so far?

Having a nightmare with it trying to bring in Jira security logs too :(
 
Last edited:

DHR

DHR

Soldato
OP
Joined
30 Apr 2003
Posts
3,434
It was RC until today I think, I'm fuming!! Got notified of it by 2.5 :eek:

Being handed one over a grok pattern and extractor too.

I’m using the following pattern which works on the following log entry...

2018-03-19 15:11:46,181 https-jsse-nio-443-exec-246 JiraServiceAccount 911x8252778x1 - 192.168.0.1 /rest/api/2/search The user ‘ServiceAccount’ has PASSED authentication.

%{TIMESTAMP_ISO8601} %{DATA:UNWANTED} %{USERNAME:UNWANTED} %{DATA:UNWANTED} %{DATA:UNWANTED} %{IPV4} %{PATH}

I need to extract the username between quotes from “The user ‘ServiceAccount’ has PASSED authentication.” To do this i’ve set a grok pattern up called JIRASECUSER which is the following regex

(?<=The user ‘).*?(?=’) DeleteEdit

Again this works in isolation as does the %{JIRASECUSER} on it’s own in the extractor.

As soon as I try to use the following the filter refuses to run:

%{TIMESTAMP_ISO8601} %{DATA:UNWANTED} %{USERNAME:UNWANTED} %{DATA:UNWANTED} %{DATA:UNWANTED} %{IPV4} %{PATH} %{JIRASECUSER}

Totally clueless as to where to go from there now. Believe any errors are logged somewhere but I can't find the server.log anywhere on the OVA appliance :rolleyes:
 

Ev0

Ev0

Soldato
Joined
18 Oct 2002
Posts
14,172
Does Graylog come with any ready made parsers, or ready made connectors to get the data in from various sources?
 
Back
Top Bottom