Against Log Silos!

Posted on January 28th, 2008 | by Anton Chuvakin |

While the world of logging is full of inconsistencies and troubles (e.g. ugly logs!), there is one that beats many others: siloed approach to logs!

There is little that I hate more than siloed approach to logs. A situation where your security team “owns” network IDS logs, the network team has firewall and router logs (as well as all SNMP traps) and the system administrator has the logs from servers and desktop, is not only sad, counterproductive, inefficient, and wasteful, but it is also dangerous.

Where does the siloed approach to logs (when they are divided by both technical and political chasms) break down most painfully? In the case of an incident response, of course! An incident response is where, instead of one query across all logs and all log sources (or whatever needed subset of logs or log sources), you would end up running around, begging, connecting, waiting, swearing, waiting, downloading logs, digging in many places at once, waiting, grep, suffering with many UIs, swearing some more, etc. You would be doing all this, instead of connecting to your shiny new log management system and running a few reports, doing drilldowns, and searching across the relevant logs!

Where else does it break down? Compliance of course! Most regulations and mandates don’t call out logs by the log source type, instead they call all logs equally. Therefore having one system to verify the compliance status is much more productive compared to digging in many systems.

Ideally, you’d break down the silo walls by deploying a log management platform across the entire organization and then controlling access to every team requiring access to the system for logs, using the interface or a web API. Apart from being a trend (e.g. see recent ESG report), it will make your IT and security operations that much more efficient - and pleasant!

On the other hand, what is bizarre, is some newer vendors, claiming to do log management, actually work to propagate, not combat, the siloed approach. For example, selling the tool for $5,000 to each of the various teams within the organization IMHO must be made illegal :-) because it builds walls, not bridges; digs holes and overall “silo-izes” your IT operation…