I recently treated myself to a long overdue syslog server for our network devices. The system is on a Linux system (I would have it no other way) and has been instrumental to troubleshooting and auditing these past few weeks. You can read this article for some log parsing tricks ranging from basic to a bit more advanced.

Cat, Grep and Tail are the commands that truly set Linux based syslog servers apart from one running on windows. Don’t get me wrong, there are other decent alternatives but it’s hard to beat a typed command and an instant result.

There are millions of ways to spin these tools together into something useful. I will only be sharing a few but it should be good enough to get you off and running on your own. As always, the sky is the limit with a Linux system.

First, let’s look at the tail command. You can do something basic and tail a single file. The last ten lines will return by default. This is great for a “test and check” troubleshooting approach.

user@syslog1# tail /var/log/logfile

You can also tail a file and grep for something specific.

user@syslog1# tail  |  grep “string to search for”

Non of this is really that big of a deal. In fact, it’s somewhat limiting if you are watching a problem that happens live without warning as most network problems do. To spice up the tail options a bit we can tail continually with the “-f” option. This option follows the end of the file. This is useful when following a firewall log, for instance.

user@syslog1#tail –f /var/log/firewall

As the log file fills tail will return live results.
As most of you may have guessed a firewall that protects anything more than a printer is going to be spewing more log entries than practical to tail and read. Why not grep the result of the live tail as well? I personally like to pick out a known piece of information like an IP address in this case.

user@syslog1#tail –f /var/log/firewall | grep “192.168.1.23”

This will return any entry with that IP address present.
When working with a switched or routed network you can experience an issue affecting multiple systems at once and sometimes it’s nice to be able to see events from two or more systems at once. Again we can use the “-f” option to follow the log but need to add the “-q” option to keep the log file names from being printed. You could do something like this:

user@syslog1#tail –fq route1.log route2.log

or with grep

user@syslog1#tail –fq route1.log route2.log | grep EIGRP

Next, if you want to channel the tailed and greped result into a file to review later you can simpley pipe it to a file like this:

user@syslog1#tail –fq route1.log route2.log | grep EIGRP  > keepForLater.txt

Remember, a single “>” will over write and start a new file and “>>” will appended if a file by that name already exists.
Tail is a great tool for monitoring problems as they are happening. What about historic issues? For this we use the “cat” command in place of tail and its options. Beware, however, that a lot more output will be generated so be ready to tighten up the regular expression you use with grep.

I hope someone finds this useful! As always, feel free to drop a comment if you see a place for improvement or something I missed!