Centralized Logging & Casper Dashboard

By | December 10, 2015

Thanks to a retweet from the venerable Ben Toms, of MacMule fame, a tweet of mine has raised several questions:

The majority of the questions raised through Twitter or the Mac Admin’s Slack Channel centered around “What are you using to build that Dashboard?” The short answer to that question is Graylog.

Graylog

Graylog is an open source log management application for collecting and analyzing log data from virtually anywhere. It consists of the following components to build out the service:

  • Elasticsearch – For storage of log data and search
  • MongoDB – Used for storage of metadata.
  • Graylog-Server – Core application & log processing.
  • Graylog-Web – Web UI for search, dashboards and alerting

Graylog Components

Without delving into the details of deploying a full Graylog setup, just be aware that they do offer precompiled virtual appliances, packages for popular Linux distributions and scripts for many configuration management/orchestration tools. These are updated and available at  https://www.graylog.org/download-graylog/.

Why do I need log analysis?

Our Graylog deployment began as a way to make part of my job much easier and less tedious: Active Directory log tracing and monitoring.

Myself and one other admin are in charge of the care and maintenance of an Active Directory domain with over 70,000 users and 50,000 computers. When things go awry, it can be difficult (and time-consuming) to trace down what happened, and why. It’s not something I particularly enjoy so I started investigating ways to make the process simpler while also making an effort to be more proactive in addressing issues in the process.

Being a public K-12 institution, having a large operational expense outlay for log management didn’t really align with our core goals of educating the future, so luckily Graylog is open source software which meant the price was right. However, don’t let that statement that fool you, this is a powerful and flexible platform.

After seeing how easily it chewed through our AD logs, I had the thought: “Why can’t we do the same thing for JAMF’s Casper Suite?”

The remainder of this post will discuss the parts of my dashboard referenced above, where I get the data, and why it might be something to take advantage of for you. Examples will show Graylog, but can be also used with other tools for analyzing log data.


JSSAccess.log

Inset below in red, are all the data I have visualized using just one log: JSSAccess.log.

JSSAccess Log Data

Starting in Casper Suite v. 9.7, failed login attempts were recorded in the JSSAccess.log file. Depending on the Server OS your JSS Web App(s) are installed on, will determine where you can find this log:

  • Mac OS X – /Library/JSS/Logs/JSSAccess.log
  • Windows – C:\Program Files\JSS\Logs\JSSAccess.log
  • Linux – /var/log/JSSAccess.log
    • Note: If doing a manual install, you will need to update the log4j.properties file to reflect where you want the log to be saved. Next you will need to touch the file and assign ownership to tomcat. Otherwise, it defaults to the Mac OS X Path above and you’re log won’t actually be written!

The log contains a line for each login attempt in the following format:

Breaking this into pieces we have:

  • Date & Time Stamp
  • Username of login (or login attempt)
  • Login Status – “Successful Login” or “Failed Login”
  • IP Address of login (or login attempt)
  • Service Entry Point – I’ve submitted a feature-request to add additional details to this log (available here) but as of JSS 9.81, the following services are listed:
    • Self Service (OS X)
    • Casper Suite Application
    • Casper Focus
    • JSS (API)
    • JSS
    • Enroll

From this information I use Graylog to do the following:

  • Count successful and failed logins
  • Display their results over time
  • Analyze my top users by their login count over time
  • Count service usage to see where I should focus my time.
  • Verify my cluster is actually still working together and all servers are responding to logins

In the course of time, this can give me great insight into what my servers are actually doing, who is using my service and if there are any unauthorized, brute-force attempts to get access to the server. Using the Graylog Stream option, I can set alerting to notify me when any one of these metrics falls out of normal in my environment so that I’m not expected to actually monitor the dashboard 24×7 to discover anomalies.

Next, lets take a look at the other metrics shown in the post.

Webserver Logging

The remaining data visualized in the graphic above I extracted from our webserver logs. We host an external software distribution point on a server running Microsoft’s IIS Web and Application server.

The IIS logs look like the following:

From this information I can monitor the files that are being downloaded, the HTTP Status Returned, response time, and thanks to some good logic on Rich Trouton’s part, CasperCheck check-ins.

I’m using a tool called NXLog to send the IIS Logs from the Windows Server to Graylog as a JSON encoded message. This makes it easy to process within Graylog as it can natively separate the fields and display their names for me without any additional setup to extract the data in a useful format.

Casper Check Monitoring

I thought this would be a fun one to include to give me a quick glance of whether my fleet is actually doing what I tell it to and if it’s well distributed.

In his script, he uses an HTTP HEAD method to check the last modified date on the uploaded Zip file like the line below:

myCurl --head $fileURL 2>/dev/null | awk -F': ' '/Last-Modified/{print $2}'

Since I know the web server name, the HTTP Method that is used and (hopefully) the path to the CasperCheck zip within my web server, I can easily construct a search in Graylog to find out how, if, and when this is happening on my server:

Casper Check Search

Which then displays all the messages that match my query and gives me the option to add this information to my Dashboard in a few different ways:

Adding Data to Graylog Dashboard

As you can see, many devices are phoning home as they are powered on in the morning, as one would expect, with a more reasonable distribution throughout the rest of the day.

This has obviously been a very broad introduction to what we are doing with Graylog. I will focus a future post(s) on some of the finer detail surrounding how I extract the log from the JSS Web servers and actually feed it into Graylog but hopefully this has given you an idea of what is possible with the service.

 

3 thoughts on “Centralized Logging & Casper Dashboard

  1. dbrowning

    In Windows my JSSAccess log is located in C:\Library\JSS\Logs\JSSAccess.log

    Reply
    1. Imperatives

      dbrowning – As stated in the article the location of the JSSAccess.log file can be changed in the log4j.properties file . Unless it is there for a specific reason I would recommend changing it to C:\Program Files\JSS\Logs\JSSAccess.log so that all of your logs are in the same location. It is easier that way.

      Fred – great article! Working with ELK and graylog…seems like there are some interesting/significant improvements coming along with the next release of graylog, especially around the collector.

      Reply
      1. Freddie Post author

        Agreed! I’m looking forward to trying out v2 of Graylog. The collector sidecar will make it even easier to get data into Graylog. Thanks for the comments!

        Reply

Leave a Reply

Your email address will not be published. Required fields are marked *