Heroku is one of the most famous cloud-based development platform as a service (PaaS) provider. It supports numerous development languages such as Ruby on Rails, Java, Node.js, Python, Scala and Clojure.
So knowing how to access your Heroku logs and what type of information you can get from them can become handy. This article will go over how to deal with your Heroku’s logs and what you can get out of them.
I. How does Heroku manage logs?
There are three types of Heroku logs:
- App Logs: Outputs from the application you pushed on the platform
- System Logs: Messages about actions taken by the Heroku platform infrastructure on behalf of your app
- API Logs: Messages about administrative actions taken by you and other developers working on your app
As an example, after we followed the Node.js tutorial provided on their website, here is what our logs end up looking like:
heroku logs --tails 2015-05-14T08:05:43.152629+00:00 app[web.1]: Somebody hit the page! 2015-05-14T08:05:43.531174+00:00 heroku[router]: at=info method=GET path="/" host=intense-lowlands-3421.herokuapp.com request_id=bfe2db27-3a78-4abf-b89c-371e8355f8af fwd="22.214.171.124" dyno=web.1 connect=1ms service=5ms status=200 bytes=191 2015-05-14T08:05:43.531456+00:00 app[web.1]: Somebody hit the page! 2015-05-14T09:07:24.599028+00:00 heroku[web.1]: Idling 2015-05-14T09:07:24.599458+00:00 heroku[web.1]: State changed from up to down 2015-05-14T09:07:28.381026+00:00 heroku[web.1]: Stopping all processes with SIGTERM 2015-05-14T09:07:30.997195+00:00 heroku[web.1]: Process exited with status 143 2015-05-19T16:05:55.162492+00:00 heroku[api]: Deploy d208654 by email@example.com 2015-05-19T16:05:55.162492+00:00 heroku[api]: Release v6 created by firstname.lastname@example.org
In this example, you can actually find the 3 types of Heroku logs mentioned above: App Logs (eg
app[web.1]), System Logs (eg
heroku[router]) and API Logs (eg
The most important thing to know when dealing with Heroku logs is that Heroku’s purpose is not to store all of the logs it is emitting. So the platform provides you with a system called Logplex.
Logplex has been designed for both collating and outing Heroku log messages. But once again, its purpose is not really to store Heroku logs, as it only keeps 1,500 lines of consolidated logs. Every log beyond that threshold will be lost in the dark universe of forgotten events… Plus tailing logs cannot really be leveraged by a team running production environments as history needs to be kept and search capabilities are limited… Which is why Heroku encourages you to route your logs to a log management system.
So if you’re data driven and really want to crunch data to inform your business or product decision making, you definitely need to extract your Heroku logs, and you have 2 choices about it:
heroku logs command or Logplex drains.
- Logs command: it allows you to get last events in your terminal. Logs command is probably only useful during the development phase of your app. Indeed, as it requires you to have an opened terminal, this command is not suitable for production environment.
- Logplex http/s drains: Heroku’s HTTP/S drains buffers log messages and submits batches of messages to an HTTPS endpoint via a
POSTrequest. The POST body contains syslog formatted messages, framed using the syslog TCP protocol octet counting framing method.
For our users out there who would want a step-by-step guide on how to connect it to logmatic.io as well as how to use tags to offset the caveat of drains not providing the names of your different application or environments , read here.
II. What insights can I get with my Heroku logs?
The Heroku router logger implements a proper Key/Value parser that provides a lot of information about HTTP calls distributed over your Dynos. This is very similar to the kind of information you get out from standard access logs and they look as follows:
at=info method=GET path="/" host=intense-lowlands-3421.herokuapp.com request_id=bfe2db27-3a78-4abf-b89c-371e8355f8af fwd="126.96.36.199" dyno=web.1 connect=1ms service=5ms status=200 bytes=191
Looks interesting, doesn’t it? From these, you are able to build great dashboards and follow the web activity of your apps using Geo IP and User-Agent parsers as we expose in our access log blogpost.
Heroku was created to make life easier for tech teams, and sending Heroku logs to your log management platform is also simple: it was made possible by the tremendous job of standardization done by Heroku!
If you are interested in other logging strategies check our articles on Nodejs or Syslog-ng logging, as well as our SSHD logs and security article. Or have a look at Logmatic.io addon on the Heroku’s app Store.