Python logging — saving logs to a file & sending logs to an API

Get your logs to where they are needed to debug your code

Mike Huls

Jun 9, 2022 — 5 min read

We’re sending our logs to where they are needed (image by Dineo Motau on Unsplash)

Python Logging — Saving Logs to a File & Sending Logs to an API

Including logging into your Python app is essential for knowing what your program does and quick debugging. It allows you to solve an error as quickly and easily as possible. Your program can be logging out useful information but how can it notify you when something goes wrong? We cannot read the logs off the terminal in a crashed app!

This article shows you two ways of saving your logs using handlers. The first is the simplest; just writing all logs to a file using the FileHandler. The second uses a HttpHandler to send your logs to an HTTP endpoint (like an API).

Let’s first understand the concept of using handlers when logging. As you might know, you can log a message by creating a logger and then calling one of the logging methods on that logger like below:

When we call the debug, info of error methods in the example above, our logger has to handle those logging calls. By default, it just prints the messages (with some meta-data as specified with the format in the basicConfig) to the console.

In the parts below we add more handlers to the logger that do other things with the logs. Per handler, we can specify the level, fields, and formats as we’ll see below.

Code examples — implementing the handlers

In this article we’ll make it our goal to add three handlers to our logger that each has their own format:

stream handler
For printing to the console. We want to print all logs (debug and up)
file handler
saves logs in a file. We want to save all logs except debug logs
HTTP handler
sends logs over HTTP (to an API for example). We want to send only error and critical logs

All of these loggers will be configured separately; they will have their own format and level.

1: Setting up our logger

First, we’ll create our logger, nice and simple:

Notice that we don’t do anything with basicConfig anymore. We set the default level with the setLevel method, next we’re re going to specify the formatting and level for each handler separately.

A log in a stream (I’m almost done with the log-jokes) (image by Zack Smith on Unsplash)

2. Adding a stream handler

We’ll configure our stream handler to send the log to our console, printing it out:

Notice that we first create a Formatter specifically for stream handler. Then we’ll define the actual StreamHandler, specifying that we want to output to sys.stdout (console) and then we set the formatter to the handler. Then we add the handler to the logging object.

The result:

We’ve successfully printed the log to our console! Check out how to create colored logging to your console with the article below:

3. Adding a file handler

The steps are exactly the same as with the stream handler. The differences are that we specify another format and a different level.

Results:

The colors get added due to a PyCharm plugin (image by author)

Opening the test.log file shows us that we haven’t just written a file, we can also clearly see that the log has a different format from the streamHandler. Also notice that we do not save the debug logs to the file. Exactly like we wanted.

Once you’ve used the filehandler for a while your logging file gets quite large. Check out this article to solve that problem.

4. Adding an HTTP Handler

The steps are not much different from the previous one:

These short few lines of code will send our log over HTTP to a URL.

IMPORTANT: This handler has one major downside, however: it blocks the thread until it has finished the request. In the future, I’ll focus on implementing a custom logging handler for a non-blocking HTTP handler. Follow me and stay tuned!

Conclusion

In this article, we’ve gone through handlers for streams, files, and HTTP. There are some others that we haven’t handled like the socket handler, and the SMTP handler. Because this article is already pretty lengthy I haven’t covered these in this article but maybe will in future ones so make sure to follow me.

I hope everything was as clear as I hope it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of programming-related topics like these:

Happy coding!

— Mike

P.S: like what I’m doing? Follow me!