Python logging is one of those things people ignore until they're knee-deep in a production incident, staring at a terminal and wondering if this is what the bad guy felt like in WarGames right before he "accidentally" launched the missiles.
Look, I get it. print() works. You don't have to read documentation. No weird configuration dictionary to copy-paste from Stack Overflow. You type print("here") and your brain gets a tiny hit of dopamine. Beautiful.
Then your Flask app throws a 500 in production, your logs are a soup of half-messages, and the only line you can find is: "something happened". Great. Real detective work there. Maybe next time just log "oops" and call it a day.
The logging stack: logger - handler - formatter
Here's what took me way too long to understand:
- Logger: where you call
logger.info(...) - Handler: where the log goes (console, file, syslog, HTTP, etc.)
- Formatter: how it looks (plain text, JSON, timestamps, request IDs)
If you only remember one thing: configure handlers and formatters once, at app startup. Not in every module. Not scattered across twelve files because you kept forgetting where you put it. That's how you get duplicate logs, missing output, and the creeping feeling that Python is gaslighting you.
getLogger(__name__) vs named loggers
I always use:
import logging
logger = logging.getLogger(__name__)
Because Python gives you a hierarchy for free. If your module is myapp.db.migrations, your logger name becomes myapp.db.migrations. You can turn down noise for one subsystem without muting the whole app.
Use a hard-coded named logger when you want a clear "channel" that isn't tied to module paths. Example: logging.getLogger("security") or logging.getLogger("billing"). I've used hard-coded loggers maybe three times in five years. __name__ handles 95% of cases.
Configure logging once (seriously, once)
If you call basicConfig() in multiple places, Python ignores every call after the first one. I wasted an entire afternoon on this once, changing the log level over and over, restarting the app, wondering if I'd broken reality.
Here's the configuration I use. Copy it. Change the log level and maybe the format string. You're done.
## logging_config.py
import logging.config
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"standard": {
"format": "%(asctime)s %(levelname)s %(name)s: %(message)s"
},
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"formatter": "standard",
},
},
"root": {
"level": "INFO",
"handlers": ["console"],
},
}
def setup_logging():
logging.config.dictConfig(LOGGING)
Then in your app entrypoint:
from logging_config import setup_logging
setup_logging()
Done. Now every module can call getLogger(__name__) and your logs will actually work.
Log levels are a contract
Log levels are a contract with Future You and whoever gets paged at 3 AM because the batch job exploded.
- DEBUG: noisy, for local dev or a short-lived production investigation
- INFO: normal business flow ("user created", "job started", "cache miss")
- WARNING: unexpected but not fatal ("retrying request", "fallback used")
- ERROR: the operation failed; someone will care
- CRITICAL: the app is on fire
Libraries default to WARNING because they don't want to fill your logs with noise about their internal retry logic. Your application code should log at INFO. I run INFO in production, then bump specific noisy loggers down to WARNING, or crank up a subsystem to DEBUG when I'm hunting a bug.
Structured logging with JSON (production likes JSON)
Text logs are fine when you're tailing one file. Production systems need search, filtering, and correlation across services. JSON works better for that.
Install:
pip install python-json-logger
Then swap the formatter:
from pythonjsonlogger import jsonlogger
LOGGING["formatters"]["json"] = {
"()": "pythonjsonlogger.jsonlogger.JsonFormatter",
"format": "%(asctime)s %(levelname)s %(name)s %(message)s",
}
LOGGING["handlers"]["console"]["formatter"] = "json"
Now your logs work with Elasticsearch, Splunk, or whatever log aggregator your ops team uses.
Add a request_id field and you can trace one user's request across three microservices instead of playing detective with timestamps.
"Just use print" (sometimes fine, sometimes a trap)
If you're writing a throwaway script or poking at an API in a Jupyter notebook, use print(). I do it all the time. This isn't a religious thing.
But in a web app, print() will cost you hours. You get no log levels, no timestamps, no way to tell which module printed what. And depending on your deployment, stdout might get buffered, swallowed by a process manager, or just disappear into the void.
Logging gives you control. When something breaks at 2 AM, you want control. It's not about crushing success, it's about recovering from catastrophic failure.
Use Sentry (and yes, DataDog is good too)
I use Sentry for exceptions because it tells me what broke, for which users, and how often. You get stack traces, breadcrumbs, release tracking, and alerts that don't require you to grep logs like it's 1999.
DataDog works great too, especially if you're already paying for their infrastructure monitoring. Pick one and wire it in early. Future You will thank you. Or at least curse you less when debugging production issues at 4 AM.
-Sethers