What to Check Before You Host a Small API

An API that serves twelve requests a minute feels finished the moment it answers the first curl command. The endpoint returns JSON, the logs show green, and the deployment script exits clean. That feeling lasts until the first night when a health check fails at 3:40 a.m. and nobody knows whether the box is down, the process died, or the certificate expired. Hosting a small API is mostly a discipline of deciding, in advance, which failures are worth waking up for and which can wait until morning.

The scale matters here. A service handling a few thousand requests a day sits in a strange middle ground: too small to justify a platform team, too important to leave entirely alone. The checks that follow are the ones that separate a service that hums along for two years from one that quietly rots and takes a side project down with it.

The difference between a process that runs and a service that survives

Process supervision gets skipped because the first deploy works. A developer starts the binary in a terminal, sees the log line, and moves on. The terminal closes, the process dies, and the API is gone. A supervisor fixes this: systemd, runit, s6, or whatever the host provides. The configuration is usually five lines. The point is that a crashed process restarts without human hands, and a process that crashes in a loop gets noticed rather than silently ignored.

Restart policy deserves a second look. A tight restart loop on a broken config can hammer a database or flood logs until disk fills. Setting a restart delay and a burst limit costs nothing and prevents a small mistake from becoming an outage with a paper trail.

Boot behavior is the other half. If the machine reboots, does the API come back on its own? Does it come back before the database is reachable? A service that starts too early and exits on a failed connection will stay down until someone intervenes. Ordering and retry logic belong in the startup path, not in a runbook nobody reads.

Whether the health check actually checks health

Most health endpoints return 200 as long as the web framework is alive. That tells an operator almost nothing. A process can hold the port open while its database connection pool is exhausted, its cache is unreachable, and every real request times out. The endpoint that should catch this is the one that touches the dependencies the service needs to do its job.

Two endpoints serve different purposes. A liveness check answers whether the process should be killed and restarted. A readiness check answers whether traffic should be routed to it. Collapsing both into one endpoint means a slow database triggers a restart storm, which makes the original problem worse. Keeping them separate means a service can be alive but not ready, which is often the honest state during a partial dependency failure.

External monitoring closes the loop. The health endpoint is only useful if something polls it from outside the machine. A local check cannot see a dead network interface, a full disk that prevents logging, or a host that has vanished from the provider's console. An external prober with a short timeout and a couple of retries catches the failures the local check cannot.

TLS expiry and the certificate that quietly lapses

Certificate expiry is the failure that surprises people because it works for months and then stops. A ninety-day certificate from a free authority renews automatically only if the renewal job runs. That job lives somewhere: a cron entry, a container sidecar, a systemd timer. If the host was rebuilt and the timer was not restored, the renewal never happens, and the first symptom is a browser warning and a broken integration.

Two habits prevent this. The first is checking the expiry date from an external monitor, not from the box itself, so the check catches a host that is down. The second is testing renewal before it matters: forcing a dry run of the renewal command on a staging host confirms the credentials and the challenge path still work. Certificate challenges fail silently when a firewall rule changes or a webroot path moves, and the failure surfaces weeks later at the worst time.

Log rotation and the disk that fills at 2 a.m.

Logs grow. A service with modest traffic can still produce gigabytes over a month if it logs every request with headers. The disk fills, the process cannot write, and the API starts returning errors that have nothing to do with its code. Log rotation with compression and a retention window is the fix, and it should be configured before the first deploy rather than after the first incident.

The rotation policy needs a size cap, not just a time window. Daily rotation on a service that can produce a gigabyte in an hour does not prevent the disk from filling between rotations. Combining size-based and time-based limits catches both steady growth and sudden bursts.

Where logs go matters too. Writing to a mounted volume that survives redeploys keeps logs available after a restart. Writing to a container's ephemeral filesystem means the evidence disappears exactly when someone needs it. On a small host, shipping logs to a remote collector or at least to a persistent volume is worth the setup.

Backups that have been restored at least once

A backup that has never been restored is a hypothesis. The database dump runs nightly, the object storage bucket fills, and everyone sleeps better. Then the day comes when a migration goes wrong, the restore is attempted, and the dump turns out to be empty, truncated, or missing a table that was added six months ago.

Restoring into a scratch environment once a quarter settles the question. It confirms the dump is complete, the credentials still work, and the restore procedure takes minutes rather than hours. The exercise also surfaces the awkward details: which environment variables the backup depends on, whether the storage bucket has versioning, and whether the restore needs a schema migration first.

Retention is the other half of the question. Keeping thirty days of dumps protects against a bad migration discovered a week later. Keeping only the last three days protects against almost nothing, because the corruption is usually discovered after the window has closed. Retention should match the time it realistically takes to notice a data problem.

Resource limits and the neighbor that eats the box

Small hosts run several things: the API, a database, maybe a reverse proxy, maybe a cron job that compiles something. Without limits, one of them can consume all available memory and trigger the kernel's out-of-memory killer, which shoots the largest process first. That process is often the database, and the resulting outage looks like a database crash rather than a memory problem.

Setting a memory limit on the API container or a systemd MemoryMax on the service keeps a runaway process from taking down its neighbors. The limit should be generous enough to avoid false kills and tight enough to leave room for everything else. The right number comes from watching normal memory use for a week, not from guessing.

CPU limits matter less on a single-tenant box but still prevent a background job from starving request handling. File descriptor limits and connection pool sizes are the quieter versions of the same problem: a service that opens more connections than the database allows will fail under load in a way that looks like a network issue.

Firewall rules and the port exposed by accident

A fresh host often starts with a permissive firewall and a note to lock it down later. Later never comes. The API port is open to the world, the database port is open to the world, and the admin panel on port 8080 is open to the world. Each of these is a small invitation.

The rule set should be explicit: allow the reverse proxy on 443, allow SSH from known addresses, deny everything else. Databases and internal services bind to localhost or a private interface. The check is to scan the host from outside and confirm only the intended ports respond. That scan takes a minute and regularly finds a service that was supposed to be internal.

Rate limiting before the first abuse, not after

An API with no rate limit is a free compute resource for anyone who finds it. A modest script can send thousands of requests per minute, exhaust the database connection pool, and degrade service for legitimate callers. The fix is a limit at the reverse proxy or in the application, chosen to match expected traffic with headroom.

The limit should be per key, not global. A global limit punishes everyone when one caller misbehaves. A per-key limit with a burst allowance handles normal spikes while containing abuse. Returning 429 with a Retry-After header tells well-behaved clients how to back off, which reduces the chance they retry in a tight loop and make things worse.

The dependency that fails and the timeout that saves the day

Every API talks to something: a database, a cache, an external service for payments or email. When that dependency slows down, the API's request handlers pile up waiting, threads or connections exhaust, and the service becomes unresponsive even though its own code is fine. Timeouts are the defense, and they are routinely left at default or set to infinity.

Every outbound call should have a timeout shorter than the request's own budget. A request that must complete in two seconds cannot afford a database query that waits thirty. Setting the timeout forces a fast failure, which surfaces the dependency problem instead of hiding it behind a hung request.

Circuit breakers and retries belong in the same conversation. Retrying a failing dependency without a limit multiplies load on something already struggling. A retry with a short backoff and a cap on attempts is safer. A circuit breaker that stops calling a dependency after repeated failures gives the dependency room to recover and gives the API a chance to return a graceful error instead of hanging.

What the on-call person needs at 3 a.m.

The final check is the least technical and the most often skipped: does the person who gets paged know what to do? A runbook that lists the service's dependencies, the commands to check each one, the location of the logs, and the escalation path turns a panic into a sequence of steps. Without it, the on-call person spends the first twenty minutes reconstructing context that should have been written down.

The runbook should name the specific failure modes the service has actually seen, not a generic list. It should say which alerts are urgent and which can wait. It should include the command to restart the service and the command to check whether the restart worked. These details feel obvious to the person who built the service and are invisible to everyone else.

A small API hosted well is a stack of unglamorous decisions made before they are needed. The process restarts on its own. The health check touches the database. The certificate renews and someone watches the date. The disk does not fill. The backup restores. The firewall denies by default. The timeouts fail fast. The runbook exists. None of these are exciting, and all of them are the difference between a service that runs for years and one that becomes a cautionary tale in a postmortem.