What is self-healing?

Software that recovers from the failures of the things it depends on — automatically, without paging you for every incident.

What is it?

Every application leans on things that can fail: a payment gateway times out, a database gets slow, an external API starts returning errors. Traditionally, surviving those failures means writing defensive code by hand in every project, and getting woken up when you get it wrong.

Self-healing is the practice of building that survival in once, so the application detects a failure and responds to it on its own: it stops hammering a dependency that is down, retries a request that failed for a fleeting reason, falls back to a safe answer, and sets aside work it cannot finish yet instead of dropping it. Like a body closing a cut without being told to, a self-healing app keeps running through trouble and recovers when the trouble passes — instead of falling over and waiting for a human.

In Baldur, self-healing is a reliability layer you add to your Python app: circuit breaker, retry, fallback, and dead-letter queue, composed behind a single decorator.

Why it matters

The failures that actually hurt happen inside your service, in the code path of a single call: a double-charged customer, one slow dependency dragging down every request, a payment lost to a transient error. Handling them well has always meant hand-rolling the same breaker, the same backoff, and the same dead-letter plumbing in every project, and getting it right every time.

Without a self-healing layer, that work either does not get done or gets done inconsistently, and the cost shows up at the worst possible time:

One slow dependency takes down the whole app. Doomed calls pile up, threads and connections drain, and a single failing service makes your service look unhealthy to its callers too.
Transient blips become customer-visible errors. A failure that would have succeeded on the very next attempt is surfaced to the user instead of being quietly retried.
Work gets lost. A request that fails after a side effect (a charge, a write) leaves you with no record to retry and no way to reconcile.
A human is the recovery mechanism. Every incident becomes a page, a manual restart, a scramble to replay by hand.

A self-healing layer turns those into contained, automatic, recoverable events.

How it works in Baldur

You wrap a call with the @baldur.protected facade, one decorator that runs a circuit breaker by default and lets you opt in retry, a fallback, and dead-letter capture:

import baldur


@baldur.protected(
    "charge-customer",
    retry=True,
    dlq=True,
    idempotency_key="order_id",
    fallback=lambda: {"status": "unavailable"},
)
def charge(order_id: str) -> dict:
    return payment_gateway.charge(order_id)

Two of these arguments are safety rails for a money path. idempotency_key gives the charge a dedup guard, so a duplicate arrival of the same order (a double-submit, a redelivered webhook, a second worker) runs the side effect only once. It does not collapse Baldur's own retries. retry=True re-executes the function, so the work still has to be safe to repeat, and it is your payment provider's idempotency key, derived from the same order ID, that covers a retry whose first attempt may already have charged. And dlq=True is an explicit opt-in: Baldur never captures request snapshots without it, and a replayed entry executes the work again, so grant it only to calls that are safe to run a second time.

The fallback is worth reading closely too. It answers unavailable, not queued — on a user-facing charge the honest answer is that nothing happened and the caller should try again. Promising a queue would commit you to charging a customer who may have already walked away, which is exactly the case replay is not for. Setting a fallback also has a structural consequence: a served fallback counts as a handled call, so the failure is not captured for replay. Here dlq=True matters only if the fallback itself ever fails. For a user-facing charge that is the honest pairing: the caller gets a final answer, and no queued copy of the charge survives to run behind their back.

From then on, Baldur watches that call and responds to failure automatically:

flowchart LR
    A["your call"] --> B{"@baldur.protected"}
    B -->|"healthy"| C["dependency<br/>API · DB · queue"]
    B -. "transient failure" .-> D["retry with backoff"]
    B -. "dependency failing" .-> E["circuit breaker opens"]
    B -. "all retries fail" .-> F["fallback answer — or DLQ capture"]

A transient failure is retried with backoff, so a one-off blip never reaches the user.
A failing dependency trips the circuit breaker, so your app stops hammering it and fails fast instead of hanging.
When a call still cannot succeed, a fallback (if you set one) serves the caller a safe answer instead of an error — and because that answer is final, the failure is not also captured for replay. Without a fallback, work you opted in with dlq=True is set aside in the dead-letter queue to replay later instead of dropped.

Adoption stays cheap because there is nothing to stand up and nothing new to learn:

Zero infrastructure to start. With no configuration, Baldur runs on an in-memory fallback: no external services and no environment variables required. Add Redis when state has to be shared across processes; declaring the environment production makes it mandatory, since Baldur refuses to boot on per-worker memory rather than quietly degrade a shared guarantee. It is a library, not a sidecar or a separate service.
One API across frameworks. The same @baldur.protected works on Django, FastAPI, and Flask.

Where the automation stops. Baldur automates the failure responses it is designed for. It does not fix bugs in your code or guarantee your app never fails. Its job is to keep your app responsive and your data safe through a failure, and to recover automatically once the dependency comes back. For a failure it cannot safely resolve on its own, Baldur's rule is to make it loud, not silent: the failure is surfaced in logs and metrics (and, with PRO's escalation channels, pushed to a human) rather than swallowed. Self-healing where it can; a clear hand-off where it cannot.

The patterns it gives you

Each pattern handles one kind of failure for you. Start with the free OSS building blocks:

When this happens	Baldur's response	Guide
A dependency starts failing	Trip the breaker, fail fast, then probe for recovery	Circuit Breaker
A call fails for a fleeting reason	Retry with backoff	Retry
A request might run twice (a retry, a double-click)	Run the side effect only once	Idempotency
A load balancer asks "are you healthy?"	Answer truthfully so traffic routes around you	Health Check
The process is told to shut down	Drain in-flight work before exiting	Graceful Shutdown
A call fails for good and the work must not be lost	Capture it with its context, replay it when the dependency recovers	DLQ + Replay

There is more in the free tier too: Metrics, System Control, Bulkhead isolation, and Precomputed Cache.

Free to start, production-grade when you need it

The OSS patterns above are free and enough to get hooked. When you run Baldur in production for a team, PRO adds the heavier machinery: operating the dead-letter backlog at scale (batch replay from the console, adaptive pacing, archive/purge), an audit trail, thread-pool bulkhead isolation, canary recovery, and self-monitoring that escalates to a human when Baldur itself gets stuck.