Add an idempotency check

Wire a Component’s pipeline to the Idempotency Service, so retried, redelivered, and replayed messages produce their side effect at most once.

Before you begin

• Read Idempotency — why duplicates are normal and what the check compares
• Have a Component under construction with the pipeline builder — on extractors and loaders idempotency is mandatory, and Build() throws until you configure it
• Have an Idempotency Service reachable through your Dapr sidecar (it ships with the runtime; for local development, docker compose -f local/compose.yaml up in the idempotency-service repo starts one)
• Know which field on your message identifies the business object (an order number, a customer number) and which field carries its business timestamp

How the check decides

Before the transform step runs, the pipeline sends the message’s id, content hash, and timestamp to the Idempotency Service and gets back proceed or ignore: an unseen id proceeds (NoPreviousData), a known id with identical content is ignored (SameData), changed content with a newer timestamp proceeds (NewerData), and changed content without a newer timestamp is ignored (StaleData). An ignored message cancels the pipeline — the remaining steps are skipped and the run completes as a traceable no-op, not an error. After a successful send, a recording step commits the message so the next delivery is recognised.

Records expire after 30 days

The service keeps each record for 30 days from its last commit. A replay of data older than that will re-run — plan long-horizon backfills accordingly.

Steps

1. Register the Idempotency Service client

WithIdempotency resolves IIdempotencyServiceClient from the service provider, so register the client in the component’s composition class (a scaffolded component already has this call):

dotnet add package Intropy.IdempotencyService.Client

src/Composition/Composition.cs

services.AddIdempotencyServiceClient();

The client targets the Dapr app id idempotency-service by default. Pass a delegate to override it (options.AppId = "..."), or bind it from configuration — AddIdempotencyServiceClient(builder.Configuration) reads the IdempotencyService section of appsettings.json.

Dedup state is scoped per component: the ComponentName you set in AddIntropyFramework is part of the storage key, so two Components checking the same message id never share or overwrite each other’s records.

2. On an Extractor, extract the id and timestamp from the message

The Extractor builder takes two extractor functions, both receiving the message and the pipeline context:

src/Composition/Composition.cs

var extractor = builder
    // ... pipeline steps ...
    .WithIdempotency(
        idExtractor:   (order, _) => order.OrderNumber,
        dateExtractor: (order, _) => order.ModifiedAt)
    .Build();

• idExtractor must return the same value every time the same business object is delivered. A business identifier from the source has that property; a UUID minted per delivery does not — it makes the check pass everything.
• dateExtractor must return the business modification time from the source, not the extraction time. It is what lets the service rank in-flight versions of the same id, so an update re-runs and a stale redelivery doesn’t. A timestamp that is fresh on every run turns every duplicate into NewerData.

3. On a Loader, fix the CloudEvent instead

The Loader builder takes no extractors:

.WithIdempotency()

It reads the id from the incoming CloudEvent’s Subject and the timestamp from its Time, both of which the deserialize step has already lifted into context. The choice you made in Step 2 therefore travels with the event: the upstream Extractor must set Subject to the business identifier and Time to the business timestamp. If either is missing, the check fails the pipeline with a technical failure naming the absent field — fix the producing Extractor, not the Loader.

4. Control what counts as “same content” (optional)

By default the content hash is a SHA-256 of the JSON-serialized message. To hash only the fields that matter — excluding enrichment noise or fields the destination ignores — implement IHashable on the message type; the framework hashes whatever GetHashString() returns:

public class OrderRecord : IHashable
{
    // ...
    public string GetHashString() => $"{OrderNumber}|{Status}|{Total}";
}

Both WithIdempotency overloads also accept a hashGenerator function for full control. Note the difference: with IHashable the framework still applies SHA-256 for you, while a hashGenerator’s return value is used as the hash verbatim.

Verify

Run the Component twice with the same input in your target environment, then inspect the second run’s trace:

• The Step.IdempotencyCheck span shows step.result=cancelled, the downstream step spans are absent, and the pipeline span still reports Ok — a duplicate is not an error
• Nothing new lands at the destination (check it directly)

Then send the same id again with changed content and a newer business timestamp: it should re-run (NewerData). Send it once more with an older timestamp: cancelled again (StaleData).

Symptoms and where to go back to:

• Every duplicate re-runs — the id or timestamp extractor is returning a per-delivery value (a fresh UUID, the extraction time). Revisit Step 2.
• The check itself fails with Failed to check idempotency — the client isn’t registered or the service isn’t reachable through Dapr; the pipeline fails closed rather than risk a duplicate side effect. Revisit Step 1.
• A Loader fails with Missing required CloudEvent.Subject (or .Time) — the upstream Extractor isn’t setting the event metadata. Revisit Step 3.