# Outbound webhooks Receive committed Champagne lifecycle changes in an external backend. Source: https://docs.champagne.dev-ukab.fr/en/webhooks Champagne sends **outbound** HTTP POSTs to each club's configured backend endpoints. This is separate from the **incoming** Stripe endpoints at `/api/webhooks/stripe`. The receiving backend can decide whether to send email, SMS or push notifications; Champagne webhooks do not imply that any of those channels were used. Champagne also has an independent member inbox and email notification worker; see [Member notifications](/en/graphql/notifications). ## Configure [#configure] An administrator opens **Settings → Integrations → Webhook endpoints** to add up to 10 endpoints per club. Each endpoint has its own URL, event selection and signing secret, so separate systems can each receive only what they need. The multi-select options and descriptions come from a global catalog table. Adding an endpoint generates its secret to store in the receiving backend; the endpoint menu reveals or resets it, sends a test, pauses or deletes the endpoint. Pause an endpoint or select no events to queue none. Only public HTTPS DNS names are accepted. Do not point a real club at webhook.site: embedded workers may deliver actual club events there. Redirects are not followed; Stripe's incoming webhook secret is unrelated. Outbound delivery runs inside the API process locally and on Scaleway. Bootstrap initializes pg-boss's `pgboss` schema and `outbound-webhooks` queue before accepting requests, then starts four application-scoped delivery fibers polling every 500 ms. There is no separate worker image, container, CRON trigger, worker token or internal drain/test endpoint. The queue uses the same `PGHOST`, `PGPORT`, `PGDATABASE`, `PGUSER`, password secret (`PGPASSWORD_FILE` locally, `PGPASSWORD` in production) and `DATABASE_CA_PEM` as the application. Its separate pool is capped at three connections per API process. The runtime database role must be able to create the `pgboss` schema and its objects. Shutdown interrupts the scoped pollers and closes the queue pool. Keep at least one API replica running (`min_scale >= 1`) to process deliveries and retries without incoming traffic. Each additional API process adds four pollers and up to three queue connections alongside five `Database` and five separate `MailDatabase` connections. Size database capacity for **13 connections per API process**, with headroom for deployment overlap and operational clients. Scaling the API also scales webhook consumers; pg-boss coordinates claims across replicas. If every API process is stopped, jobs remain in PostgreSQL and processing resumes on startup, subject to queue retention. Releases deploy the API image by digest and use the database-backed GraphQL readiness probe. Because the listener starts after queue initialization and fiber startup, this gates bootstrap; it does not prove receiver availability or ongoing queue progress. The disposable production-image check also submits a safe job referencing no business event, waits for the embedded consumer to complete it without HTTP delivery, and checks that queue sessions are absent after process exit. Post-exit absence does not prove the scoped finalizer ran: process termination also closes sockets. Backend integration coverage separately verifies live scoped disposal while the process remains alive. For a receiver-only smoke test, an authorized club administrator uses **Test webhook** or [`sendClubTestWebhook`](/en/graphql/webhooks). This sends a synthetic signed event to the saved URL without member data; it bypasses the queue and does not test background retries. Use an isolated test club and a controlled receiver when validating the complete lifecycle. ### Existing-environment cutover [#existing-environment-cutover] This development change has no compatibility migration or automatic replay of legacy pending events. Choose the pending-event policy explicitly before deploying a version whose startup pushes the schema: 1. Pause business writes that could create outbound events. If pending legacy events must be retained, let the old worker drain them using the old schema, and verify no pending events or active claims remain. If this is disposable development data, explicitly approve discarding it instead; record that those notifications will not be delivered. Do not silently drop a backlog or assume pg-boss will adopt it. 2. Quiesce the old CRON trigger and any manual drain callers. Wait for all in-flight worker requests and claims to finish; stop the old worker before any schema push. Keep writes paused so the old producer cannot recreate a backlog between draining and cutover. Never run the legacy worker against the new schema. 3. Prepare the API's PG credentials/CA, schema privileges and capacity for 13 connections per process. Set `min_scale >= 1` and an integer `max_scale >= min_scale` through a reviewed configuration change. Deploy the combined API implementation only after the legacy consumer is quiescent. 4. Verify GraphQL readiness, the embedded startup record and fresh queue snapshots. With a controlled receiver and isolated test club, verify a committed event is delivered and retry behavior works; the direct **Test webhook** mutation alone does not prove queue consumption. Resume business writes once the embedded lifecycle is verified. 5. Remove the old worker container, trigger, IAM key/policy/application and worker/synthetic credentials through a separately reviewed Terraform plan/apply against the reconciled existing state. Remove obsolete tfvars and protected GitHub environment settings after cutover. Ordinary image CD does **not** delete infrastructure or revoke credentials. Preserve unrelated API secrets, database and DNS. All steps require the environment owner's approval; this source change performs no production operation. ## Transactional queue [#transactional-queue] Events enter `webhook_outbox` and pg-boss in the same business transaction, **once per enabled endpoint of the club that selected that event type**. Each endpoint's copy is a separate delivery with its own job, retries and outcome, so a failing receiver never delays the others. The repository uses pg-boss's `fromDrizzle` transaction adapter, capturing the current Effect context so its SQL executes on the reserved transaction connection. A rollback removes both the business change and queued job. Job data contains the outbox delivery ID, which is also the job ID. Every delivery of one event carries the same envelope `id`. API requests never wait for the receiving backend. ### Embedded delivery lifecycle [#embedded-delivery-lifecycle] Business changes and queue jobs share one commit; each running API process consumes the same queue. ```mermaid flowchart LR Request[Business request] --> Transaction[Drizzle Effect transaction] Transaction --> Business[Business rows] Transaction --> Outbox[One outbox delivery per subscribed endpoint] Transaction --> Adapter[pg-boss transaction adapter] Adapter --> Queue[PostgreSQL pg-boss queue] Queue --> Consumers[Four scoped fibers per API process] Consumers --> Receiver[Signed HTTPS delivery] Receiver --> Settlement[Transactional job and outbox settlement] Settlement --> Retry[Retry or complete] Retry --> Queue ``` A `2xx` response acknowledges delivery. Timeouts, redirects and non-2xx responses use pg-boss's retry policy: **20 retries**, exponential backoff starting at **2 seconds**, capped at **1 hour**, and a **60-second lease**. Retry timing also depends on polling and expiration recovery; there is no immediate-delivery or cross-event ordering guarantee. After retry exhaustion, the job is `failed` and no longer retries automatically. Queue retention and completed/failed job deletion are configured for 30 days; delivered outbox records are pruned after 30 days. Events during an unconfigured or paused period are not queued or backfilled. Deliveries are **at least once**: a crash after the receiver accepts a request but before Champagne records its acknowledgement can cause another POST. Persist `id` as a deduplication key before returning `2xx`. ### Concurrent deliveries and configuration changes [#concurrent-deliveries-and-configuration-changes] Each fiber fetches one job through pg-boss, which atomically claims due work across processes. No database connection or business-row lock is held while waiting for the receiver. The HTTP timeout is five seconds, with a ten-second overall delivery bound; the lease is not renewed. Each delivery uses its endpoint's URL and signing secret read when it is claimed. A secret rotation can finish while an earlier request is in flight; that request still uses the previous secret. Receivers should allow a short overlap during rotation for in-flight requests. Subsequent claims use the updated configuration. Pausing or deleting the endpoint, or deselecting an event, makes subsequent attempts skip delivery. A request that already read the configuration may still reach the receiver. Settlement requires an active job with the same retry count and an unexpired lease. Queue settlement and outbox outcome updates share a transaction. A late response from an expired or replaced attempt is ignored. A crash or interruption leaves the job active until pg-boss recovers it after expiration. This includes crashes after receiver acceptance, so receivers must deduplicate by event ID. ## Request contract (v1) [#request-contract-v1] The raw request body is UTF-8 JSON: ```json { "id": "a4e3d651-a06d-4e29-bd88-9870d6374a36", "type": "reservation.status_changed", "occurredAt": "2026-09-24T12:00:00.000Z", "clubId": "51000000-0000-4000-8000-000000000051", "data": { "reservationId": "05cb1d67-3935-4d87-a636-7ae1447f7060", "personId": "d60e28ed-8e79-4b9c-bc96-d1d93484af16", "categoryId": "1a3c3841-4d29-4394-8026-94341ef6f894", "previousStatus": "booked", "status": "confirmed", "startsAt": "2026-10-01T10:00:00.000Z", "endsAt": "2026-10-01T11:00:00.000Z", "guests": 1, "amountUnit": "credit_hundredths", "creditAmount": 2500, "monetaryAmount": null, "currency": null, "paid": false, "refunded": false } } ``` `Content-Type` is `application/json`; `X-Champagne-Event-Id` repeats `id`. `X-Champagne-Signature` is `t=,v1=`, calculated over the exact bytes of `t + "." + rawBody` using the club's one-time signing secret from Settings. Verify the HMAC in constant time, reject timestamps more than five minutes from your clock, then parse the JSON; do not reserialize before checking the signature. Keep a database unique constraint on the event ID. Return `2xx` only after durable acceptance, and process notifications asynchronously on your side. ## Emitted events [#emitted-events] All events have `clubId` and stable object IDs; no names, email addresses, tokens, application answers or invoice details are included. Fetch further context through the authorized GraphQL API if needed. | Type | When | `data` fields | | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | `application.status_changed` | A candidature enters draft, submitted, under review, accepted or rejected | `applicationId`, `personId` (nullable), `previousStatus` (nullable), `status`, `revision` | | `membership.granted`, `membership.updated`, `membership.renewed`, `membership.removed` | A staff or Stripe-driven membership change is recorded (no unchanged replays) | `membershipId`, `personId`, `planId` (nullable), `expirationDate` (nullable), `paidPeriodEnd` (nullable) | | `reservation.created` | A member or staff booking is persisted (including initially confirmed free bookings) | Reservation fields listed in the example; `previousStatus` is null | | `reservation.updated` | A booking's dates, resource names or guest count change without changing status | Reservation fields listed in the example | | `reservation.status_changed` | Booking moves to another status, including payment confirmation, cancellation/refund or completion | Reservation fields listed in the example | | `reservation.attendance_changed` | A reservation is marked present, late, absent or unmarked | `reservationId`, `personId` (nullable), `categoryId`, `previousAttendance`, `attendance` | | `invoice.issued` | A matching Stripe invoice first appears as open/overdue/paid, or leaves draft | `invoiceId`, `personId`, `status`, `amount`, `currency` | | `invoice.paid` | A projected invoice first becomes paid | `invoiceId`, `personId` | | `invoice.credited` | A new credit note appears on an already projected invoice | `invoiceId`, `personId`, `creditNoteId`, `amount`, `currency` | Reservation `status` values are `pending`, `booked`, `confirmed`, `completed`, `canceled`; attendance is independent (`unmarked`, `present`, `late`, `absent`). `startsAt` and `endsAt` are null for undated conversation requests. `paid` means a Stripe payment intent has been recorded, not that a payment request has been created. Invoice amounts are in the smallest currency unit. Reservation amounts follow the explicit units below. A first sync of existing Stripe invoices can emit `invoice.issued`/`invoice.paid`; credit notes present on the initial projection are not emitted. A ticket scan does not mark attendance. Application payment/signature answers do not change application status and do not produce status events. Same-state retries and unchanged membership writes do not emit events. ### Reservation amount units [#reservation-amount-units] Reservation lifecycle events (`reservation.created`, `reservation.updated`, and `reservation.status_changed`) publish a discriminated amount contract: | `amountUnit` | `creditAmount` | `monetaryAmount` | `currency` | | ------------------- | ------------------------------ | ---------------------------- | --------------------------- | | `credit_hundredths` | Integer hundredths of a credit | `null` | `null` | | `currency_minor` | `null` | Integer currency minor units | Currency code such as `eur` | A booking priced at 25 credits emits `creditAmount: 2500`, even if the club sells 2 credits per euro. No conversion using the current exchange rate is performed. Zero-price credit bookings emit `creditAmount: 0`. This is the reservation price; it does not include the separate refundable credit deposit. The retained direct Stripe reservation checkout path uses currency minor units. Reservations with a persisted Stripe account, checkout request/session, payment intent or refund identifier emit `currency_minor`, including before payment is confirmed. A Stripe purchase of credits does not mark individual credit bookings as monetary. `paid` and `refunded` indicate Stripe payment/refund records only; `false` does not mean a credit booking has an unpaid balance. The former reservation `amount` field is removed. Previously queued or archived events without `amountUnit` retain their original payload and have ambiguous units: consumers must not interpret them as money or credits automatically. Invoice payloads retain their existing monetary `amount` and `currency` fields. ## Member notification events [#member-notification-events] The following events supplement the original catalog. New catalog entries are available to club administrators; clubs with an existing saved selection must explicitly select the new types. Member inbox/email delivery does not depend on that selection or on a webhook URL. | Type | Trigger | Additional `data` | | --------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- | | `reservation.proposed` | Staff changes a pending reservation's proposal | `reservationId`, `personId`, `proposedStartsAt`, `proposedEndsAt`, `proposedFree` | | `reservation.message` | A professional posts a chat message | `reservationId`, `personId` (no message body) | | `chat.message` | A professional starts or answers a customer chat | `chatId`, `personId` (no subject or message body) | | `guest.joined` | Guest creates their account from an invitation | `invitationId`, `personId`, `recipientId` (member inviter, nullable for staff invitations) | | `reservation.invitation_accepted` | Guest accepts a reservation invitation (by name, account or as a member) | `reservationId`, `inviteeId`, `personId` (nullable), `recipientId` (member inviter, nullable for staff invitations) | | `sponsorship.joined` | Sponsored person receives membership | `applicationId` (nullable), `sponsorId`, `personId`, `recipientId`, `eventKey` | | `reservation.reminder` | Confirmed reservation starts within 24 hours | `reservationId`, `personId`, `scheduledFor`, `validUntil`, `eventKey` | | `waitlist.joined` | Waiting member is booked, automatically when a place opens | `reservationId`, `personId`, `sessionId`, `occurrenceId` (both null for a time slot) | | `membership.renewal_due` | Stripe paid subscription period ends within 7 days | `membershipId`, `personId`, `scheduledFor`, `validUntil`, `eventKey` | | `session.canceled` | Future occurrence canceled | `sessionId`, `occurrenceId`, `personId` for each affected recipient, `reservationId` for booked members | `reservation.proposed` also includes an `eventKey` identifying the reservation and proposal version, plus `validUntil` set to the proposed start time. Member email delivery uses these fields to skip obsolete proposals; outbound webhooks retain the original lifecycle event. Cancellation emits a recipient event per affected reservation/waitlist entry; an empty occurrence emits an event without a person. Recipient-level notification preferences do not suppress external webhooks. `eventKey` deduplicates scheduled/join milestones at production; delivery still requires receiver deduplication by envelope `id`. Existing webhook pause/no-backfill semantics remain. Sponsorship joining is deduplicated per sponsored person and sponsor, not per add-on or renewal. When a guest accepts a reservation invitation, the member inviter gets a "Your guest accepted" notification instead of a second `guest.joined` alert. Staff inviters have no member inbox and receive a direct email when their guest accepts a reservation or club guest invitation. There are still no events for credit balances, sanctions, Mews/Book4Time stays or treatments, restaurant/SevenRooms no-shows, Lightspeed house accounts, Salto access or push/SMS preferences. ## Operations [#operations] `webhook_outbox` retains one row per event and endpoint: the event, `endpointId`, attempt count, settlement timestamp and generic `lastError` (`delivery-failed`, `not-selected`, `disabled`, `endpoint-deleted` or `not-configured`, never a response body or secret). pg-boss owns scheduling, leases and terminal failure state; an unsettled outbox row alone does not prove a retry is pending. Inspect delivery metadata without selecting payloads or signing secrets: ```sql select o.id, o.type, o."clubId", o."endpointId", o.attempts, o."deliveredAt", o."lastError", j.state, j.retry_count, j.retry_limit, j.start_after, j.started_on from webhook_outbox o left join pgboss.job j on j.id = o.id and j.name = 'outbound-webhooks' where o."deliveredAt" is null order by j.start_after nulls last; ``` Check `webhook.queue` logs for queued, active, retry, failed and completed counts and oldest queued age; `webhook.delivery` records sanitized outcomes. Investigate terminal failures and missing job rows before assuming automatic retry will recover them. There is no internal drain route or documented automatic replay operation. Protect database access, Terraform state and receiver logs; the receiver must not trust unsigned webhook bodies or assume delivery implies a user notification. ### Delivery dashboard [#delivery-dashboard] The production bootstrap stack provisions the private **Champagne webhook delivery** dashboard from `infra/scaleway/bootstrap/dashboards/webhooks.json` using the existing managed Cockpit Loki source. Its 27 panels cover delivery outcomes and rates, retry attempts, duration percentiles, queue states and oldest queued age, snapshot freshness, worker failures/startup and event-type/log drilldowns. Review/apply the existing bootstrap stack to publish the dashboard, and deploy the core JSON logger contract before expecting data; a dashboard definition does not prove ingestion is live. Effect's `Logger.consoleJson` emits `message` values `webhook.delivery`, `webhook.queue` and `webhook.worker`, with fields nested under `annotations`. Queries read `annotations.module`, `annotations.outcome`, `annotations.eventType`, `annotations.attempt`, `annotations.durationMs`, the queue counts/age, and worker operation/failure kind. Queue snapshots describe the same shared database from each API instance: panels take the **MAX of the latest sample per instance**, never sum replicas or accumulate repeated snapshots. The latest-sample window is three minutes; the MAX is a conservative sampled view, not an atomic live database count. Check freshness alongside counts; missing or stale samples are not healthy zeros. Native log retention limits historical views. Local development continues using its existing worktree Waggle; no local Grafana is provisioned. The dashboard adds no alert rules, ingestion/retention billing changes or new compute. ## Incoming Stripe webhook diagnostics [#incoming-stripe-webhook-diagnostics] The `/api/webhooks/stripe` and `/api/webhooks/stripe/accounts` endpoints return a server-generated `X-Request-Id`. Unexpected processing failures remain HTTP 500 with `Webhook processing failed` and generate one correlated, sanitized Effect error log. Missing or invalid signatures remain HTTP 400 without an unexpected failure log. Raw event bodies, Stripe signatures and provider error messages are not logged. This concerns incoming Stripe callbacks, not the outgoing club webhook signature or event-ID contract described above.