Change evidence report

Ground Stripe simulation contracts in real test-mode lifecycles

PR 1911 gives the deterministic Stripe double and the real Stripe sandbox one shared assertion owner. The offline lane now proves the fake implements Checkout, saved-card, refund, decline, authentication, subscription, invoice and event contracts. A separate bounded lane executes the same 69 assertions against Stripe test mode, including an actual $1.00 capture and refund plus a Test Clock renewal, then reconciles every mutable test object.

Range
54b6b4219, merge-base with origin/main…4f3041d88
Source state
feat/1909-stripe-simulation-extension at 4f3041d88, pushed as the current draft PR head
Generated

Reviewer brief

Outcome and scope

The fake and Stripe test mode pass the same lifecycle contract

The complete offline simulation passes 39 of 39 scenario files and fires 25 of 25 registered worker jobs. The real Stripe contract passes 69 shared assertions after creating an embedded Session, capturing and refunding a test card, observing decline and authentication errors, advancing a monthly Test Clock, and finding the real invoice.paid event. Account identity, a 30-request write budget, idempotency, cleanup reconciliation and a daily main-only workflow bound the external effects.

Provider grounding complements deterministic application simulation

Stripe prevents automated interaction with Checkout's payment UI and exposes no server API that completes a Checkout Session. This report therefore proves embedded Session creation and real server-side payment and billing lifecycles. Sauna's deterministic scenarios remain the evidence for Checkout fulfillment, webhook redelivery, tenant state and long business timelines.

Claim map

Evidence coverage

Each material changed behavior or state maps to direct evidence or an explicit gap.

Behavior or claimEvidenceStatus
FakePaymentProviders and Stripe test mode use one expectation ownerThe contract runner contains all 69 lifecycle assertions and accepts only an adapter for provider mechanicsCovered
The sandbox lane moves through actual test-mode card and billing statesObserved local run captured and refunded $1.00, handled decline and authentication PaymentIntents, paid an initial invoice, advanced one month and paid a distinct renewal invoiceCovered
External writes are fail-closed, bounded and reconciledTest-key and exact-account gates run before mutation; 30 writes is a hard ceiling; cleanup cancels, refunds, expires, deletes and archives with response-state checksCovered
The real Stripe sandbox lane does not become a per-PR dependencyThe default runner excludes sandbox files; a main-only protected environment runs the real contract daily or by manual dispatchCovered
A real customer completes embedded Checkout through the browserStripe disallows automated Checkout UI interaction; this boundary remains intentionally covered by deterministic fulfillment scenarios rather than a browser botGap

Observed change

Review evidence

Evidence is grouped by the reviewer question each item addresses.

Contract architecture

Table

One contract, two execution adapters

The adapter boundary changes setup and provider mechanics only. Observable assertions are shared.

SurfaceDeterministic laneStripe sandbox lane
ExecutionEvery normal simulation run, offlineDaily at 08:30 UTC and manual dispatch
CheckoutFake embedded Session create, retrieve and expireReal embedded Session create, retrieve and expire
Card lifecycleFake PaymentIntent, idempotent replay, refund and failuresActual test PaymentIntent capture, idempotent replay, refund, decline and authentication requirement
SubscriptionFake initial and renewal invoice statesStripe Test Clock initial invoice, one-month advance, renewal invoice and invoice.paid event
CleanupIn-memory resetFailed intents canceled, captured intents refunded, Session expired, clock and customer deleted, prices and products archived

Shared assertions

Code

Provider adapters cannot maintain separate expected answers

Checkout, payment, refund, decline, authentication and renewal observations flow through one assertion function. Cleanup runs even when contract assertions fail, and both errors are preserved when necessary.

Shared contract at head 4f3041d88.After
packages/simulation/src/harness/stripe-provider-contract.ts:247-330
const checkout = await adapter.observeCheckout();
expectCheckoutSnapshot(checkout.created);
expect(checkout.retrieved).toEqual(checkout.created);
expect(checkout.retrievedAfterExpiry.status).toBe("expired");

const card = await adapter.observeCardLifecycle();
expect(card.payment.status).toBe("succeeded");
expect(card.replayedPaymentId).toBe(card.payment.id);
expect(card.refund.paymentIntentId).toBe(card.payment.id);
expect(card.decline.code).toBe("card_declined");
expect(card.authenticationRequired.code).toBe("authentication_required");

const subscription = await adapter.observeSubscriptionLifecycle();
expectPaidInvoice(subscription.initial, "subscription_create");
expectPaidInvoice(subscription.renewal, "subscription_cycle");
expect(subscription.renewalEvent.type).toBe("invoice.paid");

try {
  await runAssertions(adapter);
} catch (error) {
  contractError = error;
}
try {
  await adapter.cleanup();
} catch (error) {
  cleanupError = error;
}

Sandbox safety

Code

The adapter proves mode and account before spending its write budget

A live key, missing expected account, mismatched account or disabled account fails before any mutable Stripe request. Every subsequent provider write reserves from one hard ceiling and uses a run-scoped idempotency key.

Fail-closed account and budget gates at head 4f3041d88.After
packages/simulation/src/scenarios/stripe-provider.sandbox.test.ts:66-102
if (!/^(sk|rk)_test_/.test(secretKey)) {
  throw new Error("Stripe sandbox contract requires a test-mode key");
}
const account = await stripe.accounts.retrieve();
if (account.id !== expectedAccountId) {
  throw new Error(`Stripe sandbox account mismatch`);
}
if (!account.charges_enabled) {
  throw new Error(`Stripe sandbox account is not enabled for charges`);
}

if (this.#providerWrites + count > MAX_PROVIDER_WRITES) {
  throw new Error(`Stripe sandbox write budget exceeded`);
}
this.#providerWrites += count;

return `sauna-contract-${this.#runId}-${operation}`;

Cleanup reconciliation

Code

Successful and failing runs reconcile every mutable provider object

The cleanup path cancels failed PaymentIntents, refunds captured intents, expires open Sessions, deletes the Test Clock and disposable customer, and archives every tracked Price and Product. Each response is checked, and cleanup failures fail the contract.

Bounded cleanup at head 4f3041d88.After
packages/simulation/src/scenarios/stripe-provider.sandbox.test.ts:442-552
for (const paymentIntentId of this.#paymentIntentsToCancel) {
  const paymentIntent = await this.#stripe.paymentIntents.cancel(
    paymentIntentId,
    { cancellation_reason: "abandoned" },
    { idempotencyKey: this.#idempotencyKey(`cancel-failed-${paymentIntentId}`) },
  );
  if (paymentIntent.status !== "canceled") throw new Error(/* state */);
}
for (const paymentIntentId of this.#paymentIntentsToRefund) {
  const refund = await refundStripePaymentIntent(/* stable key */);
  if (refund.status !== "succeeded") throw new Error(/* state */);
}
// Expire Sessions, delete clocks and customers, archive prices and products.
if (errors.length > 0) {
  throw new AggregateError(errors, "Stripe sandbox cleanup failed");
}

Scheduled grounding

Code

Real provider traffic is daily and main-only, not a merge gate

The workflow has read-only repository permission, non-overlapping execution, a ten-minute timeout and only the two protected environment secrets required by the contract.

Scheduled provider lane at head 4f3041d88.After
.github/workflows/stripe-provider-contract.yml:1-33
on:
  schedule:
    - cron: "30 8 * * *"
  workflow_dispatch:

permissions:
  contents: read

concurrency:
  group: stripe-provider-contract
  cancel-in-progress: false

jobs:
  stripe-contract:
    timeout-minutes: 10
    environment: provider-contracts
    steps:
      - name: Run bounded Stripe contract
        env:
          STRIPE_SECRET_KEY: ${{ secrets.STRIPE_SANDBOX_SECRET_KEY }}
          STRIPE_SANDBOX_ACCOUNT_ID: ${{ secrets.STRIPE_SANDBOX_ACCOUNT_ID }}
        run: bun run test:sim:stripe-contract

Confidence

Validation

Pass

Real Stripe sandbox lifecycle

At head 4f3041d88, fnox --config fnox.toml exec -- bun run test:sim:stripe-contract passed 1 test with 69 shared assertions in 27.18 seconds. The observed path created and expired an embedded Session, captured and refunded $1.00, exercised decline and authentication, advanced a Test Clock, observed a distinct paid renewal and invoice.paid, then completed reconciled cleanup.

Pass

Complete deterministic simulation

bun run test:sim passed 39 of 39 scenario files in 16.1 seconds and reported 25 of 25 registered worker jobs fired. The real sandbox file remained excluded from this offline lane.

Pass

Workflow and repository policy checks

bun run actions:lint passed 123 tests with 709 assertions, actionlint, pinned-action validation and PR evidence validation.

Pass

Affected package typechecks

@sauna-crm/api and @sauna-crm/simulation typechecks both exited successfully.

Pass

Independent review remediation

An independent reviewer found four cleanup and timeout risks. Head explicitly tracks catalog objects before dependent writes, cancels failed PaymentIntents, uses a 30-request exact success-path budget and reserves 60 seconds beyond both polling windows. Focused re-review found no remaining material issue.

Gap

Automated Checkout UI completion

Not exercised because Stripe prevents automated interaction with Checkout. Deterministic application scenarios cover the fulfillment and webhook-redelivery boundary instead.

Limits

Known issues, risks, and gaps

The remaining boundary is provider availability and Stripe's protected Checkout UI, not an unbounded payment side effect

Stripe remains a remote, rate-limited dependency The real lane can fail when Stripe is unavailable or changes a contract. It is scheduled daily and manually dispatchable rather than required on every pull request. The offline simulation remains the merge gate.

Checkout payment entry is intentionally not automated The sandbox creates the same embedded Session shape but actual capture is grounded through Stripe's supported test PaymentMethods and server APIs. This avoids brittle or prohibited browser automation while leaving deterministic fulfillment coverage intact.

Sandbox credentials are operational dependencies The protected provider-contracts environment is restricted to main and contains a test-mode secret plus the expected account ID. Missing, live-mode or mismatched values fail before mutation.