Skip to main content

Rollback

Services​

Read the rollout history first, since revision numbers can shift.

kubectl -n startupos rollout history deploy/<svc>
kubectl -n startupos rollout undo deploy/<svc> --to-revision=N

rollout undo restores the image and the environment together. Roll back one service at a time.

:::warning Rolling back across the enforcement change The deployments before the OIDC enforcement change accepted caller-asserted identity headers. Rolling a service back to one of those revisions re-opens that for that service. The enforcement report lists the revision numbers. :::

If rollout undo is unavailable, set the previous image tag and restore the environment values the previous revision had.

Ingress​

Targeted changes to the ingress are saved before they are applied. To restore one, replace the object from the copy saved with the change. If replace complains about resourceVersion, delete the resourceVersion, uid and creationTimestamp lines or use apply.

Keycloak​

Each step of the Keycloak change has its own rollback in docs/runbooks/keycloak-fix-final-report.md:

  • High availability: kubectl rollout undo deploy/keycloak, scale to one replica and delete the PodDisruptionBudget. No data is touched.
  • New clients: delete identity-admin and startupos-services through the admin API and delete their two Secrets.
  • Service cutover to the new client: rollout undo per service. The old client stays enabled until it is retired, so a revert is always possible until then.
  • Re-enabling the retired client reopens its old secret. It is for emergencies only.

Frontend​

Roll back with kubectl rollout undo deploy/startupos-frontend, or set the previous image. The previous frontend image does not have the status page, so /status is gone after a rollback past it.

Sources: .agents/tasks/oidc-enforcement/finish/final-report.md, .agents/tasks/status-page/ingress.md, docs/runbooks/keycloak-fix-final-report.md.