Cloud Deployment Manager - Capabilities & Spec.

Canary Endpoints:

const (
	BlueService          SelectorName = "blue"
	GreenService         SelectorName = "green"
	InitialCanaryService SelectorName = "blue"
)

Discovered during QA that we did not have the ability to change the active canary service. I have added it:

Before:

iiq-wp-platform git:(dev) kc get configmap -n wp-platform -o yaml
apiVersion: v1
items:
- apiVersion: v1
  data:
    canaryEnv: '{"selector":"blue","image":"961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud:dev"}'
  kind: ConfigMap
  ...

After request:

POST to /services
{
    "selector": "green"
}

➜  iiq-wp-platform git:(dev) kc get configmap -n wp-platform -o yaml
apiVersion: v1
items:
- apiVersion: v1
  data:
    canaryEnv: '{"selector":"green","image":"961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud:dev"}'
  kind: ConfigMap

Then simply update the image for the active canary service:

POST /services
{
  "imageTag": "216c0204fa1e71f93603c0d5087ef16d6b2ba5bce9084874bf9b2aebcddebc77"
}

➜  iiq-wp-platform git:(dev) kc describe -n wp-platform deployment/green-deployment
Name:                   green-deployment
Namespace:              wp-platform
CreationTimestamp:      Wed, 05 Apr 2023 15:06:04 -0600
Labels:                 app=green
Annotations:            deployment.kubernetes.io/revision: 63
Selector:               app=green
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:       app=green
  Annotations:  kubectl.kubernetes.io/restartedAt: 2023-04-11T05:00:45-06:00
  Containers:
   web:
    Image:      961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud@sha256:1274e8bc4d963536e88265781170e72a2939caae25127cd9d400ab124393f946

Reverting Bad Deployments

Example of broken deployment:

Once a healthy image hash is identified, we can simply update the canary deployment (or update the image on production if that one has gone bad) and the pods are restarted with the fresh code changes. The benefit of using dynamic image hashes during deployment is that our deployments themselves don’t lose historical context every rollout.

Once a healthy version of the WP app finishes building, it is ready to be deployed.

The pods, as well as any pipeline update hooks (like cache clearing, etc.) will run automatically.

➜  iiq-wp-platform git:(dev) kc describe  -n wp-platform deployment/green-deployment
Name:                   green-deployment
Namespace:              wp-platform
CreationTimestamp:      Wed, 05 Apr 2023 15:06:04 -0600
Labels:                 app=green
Annotations:            deployment.kubernetes.io/revision: 64
Selector:               app=green
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:       app=green
  Annotations:  kubectl.kubernetes.io/restartedAt: 2023-04-11T05:00:45-06:00
  Containers:
   web:
    Image:      961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud@sha256:216c0204fa1e71f93603c0d5087ef16d6b2ba5bce9084874bf9b2aebcddebc77

Provisions

Misc. Fixes

Cache clearing jobs were missing the --allow-root param.