Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Team mission

🔗 Project information

📋 Roadmap overview

...

🔍 Detailed quarterly roadmap

Q1

...

Feature

...

Initiative

...

Dates

...

Priority

...

Effort

...

Status

...

Notes

...

Status
colourRed
titleHigh
/
Status
colourBlue
titleMedium
/
Status
colourYellow
titleLow

...

Status
colourRed
titleHigh
/
Status
colourBlue
titleMedium
/
Status
colourYellow
titleLow

Status
colourYellow
titleIn progress
/
Status
titlenot started
/
Status
colourGreen
titleShipped

Q2

...

Feature

...

Initiative

...

Dates

...

Priority

...

Effort

...

Status

...

Notes

Q3

...

Feature

...

Initiative

...

Dates

...

Priority

...

Effort

...

Status

...

Notes

Q4

...

Feature

...

Initiative

...

Dates

...

Priority

...

Effort

...

Status

...

Canary Endpoints:

Code Block
languagego
const (
	BlueService          SelectorName = "blue"
	GreenService         SelectorName = "green"
	InitialCanaryService SelectorName = "blue"
)

Discovered during QA that we did not have the ability to change the active canary service. I have added it:

Before:

Code Block
iiq-wp-platform git:(dev) kc get configmap -n wp-platform -o yaml
apiVersion: v1
items:
- apiVersion: v1
  data:
    canaryEnv: '{"selector":"blue","image":"961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud:dev"}'
  kind: ConfigMap
  ...

After request:

Code Block
POST to /services
{
    "selector": "green"
}
Code Block
➜  iiq-wp-platform git:(dev) kc get configmap -n wp-platform -o yaml
apiVersion: v1
items:
- apiVersion: v1
  data:
    canaryEnv: '{"selector":"green","image":"961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud:dev"}'
  kind: ConfigMap


Then simply update the image for the active canary service:

Code Block
languagejson
POST /services
{
  "imageTag": "216c0204fa1e71f93603c0d5087ef16d6b2ba5bce9084874bf9b2aebcddebc77"
}

...

Code Block
languagebash
➜  iiq-wp-platform git:(dev) kc describe -n wp-platform deployment/green-deployment
Name:                   green-deployment
Namespace:              wp-platform
CreationTimestamp:      Wed, 05 Apr 2023 15:06:04 -0600
Labels:                 app=green
Annotations:            deployment.kubernetes.io/revision: 63
Selector:               app=green
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:       app=green
  Annotations:  kubectl.kubernetes.io/restartedAt: 2023-04-11T05:00:45-06:00
  Containers:
   web:
    Image:      961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud@sha256:1274e8bc4d963536e88265781170e72a2939caae25127cd9d400ab124393f946

Reverting Bad Deployments

Example of broken deployment:

...

Once a healthy image hash is identified, we can simply update the canary deployment (or update the image on production if that one has gone bad) and the pods are restarted with the fresh code changes. The benefit of using dynamic image hashes during deployment is that our deployments themselves don’t lose historical context every rollout.

Once a healthy version of the WP app finishes building, it is ready to be deployed.

...


The pods, as well as any pipeline update hooks (like cache clearing, etc.) will run automatically.

Code Block
languagebash
➜  iiq-wp-platform git:(dev) kc describe  -n wp-platform deployment/green-deployment
Name:                   green-deployment
Namespace:              wp-platform
CreationTimestamp:      Wed, 05 Apr 2023 15:06:04 -0600
Labels:                 app=green
Annotations:            deployment.kubernetes.io/revision: 64
Selector:               app=green
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:       app=green
  Annotations:  kubectl.kubernetes.io/restartedAt: 2023-04-11T05:00:45-06:00
  Containers:
   web:
    Image:      961406424767.dkr.ecr.us-west-2.amazonaws.com/rezfusion-cloud@sha256:216c0204fa1e71f93603c0d5087ef16d6b2ba5bce9084874bf9b2aebcddebc77



Provisions

Misc. Fixes

  • Cache clearing jobs were missing the --allow-root param.