We have an instance pool, auto scaling with minimum 2 instances. After we update the instance configuration for the instance pool, we need to bring all running instances up to date.
The process is terminating one instance at a time, wait till the load balancer get back to "OK" and repeat. Eventually, we will terminated all instances with the old instance configuration with zero downtime.
recycle_instances.sh#!/usr/bin/bash
COMPARTMENT_OCID="ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a"
LOAD_BALANCER_OCID="ocid1.loadbalancer.oc1.ca-toronto-1.aaaaaaaaxgcpd4izvacefithsabg3l4dhvl7y6gt2mcgitdlwt3ez45vzp5q"
INSTANCE_POOL_OCID="ocid1.instancepool.oc1.ca-toronto-1.aaaaaaaaswx6wld7z77u32shwivedvgn5usofurtjzfd3kdnfopbi56wqlfa"
# Wait till Load Balancer status is OK
lb_status_check () {
while [ "$LB_STATUS" != "OK" ]
do
sleep 1m
LB_STATUS=`oci lb load-balancer-health get --load-balancer-id $LOAD_BALANCER_OCID | jq -r '.data.status'`
date
echo "Load Balance Status: $LB_STATUS"
done
}
# Ensure Load Balancer is in OK status before we start
LB_STATUS=`oci lb load-balancer-health get --load-balancer-id $LOAD_BALANCER_OCID | jq -r '.data.status'`
lb_status_check
# Terminate all running instances in this instance pool
for INSTANCE_OCID in $(oci compute instance list --compartment-id $COMPARTMENT_OCID --lifecycle-state RUNNING | jq -r '.data[]| select(."freeform-tags"."oci:compute:instancepool" | contains("'$INSTANCE_POOL_OCID'"))? | .id')
do
date
echo "Terminate instance OCID: ${INSTANCE_OCID}"
oci compute instance terminate --force --instance-id ${INSTANCE_OCID}
LB_STATUS="UNKNOWN"
sleep 5m
lb_status_check
done
date
echo "Recycle all running instances completed"
No comments:
Post a Comment