Bash script for Instance Pool Lifecycle Management

This script will create a new image, a new instance configuration based on the new image and update the current instance pool with the new instance configuration.

After that, we can run recycle_instances.sh to recycle all the instances. See my previous post for details.

instance-config-ords-as-template.json

{
    "instanceType": "compute",
    "launchDetails": {
      "compartmentId": "{{compartmentId}}",
      "createVnicDetails": {
        "assignPublicIp": false
      },
      "metadata": {
        "ssh_authorized_keys": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDuPoZBbXjC/5ojt+ECoIj5KGmLHWPPreWcjkp/1metOBgRv8f6W7w615+kRcrrdtyB5Tk6MzIs6CmF8RZ1BkkSankhHG62aKkqXn7T9VDenvEHaJpJqQRkhkYzJKyYqL+04O942gSgv8Kpw1IpFWvznfelf30xaxQzcLa0tMjYvOmqTmeAndEM3E1ZVMcEq3r3OlTTjCyfBPsdRPV2hFClvQziueRrUF61lhLotPUkCKxc6Iie+OpqW5hhU8vypFT0MAB6hoTH7EO7BGmQWInQlO3Pt4m7q9dNSee731TzRceDFa5cC/uigeAFgjEY8lwM5CFrcgMW3n3B3BpfumQv PC@PCLAPPY"
      },
        "shape": "VM.Standard.E3.Flex",
        "shapeConfig": {
          "memoryInGBs": 16.0,
          "ocpus": 1.0
        },
        "sourceDetails":   {
          "bootVolumeSizeInGBs": null,
          "imageId": "{{imageId}}",
          "sourceType": "image"
        }
    }
}

update_instance_pool_image_part2_template.txt

#!/usr/bin/bash

COMPARTMENT_OCID="{{compartmentId}}"
INSTANCE_POOL_OCID="{{instancepoolId}}"
INSTANCE_CONFIG_JSON="{{INSTANCE_CONFIG_JSON}}"
INSTANCE_CONFIG_NAME="{{INSTANCE_CONFIG_NAME}}"

INSTANCE_CONFIG_OCID=`oci compute-management instance-configuration create --compartment-id $COMPARTMENT_OCID --instance-details file://$INSTANCE_CONFIG_JSON --display-name $INSTANCE_CONFIG_NAME | jq -r '.data.id'`
sleep 30
oci compute-management instance-pool update --wait-for-state RUNNING --instance-pool-id $INSTANCE_POOL_OCID --instance-configuration-id $INSTANCE_CONFIG_OCID

update_instance_pool_image_part1.sh

#!/usr/bin/bash

COMPARTMENT_OCID="ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a"
INSTANCE_POOL_OCID="ocid1.instancepool.oc1.ca-toronto-1.aaaaaaaaswx6wld7z77u32shwivedvgn5usofurtjzfd3kdnfopbi56wqlfa"
INSTANCE_OCID="ocid1.instance.oc1.ca-toronto-1.an2g6ljrmpjzp2icj3zh7m5ndqp365ahhxv5j2b2u4t7omixuugdqwojymsq"

DATETIME=`date +"%Y-%m%d-%H%M"`
IMAGE_NAME="ords-as_"$DATETIME
INSTANCE_CONFIG_JSON="instance-config-ords-as_"$DATETIME".json"
INSTANCE_CONFIG_NAME="instance-config-ords-as_"$DATETIME
UPDATE_INSTANCE_POOL_SCRIPT="update_instance_pool_image_part2.sh"

sed "s/{{INSTANCE_CONFIG_JSON}}/$INSTANCE_CONFIG_JSON/g;s/{{INSTANCE_CONFIG_NAME}}/$INSTANCE_CONFIG_NAME/g;s/{{compartmentId}}/$COMPARTMENT_OCID/g;s/{{instancepoolId}}/$INSTANCE_POOL_OCID/g" update_instance_pool_image_part2_template.txt > $UPDATE_INSTANCE_POOL_SCRIPT
chmod 755 $UPDATE_INSTANCE_POOL_SCRIPT

IMAGE_OCID=`oci compute image create --compartment-id $COMPARTMENT_OCID --instance-id $INSTANCE_OCID --display-name $IMAGE_NAME | jq -r '.data.id'`

sed "s/{{imageId}}/$IMAGE_OCID/g;s/{{compartmentId}}/$COMPARTMENT_OCID/g" instance-config-ords-as-template.json > $INSTANCE_CONFIG_JSON

echo "Custom Image $IMAGE_NAME created. This instance will be taken offline for several minutes during the imaging process. Run update_pool_instance_image_part2.sh when instance come back online."

update_instance_pool_image_part2.sh

#!/usr/bin/bash

COMPARTMENT_OCID="ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a"
INSTANCE_POOL_OCID="ocid1.instancepool.oc1.ca-toronto-1.aaaaaaaaswx6wld7z77u32shwivedvgn5usofurtjzfd3kdnfopbi56wqlfa"
INSTANCE_CONFIG_JSON="instance-config-ords-as_2021-0308-0731.json"
INSTANCE_CONFIG_NAME="instance-config-ords-as_2021-0308-0731"

INSTANCE_CONFIG_OCID=`oci compute-management instance-configuration create  --compartment-id $COMPARTMENT_OCID --instance-details file://$INSTANCE_CONFIG_JSON --display-name $INSTANCE_CONFIG_NAME | jq -r '.data.id'`
sleep 30
oci compute-management instance-pool update --wait-for-state RUNNING --instance-pool-id $INSTANCE_POOL_OCID --instance-configuration-id $INSTANCE_CONFIG_OCID

Let's Encrypt Automatic Certificate Renewal in OCI Load Balancer

First we need to setup an acme instance to run the SSL certificate renewal against Let's encrypt nightly. To do that, the acme instance must be behind the load balancer with the SSL certificate associated with it. On top of that, we only want Let's Encrypt traffic reaching this instance. To achive this, we use Path Route Sets.

Add Backend Set

Add Backends

Add Path Route Sets

Assign Path Route Sets to the Load Balancer SSL listener

At this point, only the Let's Encrypt traffic will be routed to the acme instance. Normal traffic will not be affected.

acme.sh is probably the easiest & smartest shell script to automatically issue & renew the free certificates from Let's Encrypt. On the acme instance, we will do the followings

Install acme.sh

curl https://get.acme.sh | sh -s email=pchiu@leavemealone.com

Issue SSL Certificate

~/.acme.sh/acme.sh --issue -d pws.leavemealone.com -w /opt/oracle/ords/config/ords/standalone/doc_root

Install SSL Certificate

~/.acme.sh/acme.sh --install-cert -d pws.leavemealone.com --key-file /opt/oracle/ords/config/ords/standalone/leavemealone.com.key --cert-file /opt/oracle/ords/config/ords/standalone/leavemealone.com.pem --ca-file /opt/oracle/ords/config/ords/standalone/ca.cer --reloadcmd "/home/oracle/renew_lb_certs.sh"

Create and Install pkcs8 private key

~/.acme.sh/acme.sh --renew -d pws.leavemealone.com --to-pkcs8
cp ~/.acme.sh/pws.leavemealone.com/pws.leavemealone.com.pkcs8 /opt/oracle/ords/config/ords/standalone/
chmod 600 /opt/oracle/ords/config/ords/standalone/pws.leavemealone.com.pkcs8

Set notifications

export MAIL_TO="pchiu@leavemealone.com"
export MAIL_FROM="no-reply@leavemealone.com"
acme.sh --set-notify --notify-hook mail

At this point, we have setup acme.sh to renewal the cetificate automatically for us and we will put the new certifcate in /opt/oracle/ords/config/ords/standalone

The following bash script will take the new certificate in /opt/oracle/ords/config/ords/standalone and add it to the load balancer and update the listener to use it.

renew_lb_certs.sh

#!/usr/bin/bash

LB_OCID="ocid1.loadbalancer.oc1.ca-toronto-1.aaaaaaaaxgcpd4izvacefithsabg3l4dhvl7y6gt2mcgitdlwt3ez45vzp5q"
LISTENER_NAME="listener_lb_323"
BACKEND_SET_NAME="bs_lb_2021-0122-2326"
CERT_PATH="/opt/oracle/ords/config/ords/standalone"

CERT_NAME="pws.leavemealone.com_"`date +"%Y-%m%d-%H%M"`

~/.acme.sh/acme.sh --renew -d pws.leavemealone.com --to-pkcs8
cp ~/.acme.sh/pws.leavemealone.com/pws.leavemealone.com.pkcs8 /opt/oracle/ords/config/ords/standalone/
chmod 600 /opt/oracle/ords/config/ords/standalone/pws.leavemealone.com.pkcs8

oci lb certificate create --load-balancer-id $LB_OCID --wait-for-state SUCCEEDED --certificate-name $CERT_NAME --ca-certificate-file $CERT_PATH/ca.cer --private-key-file $CERT_PATH/pws.leavemealone.com.key --public-certificate-file $CERT_PATH/pws.leavemealone.com.pem
oci lb listener update --force --wait-for-state SUCCEEDED --listener-name $LISTENER_NAME --default-backend-set-name $BACKEND_SET_NAME --port 443 --protocol HTTP --load-balancer-id $LB_OCID --ssl-certificate-name $CERT_NAME --hostname-names \[\"pws.leavemealone.com\"\] --routing-policy-name acme --rule-set-names \[\"ADD_HSTS\"\] --cipher-suite-name oci-default-http2-ssl-cipher-suite-v1

#oci lb certificate list --all --load-balancer-id $LB_OCID

We still need to manually update the certificate between load balancer and instances in private subnet

Update Certificate in Backend Set