Sunday, February 21, 2021

Secure OCI Load Balancer Setup

Encryption of data in Transit

Assumption: Load Balancer is listening on Port 80,443 on the public side and ORDS standalone is listening on Port 8080,8443 on the private side.

End to End SSL

Use SSL to communicate with ORDS standalone
Ensure Health Check is using Port 8443

Add HSTS response header

Rule Set for adding HSTS response header
Listener on Port 443 with the rule set

Setup another listener on Port 80, rediect traffic to 443

Rule Set for Redirection
Listener on Port 80 with the rule set

Run SSL Labs Test

Saturday, February 20, 2021

Bash script to recycle all running instances in an instance pool

We have an instance pool, auto scaling with minimum 2 instances. After we update the instance configuration for the instance pool, we need to bring all running instances up to date.

The process is terminating one instance at a time, wait till the load balancer get back to "OK" and repeat. Eventually, we will terminated all instances with the old instance configuration with zero downtime.

recycle_instances.sh
#!/usr/bin/bash

COMPARTMENT_OCID="ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a"
LOAD_BALANCER_OCID="ocid1.loadbalancer.oc1.ca-toronto-1.aaaaaaaaxgcpd4izvacefithsabg3l4dhvl7y6gt2mcgitdlwt3ez45vzp5q"
INSTANCE_POOL_OCID="ocid1.instancepool.oc1.ca-toronto-1.aaaaaaaaswx6wld7z77u32shwivedvgn5usofurtjzfd3kdnfopbi56wqlfa"


# Wait till Load Balancer status is OK
lb_status_check () {
while [ "$LB_STATUS" != "OK" ]
do
  sleep 1m
  LB_STATUS=`oci lb load-balancer-health get --load-balancer-id $LOAD_BALANCER_OCID | jq -r '.data.status'`
  date
  echo "Load Balance Status: $LB_STATUS"
done
}

# Ensure Load Balancer is in OK status before we start
LB_STATUS=`oci lb load-balancer-health get --load-balancer-id $LOAD_BALANCER_OCID | jq -r '.data.status'`
lb_status_check

# Terminate all running instances in this instance pool
for INSTANCE_OCID in $(oci compute instance list --compartment-id $COMPARTMENT_OCID --lifecycle-state RUNNING | jq -r '.data[]| select(."freeform-tags"."oci:compute:instancepool" | contains("'$INSTANCE_POOL_OCID'"))? | .id') 
do
  date
  echo "Terminate instance OCID: ${INSTANCE_OCID}"
  oci compute instance terminate --force --instance-id ${INSTANCE_OCID}
  LB_STATUS="UNKNOWN"
  sleep 5m
  lb_status_check
done

date
echo "Recycle all running instances completed"

Instance Pool Lifecycle Management

List all custom images

oci compute image list --all --compartment-id ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a | jq '.data[]|select(."compartment-id" | contains("ocid"))? | ."display-name"'

Create a custom image based on the master instance

oci compute image create --compartment-id ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a --instance-id ocid1.instance.oc1.ca-toronto-1.an2g6ljrmpjzp2icj3zh7m5ndqp365ahhxv5j2b2u4t7omixuugdqwojymsq --display-name ords-as-v3

Create a new instance configuration

oci compute-management instance-configuration create  --compartment-id ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a --instance-details file://instance-config-ords-as-v4.json --display-name instance-config-ords-as-v4

Update the instance pool with the new instance configuration

oci compute-management instance-pool update --instance-pool-id ocid1.instancepool.oc1.ca-toronto-1.aaaaaaaaswx6wld7z77u32shwivedvgn5usofurtjzfd3kdnfopbi56wqlfa --instance-configuration-id ocid1.instanceconfiguration.oc1.ca-toronto-1.aaaaaaaasggwcn4pa3vejt5t3wthrze46fctsfw7hmnmbetxj4qav6mykrrq

List all running instances using the old instance configuration

oci compute instance list --all --compartment-id ocid1.compartment.oc1..aaaaaaaarocn3npultgruh5iwghhvor6s3kairokq4mil5bp52va6qkk7x6a --lifecycle-state RUNNING | jq '.data[]| select(."freeform-tags"."oci:compute:instanceconfiguration" | contains("ocid"))? | {"display-name": ."display-name", id: .id, ic: ."freeform-tags"."oci:compute:instanceconfiguration"}'

Teminate all running instances using the old instance configuration

oci compute instance terminate --instance-id ocid1.instance.oc1.ca-toronto-1.an2g6ljrmpjzp2icr34kwlkjldkmohstbs7bfwgtm3e5k2plcc2lox6ow5qa  --force

oci lb load-balancer-health get --load-balancer-id ocid1.loadbalancer.oc1.ca-toronto-1.aaaaaaaaxgcpd4izvacefithsabg3l4dhvl7y6gt2mcgitdlwt3ez45vzp5q

oci compute instance terminate --instance-id ocid1.instance.oc1.ca-toronto-1.an2g6ljrmpjzp2icfnl33iqrbcy7rbpn7ixg46s6dgb4tyrbdqtm2cfiv5iq --force

oci lb load-balancer-health get --load-balancer-id ocid1.loadbalancer.oc1.ca-toronto-1.aaaaaaaaxgcpd4izvacefithsabg3l4dhvl7y6gt2mcgitdlwt3ez45vzp5q

Friday, February 5, 2021

Apply Oracle Database Oct 2020 CPU Patch

Microsoft Windows BP 19.9.0.0.20 (& associated OJVM)

Download these files from Oracle Support

1. p6880880_190000_MSWIN-x86-64.zip - OPatch


2. p32172777_199000DBRU_Generic.zip - Patch 32172777: DATAPATCH SHOULD SKIP RUNNING STANDARD.SQL
3. p31719903_190000_MSWIN-x86-64.zip - Patch 31719903: WINDOWS DATABASE BUNDLE PATCH 19.9.0.0.201020
4. p31668882_190000_MSWIN-x86-64.zip - Patch 31668882: OJVM RELEASE UPDATE 19.9.0.0.0

Update OPatch

Open command window as Administrator

rename %ORACLE_HOME%\OPatch to %ORACLE_HOME%\OPatch_20210205
unzip p6880880_190000_MSWIN-x86-64.zip to %ORACLE_HOME%\OPatch

Apply Patch 31719903: WINDOWS DATABASE BUNDLE PATCH 19.9.0.0.201020

Take Windows Services screenshot, shutdown all Oracle services, change them to MANUAL startup and reboot

unzip p31719903_190000_MSWIN-x86-64.zip
cd 31719903
%ORACLE_HOME%\OPatch\opatch apply

Apply Patch 32172777: DATAPATCH SHOULD SKIP RUNNING STANDARD.SQL

unzip p32172777_199000DBRU_Generic.zip
cd 32172777
rename %ORACLE_HOME%\sqlpatch\sqlpatch.pm to sqlpatch_20210205.pm
copy files\sqlpatch\sqlpatch.pm to %ORACLE_HOME%\sqlpatch\

Start listener, start Oracle database service

sqlplus / as sysdba
startup
alter pluggable database all open;
exit

cd /D %ORACLE_HOME%\OPatch
datapatch -verbose

Enabling new optimizer fixes

SELECT * FROM V$SYSTEM_FIX_CONTROL;

EXECUTE dbms_optim_bundle.getBugsforBundle;

EXECUTE dbms_optim_bundle.enable_optim_fixes('ON','BOTH', 'YES');

CREATE PFILE FROM SPFILE;

Apply Patch 31668882: OJVM RELEASE UPDATE 19.9.0.0.0

Reboot

unzip p31668882_190000_MSWIN-x86-64.zip
cd 31668882 
%ORACLE_HOME%\OPatch\opatch prereq CheckConflictAgainstOHWithDetail -ph .
%ORACLE_HOME%\OPatch\opatch apply

Start listener, start Oracle database service

sqlplus / as sysdba
shutdown
startup upgrade
alter pluggable database all open upgrade
exit

cd /D %ORACLE_HOME%\OPatch
datapatch -verbose

Base on the screenshot taken earlier, put back the original startup setting

sqlplus / as sysdba
shutdown
exit

reboot

Re-compile all database objects

sqlplus / as sysdba
exec utl_recomp.recomp_parallel;