be0e85ec77
Continue/complete the rekey procedure when vault-manager is interrupted
(kill -9). Fixes include:
- Refactor logic of rekeyRecover function
- additionally handle specific failure scenarios to permit the rekey
procedure to continue
- correct return codes of procedure functions to fall through to the
recovery procedure
- resort the tests of needsShuffle
- misc adjustment of logs and comments
The additional handling of failure scenarios includes:
- partial deletion of cluster-rekey secrets after copying to
cluster-key
- restart rekey on failure during authentication
Test Plan: PASS vault sanity, ha sanity
PASS IPv4 and IPv6
PASS system application-update, and platform application update
PASS rekey operation without interuption
PASS bashate the rendered init.sh
Stability testing includes kubectl deleting pods and kill -9 processes
during rekey operation at intervals spread across the procedure, with
slight random time added to each interval
PASS delete a standby vault server pod
PASS delete the active vault server pod
PASS delete the vault-manager pod
PASS delete the vault-manager pod and a random vault server pod
PASS delete the vault-manager pod and the active pod
PASS delete the vault-manager pod and a standby pod
PASS kill -9 vault-manager process
PASS kill -9 active vault server process
PASS kill -9 standby vault server process
PASS kill -9 random selection of vault and vault-manager processes
Story: 2010930
Task: 49174
Change-Id: I508e93a36de9ca8b4c8fa1da7941fe49936de159
Signed-off-by: Michel Thebeau <Michel.Thebeau@windriver.com>