3ac15ffad8
This commit introduces a systematic approach for performing
standalone installations tailored for factory environments by:
1. Implementing a factory-install framework -
services with staged execution to facilitate the
operations necessary for installing, bootstrapping, and
deploying a site with factory defaults.
2. Leveraging cloud-init to stage necessary configurations
(using default cloud-init nocloud seed data) and to initiate
the factory installation process.
Overall, key features include:
- Configuration: Sets up factory default user and network
- Factory Staging: Sets up necessary services, configurations
and scripts for factory installation
- Service Execution: Utilizes systemd service for
executing various installation stages:
- Bootstrap: Executes the Ansible bootstrap playbook
- Config: Executes Ansible deployment manager playbook
- Setup: System setup scripts are triggered post-
deployment to ensure deployment is reconciled
- Tests: Test scripts are executed post-deployment
to validate system with health checks
- Localized Installation: Ensures factory installation
processes are streamlined for autonomous running, focusing on
local execution (no external dependencies) without site-specific
configurations
Test Plan:
1. PASS: Verify successful end-to-end factory installation.
Ensure the System in the unlocked-enabled-available state.
Furthermore, ensure:
1) No alarms raised
2) System/host reconciled
3) No errors reported in /var/log/factory-install
2. PASS: Verify cloud-int factory default user and network setup
- Sysadmin user password and OAM Network configured
- Ensure no errors reported in /var/log/cloud-init.log
and /var/log/cloud-init-output.log
3. PASS: Verify factory install staging, ensure:
1) /var/lib/factory-install created with
setup, tests, scripts and config contents
2) Config contents copied to HOME dir
3) factory-install systemd path, service, target units
copied to /etc/systemd/system/
4) /var/lib/factory-install/enabled flag set at start
5) factory-install.target service triggered
6) /var/lib/factory-install/state/final flag set upon
completion
4. PASS: Verify factory install services execution, ensure:
1) Streamlined staged execution. At most 1
factory install service unit must be active at a time,
strictly in the following order:
bootstrap, config, setup and test
2) All 4 path units must be active at start, with each
stage transition, the previous path+service
unit must go inactive
3) Intended service states pre/post host unlock
4) Intended service state after forced system reboot
5) /var/lib/factory-install/{stage,state}/ has all 4
flags set upon completion, and no flags at the start
5. PASS: Validate standalone bootstrap and deployment
(no network service dependencies)
6. PASS: Verify hardware check step
7. PASS: Verify test stage executed
Story: 2011100
Task: 49972
Depends-on: https://review.opendev.org/c/starlingx/metal/+/914995
Depends-on: https://review.opendev.org/c/starlingx/utilities/+/914993
Co-authored-by: Kyle MacLeod <kyle.macleod@windriver.com>
Change-Id: I63a5e62bcd119ccbefc35184afee4c102821995c
Signed-off-by: Salman Rana <salman.rana@windriver.com>
utilities
This file serves as documentation for the components and features included on the utilities repository.
PCI IRQ Affinity Agent
While in OpenStack it is possible to enable instances to use PCI devices, the interrupts generated by these devices may be handled by host CPUs that are unrelated to the instance, and this can lead to a performance that is lower than it could be if the device interrupts were handled by the instance CPUs.
The agent only acts over instances with dedicated vCPUs. For instances using shared vCPUs no action will be taken by the agent.
The expected outcome from the agent operation is achieving a higher performance by assigning the instances core to handle the interrupts from PCI devices used by these instances and avoid interrupts consuming excessive cycles from the platform cores.
Agent operation
The agent operates by listening to RabbitMQ notifications from Nova. When an instance is created or moved to the host, the agent checks for an specific flavor spec (detailed below) and if it does then it queries libvirt to map the instance vCPUs into pCPUs from the host.
Once the agent has the CPU mapping, it determines the IRQ for each PCI device used by the instance, and then it loops over all PCI devices and determines which host NUMA node is associated with the device, the pCPUs that are associated with the NUMA node and finally set the CPU affinity for the IRQs of the PCI device based on the pCPU list.
There is also a periodic audit that runs every minute and loops over the existing IRQs, so that if there are new IRQs that weren't mapped before the agent maps them, and if there are PCI devices that aren't associated to an instance that they were before, their IRQ affinity is reset to the default value.
Flavor spec
The PCI IRQ Affinity Agent uses a specific flavor spec for PCI interrupt affining, that is used to determine which vCPUs assigned to the instance must handle the interrupts from the PCI devices:
hw:pci_irq_affinity_mask=<vcpus_cpulist>
Where vcpus_cpulist can assume a comma-separated list of
values that can be expressed as:
int: the vCPU expressed byintwill be assigned to handle the interruptions from the PCI devicesint1-int2: the vCPUs betweenint1andint2(inclusive) will be used to handle the interruptions from the PCI devices^int: the vCPU expressed byintwill not be assigned to handle the interruptions from the PCI devices and shall be used to exclude a vCPU that was included in a previous range
NOTE: int must be a value between
0 and flavor.vcpus - 1
Example: hw_pci_irq_affinity_mask=1-4,^3,6 means that
vCPUs with indexes 1,2,4 and 6 from the vCPU list that Nova
allocates to the instance will be assigned to handle interruptions from
the PCI devices.
Limitations
- No CPU affining is performed for instances using shared CPUs (i.e.,
when using flavor spec
hw:cpu_policy=shared) - No CPU affining will be performed when invalid ranges are specified on the flavor spec, the agent instead will log error messages indicating the problem
Agent packaging
The agent code resides on the starlingx/utilities repo,
along with the spec and docker_image files that are used to build a
CentOS image with the agent wheel installed on it.
The agent is deployed by Armada along with the other OpenStack helm
charts; refer to PCI
IRQ Affinity Agent helm chart on
starlingx/openstack-armada-app repository.