Enzo Candotti dbc1fa3dcd DCorch API Proxy enhacements for scalability
This commit updates the two funtionalities of dcorch-api-proxy,
notify() and enqueue_work(), in order to ensure shared resource
changes are sync'ed timely post dcorch redesign.

The logic of the notify() method is updated to utilize the new
EngineWorkerService through an RPC call.

For a high number of subclouds, the enqueue_work method in utils.py
significantly delayed the execution of a dcorch proxy command due to
creating objects in the orch_request table one by one within a for
loop for each subcloud. This logic has been changed to use
SQLAlchemy's bulk_insert_mappings method, which allows for inserting
all requests in a single session. This optimization reduced the
execution time from several minutes to instantaneous.

Test Plan:
PASS: Run a command with '--os-region-name SystemController' on a
scale LAB. Verify that:
   - The proxy creates the orchestration requests for each subcloud
     in the `orch_request` table.
   - The proxy updates the subcloud_sync table for the managed
     subclouds, setting `sync_request` to 'requested'.
   - The changes are synchronized to all the subclouds by the
     `sync_job`.
   - The delay for completing the proxy commandd is reduced
     significantly.

Story: 2011106
Task: 50308

Change-Id: I56d37df8ab78ddbe72f7670555c7a01bcb3d2917
Signed-off-by: Enzo Candotti <enzo.candotti@windriver.com>
2024-06-14 10:34:13 -03:00
..
2024-01-18 10:48:08 -03:00
2024-01-18 10:48:08 -03:00

api

DC Orchestrator API is Web Server Gateway Interface (WSGI) applications to receive and process API calls, including keystonemiddleware to do the authentication, parameter check and validation, convert API calls to job rpc message, and then send the job to DC Orchestrator Engine through the queue. If the job will be processed by DC Orchestrator Engine in synchronous way, the DC Orchestrator API will wait for the response from the DC Orchestrator Engine. Otherwise, the DC Orchestrator API will send response to the API caller first, and then send the job to DC Orchestrator Engine in asynchronous way.

Multiple DC Orchestrator API could run in parallel, and also can work in multi-worker mode.

Multiple DC Orchestrator API will be designed and run in stateless mode, persistent data will be accessed (read and write) from the DC Orchestrator Database through the DAL module.

Setup and encapsulate the API WSGI app

app.py:

Setup and encapsulate the API WSGI app, including integrate the keystonemiddleware app

apicfg.py:

API configuration loading and init

enforcer.py

Enforces policies on the version2 API's