Add image precaching docs for aggregates

Related to blueprint image-precache-support

Partial-Bug: #1847302
Change-Id: I7a57e5e09b2a1760a9c5aeac402911895dfce07d

doc/source/admin/aggregates.rst

@@ -355,6 +355,38 @@ configuration:
 - :oslo.config:option:`filter_scheduler.aggregate_image_properties_isolation_separator`
 - :oslo.config:option:`filter_scheduler.aggregate_image_properties_isolation_namespace`
 
+.. _image-caching-aggregates:
+
+Image Caching
+-------------
+
+Aggregates can be used as a way to target multiple compute nodes for the purpose of
+requesting that images be pre-cached for performance reasons.
+
+.. note::
+
+    Some of the virt drivers provide image caching support, which improves performance
+    of second-and-later boots of the same image by keeping the base image in an on-disk
+    cache. This avoids the need to re-download the image from Glance, which reduces
+    network utilization and time-to-boot latency. Image pre-caching is the act of priming
+    that cache with images ahead of time to improve performance of the first boot.
+
+Assuming an aggregate called ``my-aggregate`` where two images should
+be pre-cached, running the following command will initiate the
+request:
+
+.. code-block:: console
+
+    $ nova aggregate-cache-images my-aggregate image1 image2
+
+Note that image pre-caching happens asynchronously in a best-effort
+manner. The images and aggregate provided are checked by the server
+when the command is run, but the compute nodes are not checked to see
+if they support image caching until the process runs. Progress and
+results are logged by each compute, and the process sends
+``aggregate.cache_images.start`` and ``aggregate.cache_images.end``
+notifications, which may be useful for monitoring the operation
+externally.
 
 References
 ----------

doc/source/admin/image-caching.rst

@@ -0,0 +1,77 @@
+=============
+Image Caching
+=============
+
+Nova supports caching base images on compute nodes when using a
+supported virt driver. As of the Ussuri release, that list includes
+``hyperv``, ``libvirt``, and ``vmwareapi``.
+
+What is Image Caching?
+----------------------
+
+In order to understand what image caching is and why it is beneficial,
+it helps to be familiar with the process by which an instance is
+booted from a given base image. When a new instance is created on a
+compute node, the following general steps are performed by the compute
+manager in conjunction with the virt driver:
+
+#. Download the base image from glance
+#. Copy or COW the base image to create a new root disk image for the instance
+#. Boot the instance using the new root disk image
+
+The first step involves downloading the entire base image to the local
+disk on the compute node, which could involve many gigabytes of
+network traffic, storage, and many minutes of latency between the
+start of the boot process and actually running the instance. When the
+virt driver supports image caching, step #1 above may be skipped if
+the base image is already present on the compute node. This is most
+often the case when another instance has been booted on that node from
+the same base image recently. If present, the download operation can
+be skipped, which greatly reduces the time-to-boot for the second and
+subsequent instances that use the same base image, as well as avoids
+load on the glance server and the network connection.
+
+By default, the compute node will periodically scan the images it has
+cached, looking for base images that are not used by any instances on
+the node that are older than a configured lifetime (24 hours by
+default). Those unused images are deleted from the cache directory
+until they are needed again.
+
+For more information about configuring image cache behavior, see the
+documentation for the following configuration options:
+
+- :oslo.config:option:`image_cache_subdirectory_name`
+- :oslo.config:option:`image_cache_manager_interval`
+- :oslo.config:option:`remove_unused_base_images`
+- :oslo.config:option:`remove_unused_original_minimum_age_seconds`
+- :oslo.config:option:`image_cache.precache_concurrency`
+
+.. note::
+
+   Some ephemeral backend drivers may not use or need image caching,
+   or may not behave in the same way as others. For example, when
+   using the ``rbd`` backend with the ``libvirt`` driver and a shared
+   pool with glance, images are COW'd at the storage level and thus
+   need not be downloaded (and thus cached) at the compute node at
+   all.
+
+Image pre-caching
+-----------------
+
+It may be beneficial to pre-cache images on compute nodes in order to
+achieve low time-to-boot latency for new instances immediately. This
+is often useful when rolling out a new version of an application where
+downtime is important and having the new images already available on
+the compute nodes is critical.
+
+Nova provides (since the Ussuri release) a mechanism to request that
+images be cached without having to boot an actual instance on a
+node. This best-effort service operates at the host aggregate level in
+order to provide an efficient way to indicate that a large number of
+computes should receive a given set of images. If the computes that
+should pre-cache an image are not already in a defined host aggregate,
+that must be done first.
+
+For information on how to perform aggregate-based image pre-caching,
+see the :ref:`image-caching-aggregates` section of the Host aggregates
+documentation.

doc/source/admin/index.rst

@@ -32,6 +32,7 @@ operating system, and exposes functionality over a web-based API.
    evacuate.rst
    flavors.rst
    huge-pages.rst
+   image-caching.rst
    live-migration-usage.rst
    manage-logs.rst
    manage-the-cloud.rst