diff --git a/doc/source/develop/layering.rst b/doc/source/develop/layering.rst new file mode 100644 index 0000000..70d4056 --- /dev/null +++ b/doc/source/develop/layering.rst @@ -0,0 +1,344 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _layering-and-deduplication: + +Layering and Deduplication +========================== + +Airship Layering Structure +-------------------------- + +Airship addresses the DRY (Don't Repeat Yourself) principle by promoting +the use of conceptual "layers" of declarative intent. Some configuration +is completely generic and reusable - for example, the basic Pod, +Deployment, and HelmRelease resources that make up a Kubernetes +application deployment. Other information will be fully site-specific - +e.g., the IP addresses and hostnames unique to a site. The site-specific +information typically needs to be injected into the generic resources, +which can be conceptualized as overlaying the site-specific layer +on top of the generic, reusable base. Although any number of layers could +be applied in arbitrary ways, Airship has chosen the following conceptual +layer types to drive consistency and reusability: + +* Functions: basic, independent building blocks of a Kubernetes workload. + Typical examples include a single HelmRelease or an SDN. +* Composites: Integrations of multiple Functions, integrating/configuring + them for a common purpose. Typical examples include OpenStack or + interrelated logging and monitoring components. Composites can also + pull in other composites to further customize their contents. +* Types: Prototypical deployment plans. A Type defines composites + into a deployable stack, including control plane, workload, and + host definition functions. A Type will define Phases of deployment, + which serve to sequence a deployment (or upgrade) into stages that + must be performed sequentially - e.g., the Kubernetes cluster must + be created before a software workload is deployed into it. + A type can pull in any number of Composites, and when appropriate + can inherit from exactly one other Type. + Types typically represent "use cases" such as a Network Cloud, a + CI/CD system, or a basic Kubernetes deployment. +* Sites: A Site is a realization of exactly one Type, and defines (only) the + site-specific configuration necessary to deploy the Type to a + specific place. + +The layers above are simply conventions. In practice, each layer is represented +by a Kustomization (see `Kustomize Overview`_ below), +and relationships between layers are captured by a +``kustomization.yaml`` referencing an external Kustomization. Additionally, +Airship layers may refer to layers in other code repositories, and Airship +will ensure all required projects are present and in expected locations relative +to a site defintion. This allows for operator-specific (upstream or downstream) +repositories to inherit and reuse the bulk of declarative intent from +common, upstream sources; e.g. the Airship Treasuremap_ project. +By convention, Airship manifests can be found in a project's +``manifests/`` folder, further categorized by the layer type -- e.g., +``manifests/function/my-chart`` or ``manifests/site/my-site``. + +TODO: add pictures + +The fundamental mechanism that Kustomize provides for layering is "patching", +which operates on a specific YAML resource and changes individual elements +or full YAML subtrees within the resources. Kustomize supports both +JSON patching and Strategic Merge patching; see `Kustomize Patching`_ +for more details. + +Kustomize Overview +------------------ + +Airship uses the Kustomize_ tool to help organize, reuse, and deduplicate +declarative intent. Kustomize has widespread usage in the Kubernetes +community, and provides a standalone command-line interface that lets a user +start with "generic" manifests in one location, and then apply overrides +from a different location. This can be used to perform operator- or +site-specific customization (hence the name) of resource namespaces, +patches against arbitrary YAML keys, and many other features. + +Kustomize also provides a Go library interface to drive its functionality, +which is used by projects such as ``kubectl``, ``kpt``, and ``airshipctl``. +Airshipctl incorporates Kustomize into its higher-order functionality, so +that it's invoked as part of ``airshipctl phase run``, rendering +manifests into a deployable form before deploying them directly to a +Kubernetes apiserver. However, Airship follows the design decision for +its YAMLs to be fully renderable using the stock ``kustomize`` command. + +The basic building block for Kustomize manifests is called a "kustomization", +which typically consists of a directory folder containing Kubernetes +resources, patches against them, and other information; and the file +``kustomization.yaml``, which acts as a roadmap to Kustomize on what +to do with them. A ``kustomization.yaml`` can also point out to other +kustomization directories (each with their own ``kustomization.yaml``) to pull +them in as base resources, resulting in a tree structure. In an Airship +context, the root of the tree would represent an Airship deployment phase, +and the leaves would be functions that capture something fine-grained, like +a HelmRelease chart resource. + +Plugins +------- + +Airship extends Kustomize via plugins to perform certain activities which +must be done in the middle of rendering documents (as opposed to before +or after rendering). The plugins are implemented as containerized +"KRM Functions", a standard which originated in the KPT project community. +Kustomize invokes these plugins to create or modify +YAML resources when instructed to via ``kustomization.yaml``. +The KRM Functions that are leveraged by airshipctl include: + +ReplacementTransformer +^^^^^^^^^^^^^^^^^^^^^^ + +Kustomize is very good at layering patches on top of resources, but has +some weakness when performing substitution/replacement type operations, +i.e., taking information from one document and injecting it into another +document. This operation is very critical to Airship from a de-duplication +perspective, since the same information (e.g. a Kubernetes version or +some networking information) would be needed in multiple resources. +Replacement helps satisfy the DRY principle. Although Kustomize has a few +flavors of variable replacement-type functionality, a plugin +was required to consistently support simple replacement, injection of +YAML trees into arbitrary paths of a target document, and substring replacement. + +Airship's ReplacementTransformer function is based on the example plugin +of the same name from the Kustomize project, and extends it with substring +replacement, improved error handling, and other functionality. + +The ReplacementTransformer is invoked by Kustomize when a transformer +plugin configuration with ``kind: ReplacementTransformer`` and +``apiVersion: airshipit.org/v1alpha1`` is referenced in a +``kustomization.yaml``. The plugin configuration payload contains +a list of replacement instructions, each with "source" and "destination" +definitions. + +See the `Replacement Transformer`_ documentation for examples and usage. + +Templater +^^^^^^^^^ + +While ReplacementTransformer modifies existing resources, the Templater +is a Kustomize "generator" plugin that creates brand new resources based +on a Go Template. This is helpful when you have a number of resources +that are nearly identical, and allows the common parts to be +de-duplicated into a template. An example of this would be resources +that are specific per-host (like Metal3 ``BareMetalHost``). + +The ReplacementTransformer and Templater can also be combined in a chain, +with the ReplacementTransformer injecting information into a Templater +plugin configuration (for example, the number and configuration of +``BareMetalHost`` resources to generate), so that it has all of the information +it needs to generate resources for a particular site. + +See the `Templater`_ documentation for examples and usage. + +Encryption, Decryption, and Secret Generation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Encryption and decryption is handled via the `sops` KRM Function maintained +by the KPT community. This function in turn uses the Mozilla SOPS tool +to perform asymmetric key-based decryption on encrypted values within +YAML resources. + +In addition, passwords, keys, and other sensitive secrets can be generated +from scratch on a site-by-site basis. This helps ensure that secrets are +properly randomized and are unique to particular deployments. This process +uses the Templater function to describe what secrets should be generated, +and the SOPS plugin to encrypt them immediately, before being written +to disk. + +See the `Secrets generation and encryption` guide for examples and usage. + +Replacement +----------- + +The resource patching functionality that Kustomize provides out-of-box +solves for many config deduplication needs, particularly when an operator +wants to define some override YAML and then apply it on top of exactly +one resource. However, it does not solve for the case where some piece +of configuration, e.g. a Kubernetes version number, needs to be applied +against multiple resources that need it. Using patches alone, this would +result in duplicating the same version number defined in multiple patches +within the same layer, increasing both maintenace effort and opportunity +for human error. To address this, Airship uses the ReplacementTransformer +plugin described above. + +The ReplacementTransformer can inject individual YAML elements or trees +of YAML structure into a specific path of a target resource. The source +for the injected YAML can come from any arbitrary resource. For the sake of +maintainability and readability, Airship uses "variable catalogue" resources +as replacement sources. These catalogues group related configuration +together with an expected document name and YAML structure, and their +sole purpose is to serve as a replacement source. Today, these catalogues +are a mix of free-form ``kind: VariableCatalogue``, and well-schema'd +per-catalogue kinds. The plan is to migrate all of them to well-defined +schemas over time. + +In general, Airship defines three things: good default configuration +at the Function level, default/example catalogues that also contain +(typically the same) default values, and ReplacementTransformer +configuration that copies data from one to the other. This conforms to +the Kustomize philosophy of base resources being "complete" in and of +themselves. In practice, we encourage operators to supply their own +catalogues, rather than basing on the upstream default/example catalogues. + +Versions Catalogues +^^^^^^^^^^^^^^^^^^^ + +Software versioning is frequently an example of information that should +be defined once, and be consumed in multiple locations. However, a +more compelling reason to pool versioning information together into +a single source catalogue, even when it will be consumed by exactly +one target document, is simply to put it all in the same place. +Versioning information also includes definition of registries +and repositories, so by defining all of your versions in one place, +it becomes more straight forward to change them all at once. +For example, an operator may choose to pull all Docker containers from +a downstream container registry instead of the default upstream registry. +Another example would be upgrading the versions of a number of +interrelated software components at the same time (e.g. a new OpenStack +release). On the other hand, a monolithic versions catalog would run the +risk of coupling unrelated software components together. + +Airship balances these concerns by typically defining one versions catalogue +per manifest repository (e.g. ``airshipctl``, ``treasuremap``, +``openstack-helm``), with a naming convention of ``versions-``. +This keeps the definition of default versions close to the Functions that define +them and the Composites that integrate them, and avoids cross-repo dependency +concerns. Within their home repository, the base catalogue should live in +a standalone Function (called something like ``catalogues-``, so +that it can easily be swapped out for alternate version definitions. +The per-repository catalogues all share the ``kind: VersionsCatalogue`` +schema, which is defined in the ``airshipctl`` repository. + +The definition of the replacement "rules", captured in the +ReplacementTransformer plugin configuration, will typically be done at the +Composite level as part of the integration work for which they're responsible. +Those Composites should each feature a README that details any catalogues +needed to render them. + +Note that versioning for Cluster API (CAPI_) providers will be handled slightly +differently, for a couple reasons: + +1. CAPI provider versions are a part of the directory structure (e.g. + ``manifests/function/capm3/v0.3.1/``, which are also closely coupled to + airshipctl itself. So, versioning of CAPI components is out-of-scope + for a catalogue-driven perspective, and is left to the airshipctl project. +2. CAPI container versions/locations must be handled differently, because + those resources are not pulled in via the same phase-driven approach + as everything else, and are instead referenced indirectly via a + ``Clusterctl`` resource (which is a normal, phase-included resource). + Since this ``Clusterctl`` config can pass variables into the clusterctl + rendering process, the solution will be: first, use the + ReplacementTransformer plugin to substitute container versions from the + ``versions-airshipctl`` catalogue into the ``Clusterctl`` resource; + then, let clusterctl itself push those values into the appropriate + CAPI resources via its variable replacement functionality. +3. To perform upgrades of CAPI components, two sets of versions, the old + and the new, will need to be substituted into the ``Clusterctl`` + resource simultaneously. Therefore, their version catalogue will need to + contain multiple sets of container versions, using the provider version + number as part of the YAML path (just like it's part of the directory + path above). + +Similarly to container versioning, ``HelmRelease`` resources will refer +out to Helm charts by location and version. This is still being defined +as of this writing, but in production use cases at least, Airship will use +a "Helm Collator" that will cache built charts in a single container image +and then serve the charts within the cluster. We may also support deploying +charts dynamically from github (as in Airship 1) if the Helm Controller +continues to support that feature. In any case, the plan is for the +chart locations/versions to be encoded in a versions catalogue(s), and for +that to drive the collator build process and/or live chart rendering. + +Note that the trigger gets pulled on replacement only at the Site/Phase level; +one can apply patches to catalogues till then. For example, if a catalogue +is defined at the Type level with normal default values, the values +can be overridden in the catalogue resource at the Site level before the +catalogue is actually used. + +TODO: include a lifecycle diagram that shows documents and replacements +getting aggregated, and then ultimately executed at site level. + +Note that Kubernetes resource versions are a different animal, and are not +addressed via catalogue replacement. + + +Host Catalogue and Host Generation Catalogue +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +TODO + +Network Catalogue +^^^^^^^^^^^^^^^^^ + +Networking is another example of a set of values that all change at once: +on an operator-by-operator basis (for shared network services like DNS), +and on a site-by-site basis (for subnet IP address ranges). +This information is extracted into a VariableCatalogue with +``name: networks``. Individual functions that consume the information will +provide their own replacent rules to do so. + +A default/example set of values is defined in the ``airshipctl-catalogues`` +function, and it can be patched (or duplicated) at the Type or Site levels to +apply operator- and site-specific information, respectively. + +Note that per-host IP addresses are generally specified in the Host Catalogue +rather than the Network Catalogue. + +Today, the Network Catalogue is specific to Functions in the ``airshipctl`` +repository, which defines a ``kind: NetworkCatalogue`` schema for it. + +Endpoint Catalogues +^^^^^^^^^^^^^^^^^^^ + +For internal cluster endpoints that are expected to be generally the same +between use cases, operator-specific types, and sites, there may be no need +to externalize the endpoint into the catalogue. In those cases, an operator +may still choose override the endpoint via a Kustomize patch. + +TODO + +Phases +------ + +TODO + +Treasuremap +----------- + +TODO + +.. _CAPI: https://github.com/kubernetes-sigs/cluster-api +.. _Kustomize: https://kubernetes-sigs.github.io/kustomize/ +.. _Kustomize Patching: https://github.com/kubernetes-sigs/kustomize/blob/master/examples/patchMultipleObjects.md +.. _Replacement Transformer: https://github.com/airshipit/airshipctl/tree/master/krm-functions/replacement-transformer +.. _Secrets generation and encryption: https://github.com/airshipit/airshipctl/blob/master/docs/source/secrets-guidelines.md +.. _Templater: https://github.com/airshipit/airshipctl/tree/master/krm-functions/templater +.. _Treasuremap: https://opendev.org/airship/treasuremap diff --git a/doc/source/index.rst b/doc/source/index.rst index 1cdcdda..d83af59 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -38,6 +38,7 @@ developers. What is Airship? learn/vulnerabilities.rst + develop/layering.rst .. toctree:: :caption: Try Airship