rewriting history

start to rewrite the purpose of 'ceilometer' and how it fits with
'telemetry'. this touches index and overview sections only.

updates
- fix gnocchi links
- correct goal of ceilometer (data collection only)
- make ceilometer api optional
- highlight purpose of ceilometer db vs gnocchi
- point out gnocchi is more responsive if not requiring full fidelity

Change-Id: Idce892227281d2cb1fe3c4c0cfdf048b487a3b7f

doc/source/architecture.rst

@@ -26,15 +26,16 @@ High-Level Architecture
 
 Each of Ceilometer's services are designed to scale horizontally. Additional
 workers and nodes can be added depending on the expected load. Ceilometer
-offers five core services, the data agents designed to work independently from
+offers three core services, the data agents designed to work independently from
 collection, but also designed to work together as a complete solution:
 
 1. polling agent - daemon designed to poll OpenStack services and build Meters.
 2. notification agent - daemon designed to listen to notifications on message queue,
    convert them to Events and Samples, and apply pipeline actions.
-3. collector - daemon designed to gather and record event and metering data
+3. (optional) collector - daemon designed to gather and record event and metering data
-   created by notification and polling agents.
+   created by notification and polling agents (if using Gnocchi or full-fidelity storage).
-4. api - service to query and view data recorded by collector service.
+4. (optional) api - service to query and view data recorded by collector
+   in internal full-fidelity database (if enabled).
 
 As Ceilometer has grown to capture more data, it became apparent that data
 storage would need to be optimised. To address this, Gnocchi_ (resource metering
@@ -42,7 +43,7 @@ as a service) was developed to capture the data in a time series database to
 optimise storage and querying. Gnocchi is intended to replace the existing
 metering database interface.
 
-.. _Gnocchi: https://wiki.openstack.org/wiki/Gnocchi
+.. _Gnocchi: http://docs.openstack.org/developer/gnocchi/
 
 .. figure:: ./ceilo-gnocchi-arch.png
    :width: 100%
@@ -255,12 +256,24 @@ divide the data into their own databases, tailored for its purpose. For
 example, a deployer could choose to store alarms in an SQL backend while
 storing events and metering data in a NoSQL backend.
 
+Ceilometer's storage service is designed to handle use cases where full-fidelity
+of the data is required (e.g. auditing). To handle responsive, long-term data
+queries, solutions that strip away some of the data's resolution, such as
+Gnocchi, are recommended.
+
+.. note::
+
+   As of Liberty, alarming support, and subsequently its database, is handled
+   by Aodh_.
+
 .. note::
 
    We do not guarantee that we won't change the DB schema, so it is
    highly recommended to access the database through the API and not use
    direct queries.
 
+_Aodh: http://docs.openstack.org/developer/aodh/
+
 
 Accessing the data
 ==================
@@ -268,7 +281,7 @@ Accessing the data
 API Service
 -----------
 
-If the collected data from polling and notification agents are stored in supported
+If the collected data from polling and notification agents are stored in Ceilometer's
 database(s) (see the section :ref:`which-db`), it is possible that the schema of
 these database(s) may evolve over time. For this reasons, we offer a REST API
 and recommend that you access the collected data via the API rather than by

doc/source/index.rst

@@ -17,25 +17,13 @@
 Welcome to the Ceilometer developer documentation!
 ==================================================
 
-The :term:`Ceilometer` project aims to deliver a unique point of
+The :term:`Ceilometer` project is a data collection service that provides the
-contact for billing systems to acquire all of the measurements they
+ability to normalise and transform data across all current OpenStack core
-need to establish customer billing, across all current OpenStack core
 components with work underway to support future OpenStack components.
 
-What is the purpose of the project and vision for it?
+Ceilometer is a component of the Telemetry project. Its data can be used to
-=====================================================
+provide customer billing, resource tracking, and alarming capabilities
-
+across all OpenStack core components.
-* Provide efficient collection of metering data, in terms of CPU and
-  network costs.
-* Allow deployers to integrate with the metering system directly or by
-  replacing components.
-* Data may be collected by monitoring notifications sent from existing
-  services or by polling the infrastructure.
-* Allow deployers to configure the type of data collected to meet
-  their operating requirements.
-* The data collected by the metering system is made visible to some
-  users through a REST API.
-* The metering messages are signed and :term:`non-repudiable`.
 
 This documentation offers information on how Ceilometer works and how to
 contribute to the project.

doc/source/install/dbreco.rst

@@ -15,9 +15,17 @@
 
 .. _choosing_db_backend:
 
-=============================
+=====================================
- Choosing a database backend
+ Choosing a database backend - Legacy
-=============================
+=====================================
+
+.. note:
+
+   Ceilometer's existing database capabilities is intended for post processing
+   and auditing purposes where responsiveness is not a requirement. It
+   captures the full fidelity of each datapoint and thus is not designed
+   for low latency use cases. For more responsive use cases, it's recommended
+   to store data in an alternative source such as Gnocchi_
 
 Selecting a database backend for Ceilometer should not be done lightly for
 numerous reasons:
@@ -49,3 +57,5 @@ MySQL              Yes                           Yes                 Yes
 PostgreSQL         Yes                           Yes                 Yes
 HBase              Yes                           Yes, except groupby Yes
 ================== ============================= =================== ======
+
+.. _Gnocchi: http://gnocchi.xyz

doc/source/measurements.rst

@@ -33,15 +33,3 @@ Adding new meters
 If you would like to add new meters please check the
 :ref:`add_new_meters` page under in the Contributing
 section.
-
-OSprofiler data
-===============
-
-All messages with event type "profiler.*" will be collected as profiling data.
-Using notification plugin profiler/notifications.py.
-
-.. note::
-
-  Be sparing with heavy usage of OSprofiler, especially in case of complex
-  operations like booting and deleting instance that may create over 100kb of
-  sample data per each request.

doc/source/webapi/index.rst

@@ -5,7 +5,6 @@
 .. toctree::
    :maxdepth: 2
 
-   v1
    v2
 
 You can get API version list via request to endpoint root path. For example::

doc/source/webapi/v1.rst

@@ -1,8 +0,0 @@
-============
- V1 Web API
-============
-
-.. warning::
-
-   The V1 API is being deprecated during the Havana development
-   cycle and no longer exists since Juno. 

doc/source/webapi/v2.rst

@@ -40,14 +40,27 @@ Samples and Statistics
 .. autotype:: ceilometer.api.controllers.v2.meters.Statistics
    :members:
 
-When a simple statistics request is invoked (using GET /v2/meters/<meter_name>/statistics), it will return the standard set of *Statistics*: *avg*, *sum*,
+When a simple statistics request is invoked (using GET /v2/meters/<meter_name>/statistics),
-*min*, *max*, and *count*.
+it will return the standard set of *Statistics*: *avg*, *sum*, *min*, *max*, and *count*.
+
+.. note::
+
+   If using Ceilometer data for statistics, it's recommended to use a backend
+   such as Gnocchi_ rather than Ceilometer's interface. Gnocchi is designed
+   specifically for this use case by providing a light-weight, aggregated model.
+   As they manage data differently, the API models returned by Ceilometer and Gnocchi
+   are different. The Gnocchi API can be found here_.
+
+.. _Gnocchi: http://docs.openstack.org/developer/gnocchi/
+.. _here: http://docs.openstack.org/developer/gnocchi/rest.html
 
 Selectable Aggregates
 +++++++++++++++++++++
 
-The Statistics API has been extended to include the aggregate functions *stddev* and *cardinality*. You can explicitly select these functions or any from the
+The Statistics API has been extended to include the aggregate functions
-standard set by specifying an aggregate function in the statistics query::
+*stddev* and *cardinality*. You can explicitly select these functions or any
+from the standard set by specifying an aggregate function in the statistics
+query::
 
     GET /v2/meters/<meter_name>/statistics?aggregate.func=<name>&aggregate.param=<value>
 
@@ -59,7 +72,9 @@ Duplicate aggregate function and parameter pairs are silently discarded from the
 
 are, on the other hand, both allowed by the API and supported by the storage drivers.  See the :ref:`functional-examples` section for more detail.
 
-.. note:: Currently only *cardinality* needs aggregate.param to be specified.
+.. note::
+
+   Currently only *cardinality* needs aggregate.param to be specified.
 
 .. autotype:: ceilometer.api.controllers.v2.meters.Aggregate
    :members:
@@ -116,7 +131,9 @@ nature of *resource_metadata* field, there is no practical or consistent way
 to validate the query fields under *metadata* domain like it is done for
 all other fields.
 
-.. note:: The API call will return HTTP 200 OK status for both of the
+.. note::
+
+   The API call will return HTTP 200 OK status for both of the
    following cases: when a query with *metadata.<field>* does not match its
    value, and when *<field>* itself does not exist in any of the records being
    queried.
@@ -156,7 +173,9 @@ supported: *=*, *!=*, *<*, *<=*, *>*, *>=* and *in*; and the following logical
 operators can be used: *and* *or* and *not*. The field names are validated
 against the database models. See :ref:`api-queries` for how to query the API.
 
-.. note:: The *not* operator has different meaning in MongoDB and in SQL DB engine.
+.. note::
+
+   The *not* operator has different meaning in MongoDB and in SQL DB engine.
    If the *not* operator is applied on a non existent metadata field then
    the result depends on the DB engine. For example if
    {"not": {"metadata.nonexistent_field" : "some value"}} filter is used in a query
@@ -195,8 +214,11 @@ Ceilometer CLI Commands::
 
      $ ceilometer --debug --os-username <username_here> --os-password <password_here> --os-auth-url http://localhost:5000/v2.0/ --os-tenant-name admin  meter-list
 
-.. note:: The *username*, *password*, and *tenant-name* options are required to be present in these commands or specified via environment variables. Note that the in-line commands will override the environment variables.
+.. note::
 
+   The *username*, *password*, and *tenant-name* options are required to be
+   present in these arguments or specified via environment variables. Note that
+   the in-line arguments will override the environment variables.
 
 .. _api-queries:
 
@@ -205,7 +227,13 @@ API Queries
 
 Ceilometer API calls:
 
-.. note:: To successfully query the Ceilometer you must first get a project-specific token from the Keystone service and add it to any API calls that you execute against that project. See the `OpenStack credentials documentation <http://docs.openstack.org/api/quick-start/content/index.html#getting-credentials-a00665>`_ for additional details.
+.. note::
+
+   To successfully query Ceilometer you must first get a project-specific
+   token from the Keystone service and add it to any API calls that you
+   execute against that project. See the
+   `OpenStack credentials documentation <http://docs.openstack.org/api/quick-start/content/index.html#getting-credentials-a00665>`_
+    for additional details.
 
 A simple query to return a list of available meters::
 
@@ -251,7 +279,11 @@ the CPU utilization for a given instance (identified by *resource_id*)::
        "http://localhost:8777/v2/meters/cpu_util/statistics?aggregate.func=max&aggregate.func=stddev"\
        "&q.field=resource_id&q.op=eq&q.value=64da755c-9120-4236-bee1-54acafe24980"
 
-.. note:: If any of the requested aggregates are not supported by the storage driver, a HTTP 400 error code will be returned along with an appropriate error message.
+.. note::
+
+   If any of the requested aggregates are not supported by the storage driver,
+   a HTTP 400 error code will be returned along with an appropriate error
+   message.
 
 JSON based example::