Rate Monasca / Ceilometer metrics with CloudKitty
This blogpost describes how to install and configure Monasca-Ceilometer and CloudKitty in order to allow CloudKitty to collect metrics through Monasca’s API.
Intended audience: OpenStack administrators, familiar with Ceilometer and CloudKitty. Having experience with Monasca is a big plus.
By Luka Peschke, Cloud Consultant and CloudKitty Developer @ Objectif Libre
Since the OpenStack Queens release, it is possible to integrate CloudKitty (OpenStack’s official rating component) with Monasca. In order to achieve this, we will install Monasca-Ceilometer. This projects allows Ceilometer to publish metrics to Monasca.
Pre-requisites
For the following steps, you will need a running OpenStack cloud with Monasca and Ceilometer (Ceilometer must be installed from source).
Step 1: Installing Monasca-Ceilometer
First, clone monasca-ceilometer:
$ git clone https://github.com/openstack/monasca-ceilometer /tmp/monasca-ceilometer
$ cd /tmp/monasca-ceilometer/ceilosca/ceilometer
Then, copy the relevant files to Ceilometer’s source directory:
$ cp monasca_client.py $CEILO_SOURCE_DIR/ceilometer/ $ cp publisher/* $CEILO_SOURCE_DIR/ceilometer/publisher/ $ cp -r ceilosca_mapping/ $CEILO_SOURCE_DIR/ceilometer/ $ cp opts.py monasca_ceilometer_opts.py $CEILO_SOURCE_DIR/ceilometer/
You will also need to copy some configuration files:
$ cp /tmp/monasca-ceilometer/etc/ceilometer/monasca_pipeline.yaml \ /tmp/monasca-ceilometer/etc/ceilometer/ceilosca_pipeline.yaml \ /tmp/monasca-ceilometer/etc/ceilometer/monasca_field_definitions.yaml \ /etc/ceilometer/
Once the files are copied, you will need to edit Ceilometer’s setup.cfg file. Add the following line to the ceilometer.sample.publisher = section:
monasca = ceilometer.publisher.monclient:MonascaPublisher
Event publishing isn’t supported in Monasca-Ceilometer yet, so you don’t have to edit ceilometer.event.publisher.
Ceilometer’s configuration files need to be adapted. Edit /etc/ceilometer/pipeline.yaml to publish metrics to Monasca. For example:
sources:
- name: meter_source
meters:
- "*"
sinks:
- meter_sink
[...]
sinks:
- name: meter_sink
transformers:
publishers:
- monasca://http://10.0.2.15:8070/v2.0
Then, edit ceilometer.conf:
# Add the following section: [monasca] service_auth_url = $KEYSTONE_URL service_region_name = RegionOne service_password = ******** service_username = ceilometer service_project_name = service service_domain_name = Default service_auth_type = password enable_api_pagination = True database_retry_interval = 5 database_max_retries = 5 # Add the following to the [database] section: [database] connection = monasca://http://10.0.2.15:8070/v2.0
You may also want to edit /etc/ceilometer/monasca_field_definitions.yml to add some dimensions (attributes) to your resources.
Once the config files are correct, (re)install Ceilometer:
$ cd $CEILO_SOURCE_DIR $ pip install . $ pip install python-monascaclient
Finally, add the “monasca-user” role to Ceilometer:
$ openstack role add --user ceilometer --project service monasca-user
You may now (re)start the different Ceilometer services. As Ceilometer publishes metrics identified on the “service” tenant, you can list your metrics like this:
$ openstack project list +----------------------------------+-------------------+ | ID | Name | +----------------------------------+-------------------+ | 9a7d3fdd2eec47b1b11a9c5a265e3d87 | myproject | | e5f331d9e8d1469aba1c80e5a79cdd01 | service | | f7b7b3560b8f455bbf406bd0cc12b1c3 | admin | +----------------------------------+-------------------+ # List metrics for 'myproject' tenant $ monasca metric-list --tenant-id e5f331d9e8d1469aba1c80e5a79cdd01 \ --dimensions project_id=9a7d3fdd2eec47b1b11a9c5a265e3d87
Step 2: Installing CloudKitty
First, clone and install Cloudkitty:
$ git clone https://github.com/openstack/cloudkitty -b stable/queens /opt/cloudkitty
$ pip install ./cloudkitty
# Copy the config files
$ mkdir /etc/cloudkitty
$ cp /opt/cloudkitty/etc/cloudkitty/* /etc/cloudkitty
Then, create the ‘cloudkitty’ database:
$ mysql -uroot -padminpass << EOF
CREATE DATABASE cloudkitty;
GRANT ALL PRIVILEGES ON cloudkitty.* \
TO $CK_DB_USER@'%' IDENTIFIED BY $CK_DB_PASS;
FLUSH PRIVILEGES;
EOF
Now, we need to add some basic configuration to Cloudkitty:
$ cat > /etc/cloudkitty/cloudkitty.conf << EOF
[DEFAULT]
debug = True
transport_url = rabbit://$RABBIT_USER:$RABBIT_PASS@$RABBIT_URL:$RABBIT_PORT/
# [oslo_messaging_notifications]
# topics = notifications
[service_credentials]
auth_url = $KEYSTONE_URL
region_name = RegionOne
password = ******
username = cloudkitty
project_name = service
project_domain_id = default
user_domain_id = default
auth_type = password
[keystone_authtoken]
auth_section = service_credentials
[database]
connection = mysql+pymysql://CK_DB_USER:CK_PASS@127.0.0.1/cloudkitty?charset=utf8
# The storage backend that CloudKitty should use
[storage]
backend = sqlalchemy
[tenant_fetcher]
backend = keystone
[keystone_fetcher]
auth_section = service_credentials
keystone_version = 3
# Auth infos for monascaclient
[collector_monasca]
auth_section = service_credentials
EOF
Finally, we will need to configure the metric collection. Edit /etc/cloudkitty/metrics.yml:
- name: OpenStack
collector: monasca
# CloudKitty will collect metrics every 3600 seconds. Each metric will be
# charged per 1hour periods
period: 3600
# CloudKitty will wait 2 collect periods, ie. the period from 9AM to 10AM
# will be processed at 12AM. This leaves some time to the storage backend
# for metric aggregation
wait_periods: 2
# For this post, we will rate cinder volumes and glance images
services:
- volume
- image
# Monasca has no resource notion, so we directly map services to metrics
services_objects:
volume: volume.size
image: image.size
# Specifying the aggregation method for each metric
services_metrics:
volume:
- volume.size: max
image:
- image.size: max
# Unit specification and conversion
metrics_units:
volume:
volume.size:
unit: GiB
# The image.size metric is in bytes, but we want to rate Mebibytes.
# You can specify a factor and an offset for each metric, the measures
# will be converted the following way:
# result = measures * factor + offset
# (factor being 1 and offset 0 by default)
image:
image:
unit: MiB
factor: 1/1048576
That’s it for the configuration part. CloudKitty must now be registered as an OpenStack service:
# Register the rating service $ openstack user create --password ****** cloudkitty $ openstack service create --name cloudkitty rating $ openstack role add --user cloudkitty --project service admin $ openstack role create rating # Create the required endpoints $ openstack endpoint create --region RegionOne rating admin http://$CK_URL:8889 $ openstack endpoint create --region RegionOne rating public http://$CK_URL:8889 $ openstack endpoint create --region RegionOne rating internal http://$CK_URL:8889 # IMPORTANT: Allow cloudkitty to fetch monasca metrics $ openstack role add --user cloudkitty --project service monasca-user # Configure CloudKitty to rate our project $ openstack role add --user cloudkitty --project myproject rating # Check that everything is fine $ openstack role assignment list --user cloudkitty --names +--------------+--------------------+-------+-------------------+--------+-----------+ | Role | User | Group | Project | Domain | Inherited | +--------------+--------------------+-------+-------------------+--------+-----------+ | rating | cloudkitty@Default | | myproject@Default | | False | | admin | cloudkitty@Default | | service@Default | | False | | monasca-user | cloudkitty@Default | | service@Default | | False | +--------------+--------------------+-------+-------------------+--------+-----------+
We can now initialize CloudKitty’s storage backends:
$ cloudkitty-dbsync upgrade $ cloudkitty-storage-init
It is time to define our rating rules. This implies to start the API. The API should be served as a WSGI app with a real server (see https://docs.openstack.org/cloudkitty/latest/install/mod_wsgi.html). But if you are on a testing environment, you may use the following command:
$ cloudkitty-api --port 8889 -- --config-dir /etc/cloudkitty/
Step 3: Profit!
In order to define our rating rules, we will use CloudKitty’s Hashmap module (https://docs.openstack.org/cloudkitty/latest/admin/rating/hashmap.html):
# Enable the hashmap module $ cloudkitty module-enable -n hashmap +---------+---------+----------+ | Module | Enabled | Priority | +---------+---------+----------+ | hashmap | True | 1 | +---------+---------+----------+ # Create the image service $ cloudkitty hashmap-service-create -n image +------------+--------------------------------------+ | Property | Value | +------------+--------------------------------------+ | name | image | | service_id | dfcf1764-b76f-45a1-b671-339cfcc65abf | +------------+--------------------------------------+ # We charge every stored image 0.002 per MiB per collect period (1 hour). # This is a global rating rule, but you may specify a tenant if you want to # apply different rates between your tenants $ cloudkitty hashmap-mapping-create -s dfcf1764-b76f-45a1-b671-339cfcc65abf -t flat -c 0.002 +------------+--------------------------------------+ | Property | Value | +------------+--------------------------------------+ | cost | 0.00200000 | | field_id | None | | group_id | None | | mapping_id | e47704bf-831a-459c-88b0-ec10c8ac9c21 | | service_id | dfcf1764-b76f-45a1-b671-339cfcc65abf | | tenant_id | None | | type | flat | | value | | +------------+--------------------------------------+ $ cloudkitty hashmap-service-create -n volume +------------+--------------------------------------+ | Property | Value | +------------+--------------------------------------+ | name | volume | | service_id | 67fcaa4d-ca36-42ff-bd70-82f860d298e3 | +------------+--------------------------------------+ # Volumes will be charged 0.001 per GiB per hour $ cloudkitty hashmap-mapping-create -s 67fcaa4d-ca36-42ff-bd70-82f860d298e3 -c 0.001 -t flat +------------+--------------------------------------+ | Property | Value | +------------+--------------------------------------+ | cost | 0.00100000 | | field_id | None | | group_id | None | | mapping_id | 72a65d37-89f8-4573-ba51-8980916dc258 | | service_id | 67fcaa4d-ca36-42ff-bd70-82f860d298e3 | | tenant_id | None | | type | flat | | value | | +------------+--------------------------------------+
All we need now is some data for the ‘myproject’ tenant. Create a few Cinder volumes, upload some images to Glance, and wait for Ceilometer to publish metrics to Monasca.
If you want immediate results, you may set the collector’s wait_period to 0, but this is not recommended in production (Monasca, Gnocchi, etc… need some time to aggregate measures so you could miss some).
You can now start CloudKitty’s processor:
$ cloudkitty-processor --config-file /etc/cloudkitty/cloudkitty.conf
The processor collects metrics starting at the beginning of the current month. Wait for a while until the processor has caught up with the current time.
Once you’ve waited, you can get the cost of your cloud:
# The following commands are ran as a user of the 'myproject' tenant # Get the total cost $ cloudkitty summary-get +----------------------------------+---------------+---------+---------------------+ | Tenant ID | Resource Type | Rate | Begin Time | End Time | +----------------------------------+---------------+---------+---------------------+ | 9a7[...]87 | ALL | 0.35468 | 2018-02-01T00:00:00 | 2018-03-01T00:00:00 | +----------------------------------+---------------+---------+---------------------+ # Get the total cost grouped by resource type $ cloudkitty summary-get -g res_type +----------------------------------+---------------+---------+---------------------+ | Tenant ID | Resource Type | Rate | Begin Time | End Time | +----------------------------------+---------------+---------+---------------------+ | 9a7[...]87 | image | 0.30368 | 2018-02-01T00:00:00 | 2018-03-01T00:00:00 | | 9a7[...]87 | volume | 0.051 | 2018-02-01T00:00:00 | 2018-03-01T00:00:00 | +----------------------------------+---------------+---------+---------------------+
Conclusion
It is now possible to integrate CloudKitty, Ceilometer and Monasca and to apply field mappings on any dimension specified in /etc/ceilometer/monasca_field_definitions.yml.
Given that CloudKitty services and metrics can be configured via YAML now, you can rate any metric published to Monasca by Ceilometer.
