WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] [PATCH 9/9] blkio-cgroup-v12: The document of a cgroup suppo

The document of a cgroup support for dm-ioband.

Signed-off-by: Hirokazu Takahashi <taka@xxxxxxxxxxxxx>
Signed-off-by: Ryo Tsuruta <ryov@xxxxxxxxxxxxx>

---
 Documentation/cgroups/blkio.txt |  314 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 313 insertions(+), 1 deletion(-)

Index: linux-2.6.31/Documentation/cgroups/blkio.txt
===================================================================
--- linux-2.6.31.orig/Documentation/cgroups/blkio.txt
+++ linux-2.6.31/Documentation/cgroups/blkio.txt
@@ -11,6 +11,9 @@ I/O with a little enhancement.
 
 2. Setting up blkio-cgroup
 
+Note: If dm-ioband is to be used with blkio-cgroup, then the dm-ioband
+patch needs to be applied first.
+
 The following kernel config options are required.
 
 CONFIG_CGROUPS=y
@@ -43,7 +46,316 @@ determined by retrieving the ID number f
 the page cgroup is associated with the page which is involved in the
 I/O.
 
-4. Contact
+If the dm-ioband support patch was applied then the blkio.devices and
+blkio.settings files will also be present.
+
+4. Using dm-ioband and blkio-cgroup
+
+This section describes how to set up dm-ioband and blkio-cgroup in
+order to control bandwidth on a per cgroup per logical volume basis.
+The example used in this section assumes that there are two LVM volume
+groups on individual hard disks and two logical volumes on each volume
+group.
+
+                         Table. LVM configurations
+
+     --------------------------------------------------------------
+    |   LVM volume group   |  vg0 on /dev/sda  |  vg1 on /dev/sdb  |
+    |----------------------+-------------------+-------------------|
+    |  LVM logical volume  |   lv0   |   lv1   |   lv0   |   lv1   |
+     --------------------------------------------------------------
+
+4.1. Creating a dm-ioband logical device
+
+A dm-ioband logical device needs to be created and stacked on the
+device that is to bandwidth controlled. In this example the dm-ioband
+logical devices are stacked on each of the existing LVM logical
+volumes. By using the LVM facilities there is no need to unmount any
+logical volumes, even in the case of a volume being used as the root
+device. The following script is an example of how to stack and remove
+dm-ioband devices.
+
+==================== cut here (ioband.sh) ====================
+#!/bin/sh
+#
+# NOTE: You must run "ioband.sh stop" to restore the device-mapper
+# settings before changing logical volume settings, such as activate,
+# rename, resize and so on. These constraints would be eliminated by
+# enhancing LVM tools to support dm-ioband.
+
+logvols="vg0-lv0 vg0-lv1 vg1-lv0 vg1-lv1"
+
+start()
+{
+       for lv in $logvols; do
+               volgrp=${lv%%-*}
+               orig=${lv}-orig
+
+               # clone an existing logical volume.
+               /sbin/dmsetup table $lv | /sbin/dmsetup create $orig
+
+               # stack a dm-ioband device on the clone.
+               size=$(/sbin/blockdev --getsize /dev/mapper/$orig)
+               cat<<-EOM | /sbin/dmsetup load ${lv}
+       0 $size ioband /dev/mapper/${orig} ${volgrp} 0 0 cgroup weight 0 :100
+               EOM
+
+               # activate the new setting.
+               /sbin/dmsetup resume $lv
+       done
+}
+
+stop()
+{
+       for lv in $logvols; do
+               orig=${lv}-orig
+
+               # restore the original setting.
+               /sbin/dmsetup table $orig | /sbin/dmsetup load $lv
+
+               # activate the new setting.
+               /sbin/dmsetup resume $lv
+
+               # remove the clone.
+               /sbin/dmsetup remove $orig
+       done
+}
+
+case "$1" in
+       start)
+               start
+        ;;
+       stop)
+               stop
+        ;;
+esac
+exit 0
+==================== cut here (ioband.sh) ====================
+
+The following diagram shows how dm-ioband devices are stacked on and
+removed from the logical volumes.
+
+           Figure. stacking and removing dm-ioband devices
+
+                     run "ioband.sh start"
+                              ===>
+
+     -----------------------        -----------------------
+    |    lv0    |    lv1    |      |    lv0    |    lv1    |
+    |(dm-linear)|(dm-linear)|      |(dm-ioband)|(dm-ioband)|
+    |-----------------------|      |-----------+-----------|
+    |         vg0           |      | lv0-orig  | lv1-orig  |
+     -----------------------       |(dm-linear)|(dm-linear)|
+                                   |-----------------------|
+                                   |          vg0          |
+                                    -----------------------
+                              <===
+                      run "ioband.sh stop"
+
+After creating the dm-ioband devices, the settings can be observed by
+reading the blkio.devices file.
+
+# cat /cgroup/blkio.devices
+vg0 policy=weight io_throttle=4 io_limit=192 token=768 carryover=2
+  vg0-lv0
+  vg0-lv1
+vg1 policy=weight io_throttle=4 io_limit=192 token=768 carryover=2
+  vg1-lv0
+  vg1-lv1
+
+The first field in the first line is the symbolic name for an ioband
+device group, and the subsequent fields are settings for the ioband
+device group. The settings can be changed by writing to the
+blkio.devices, for example:
+
+# echo vg1 policy range-bw > /cgroup/blkio.devices
+
+Please refer to Document/device-mapper/ioband.txt which describes the
+details of the ioband device group settings.
+
+The second and the third indented lines "vg0-lv0" and "vg0-lv1" are
+the names of the dm-ioband devices that belong to the ioband device
+group. Typically, dm-ioband devices that reside on the same hard disk
+should belong to the same ioband device group in order to share the
+bandwidth of the hard disk.
+
+dm-ioband is not restricted to working with LVM, it may work in
+conjunction with any type of block device. Please refer to
+Documentation/device-mapper/ioband.txt for more details.
+
+4.2 Setting up dm-ioband through the blkio-cgroup interface
+
+The following table shows the given settings for this example. The
+bandwidth will be assigned on a per cgroup per logical volume basis.
+
+                   Table. Settings for each cgroup
+
+     --------------------------------------------------------------
+    |   LVM volume group   |  vg0 on /dev/sda  |  vg1 on /dev/sdb  |
+    |----------------------+-------------------+-------------------|
+    |  LVM logical volume  |   lv0   |   lv1   |   lv0   |   lv1   |
+    |----------------------+-------------------+-------------------|
+    |   bandwidth control  |     relative      |     absolute      |
+    |        policy        |      weight       |  bandwidth limit  |
+    |----------------------+-------------------+-------------------|
+    |         unit         |     weight [%]    | throughput [KB/s] |
+    |----------------------+-------------------+-------------------|
+    | settings for cgroup1 |    30   |    50   |   400   |   900   |
+    |----------------------+---------+---------+---------+---------|
+    | settings for cgroup2 |    60   |    20   |   200   |   600   |
+    |----------------------+---------+---------+---------+---------|
+    |    for root cgroup   |    70   |    30   |   100   |   300   |
+     --------------------------------------------------------------
+
+The set-up is described step-by-step below.
+
+1) Create new cgroups using the mkdir command
+
+# mkdir /cgroup/1
+# mkdir /cgroup/2
+
+2) Set bandwidth control policy on each ioband device group
+
+The set-up of bandwidth control policy is done by writing to
+blkio.devices file.
+
+# echo vg0 policy weight > /cgroup/blkio.devices
+# echo vg1 policy range-bw > /cgroup/blkio.devices
+
+3) Set up the root cgroup
+
+The root cgroup represents the default blkio-cgroup. If an I/O is
+performed by a process in a cgroup and the cgroup is not set up by
+blkio-cgroup, the I/O is charged to the root cgroup.
+
+The set-up of the root cgroup is done by writing to blkio.settings
+file in the cgroup's root directory. The following commands write
+the settings of each logical volume to that file.
+
+# echo vg0-lv0 70 > /cgroup/bklio.settings
+# echo vg0-lv1 30 > /cgroup/bklio.settings
+# echo vg1-lv0 100:100 > /cgroup/blkio.settings
+# echo vg1-lv1 300:300 > /cgroup/blkio.settings
+
+The settings can be verified by reading the blkio.settings file. The
+first field is the symbolic name for an ioband device group, and the
+second field is an ioband device name. The following example shows
+that vg0-lv0 and vg0-lv1 belong to the same ioband device group and
+share the bandwidth of sda according to their weights.
+
+# cat /cgroup/blkio.settings
+sda vg0-lv0 weight=70%
+sda vg0-lv1 weight=30%
+sdb vg1-lv0 range-bw=100:100
+sdb vg1-lv1 range-bw=300:300
+
+4) Set up cgroup1 and cgroup2
+
+New cgroups are set up in the same manner as the root cgroup.
+
+Settings for cgroup1
+# echo vg0-lv0 30 > /cgroup/1/blkio.settings
+# echo vg0-lv1 50 > /cgroup/1/bklio.settings
+# echo vg1-lv0 400:400 > /cgroup/1/blkio.settings
+# echo vg1-lv1 900:900 > /cgroup/1/bklio.settings
+
+Settings for cgroup2
+# echo vg0-lv0 60 > /cgroup/2/blkio.settings
+# echo vg0-lv1 20 > /cgroup/2/bklio.settings
+# echo vg1-lv0 200:200 > /cgroup/2/blkio.settings
+# echo vg1-lv1 600:600 > /cgroup/2/bklio.settings
+
+Again, the settings can be verified by reading the appropriate
+blkio.settings file.
+
+# cat /cgroup/1/blkio.settings
+vg0-lv0 weight=30%
+vg0-lv1 weight=50%
+vg1-lv0 range-bw=400:400
+vg1-lv1 range-bw=900:900
+
+If only the logical volume name is specified, the entry for the
+logical volume is removed.
+
+# echo vg0-lv1 > /cgroup/1/vlkio.setting
+# cat /cgroup/1/blkio.settings
+vg0-lv0 weight=30%
+vg0-lv1 weight=50%
+vg1-lv0 range-bw=400:400
+
+4.3 How bandwidth is distributed in the weight policy.
+
+The weight policy assigns bandwidth proportional to the weight of each
+cgroup in a hierarchical manner. The bandwidth assigned to a parent
+cgroup is distributed among the parent and its children according to
+their weight. For example, if there are two child cgroups under the
+parent cgroup, cgroup1 is assigned 60% of the parent bandwidth, and
+cgroup2 is assigned 30%, then 10% (100% - 60% + 30%) remains for the
+parent cgroup.
+
+        Figure. bandwidth distribution among a parent and children
+
+                    (100% - 30% - 60% = 10%)
+                            parent
+                           /      \
+                       cgroup1    cgroup2
+                        (30%)      (60%)
+
+The followings show how the bandwidth is calculated ans assigned to
+each cgroup in the given settings which are shown above.
+
+           Figure. hierarchical settings by the weight policy
+
+                (70%)  ---  /dev/sda ---  (30%) 
+
+               vg0/lv0                   vg0/lv1
+
+                (10%)                     (30%)
+             root(parent)              root(parent)
+              /      \                  /      \
+          cgroup1    cgroup2        cgroup1    cgroup2
+           (30%)      (60%)          (50%)      (20%)
+
+
+             Table. actual bandwidth assigned to each cgroup
+
+        ------------------------------------------------------------
+       |          |         | weight     | actual bandwidth         |
+       | shared   | logical | for a root | assigned to each cgroup  |
+       | device   | volume  | group      | against /dev/sda         |
+       |----------+---------+------------+--------------------------|
+       |          |         |            | parent   70% * 10% =  7% |
+       |          | vg0/lv0 |     70%    | cgroup1  70% * 30% = 21% |
+       |          |         |            | cgroup2  70% * 60% = 42% |  
+       | /dev/sda |---------+------------+--------------------------|
+       |          |         |            | parent   30% * 30% =  9% |
+       |          | vg1/lv1 |     30%    | cgroup1  30% * 50% = 15% |
+       |          |         |            | cgruop2  30% * 20% =  6% |
+        ------------------------------------------------------------
+
+4.4 Getting IO statistics per cgroup.
+
+The blkio.stats file provides IO statistics per dm-ioband per cgroup.
+This file consists of 12 fields separated by whitespace. The format is
+almost the same as /proc/diskstats and /sys/block/dev/stat files, but
+some fields are reserved for future use and they always return 0.
+
+Field #   Name            units         description
+-------   ----            -----         -----------
+1         device name                   name of dm-ioband device
+2         read I/Os       requests      number of read I/Os processed
+3                                       *reserved*
+4         read sectors    sectors       number of sectors read
+5                                       *reserved*
+6         write I/Os      requests      number of write I/Os processed
+7                                       *reserved*
+8         write sectors   sectors       number of sectors written
+9                                       *reserved*
+10        in_flight       requests      number of I/Os currently in flight
+11                                      *reserved*
+12                                      *reserved*
+
+5. Contact
 
 Linux Block I/O Bandwidth Control Project
 http://sourceforge.net/projects/ioband/

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel