blkio-controller.txt: Remove references to CFQ

Plan is to use the same cgroup based management interface for blkio controller

and based on user options switch IO policies in the background.

Currently two IO control policies are implemented. First one is proportional

weight time based division of disk policy. It is implemented in CFQ. Hence

this policy takes effect only on leaf nodes when CFQ is being used. The second

one is throttling policy which can be used to specify upper IO rate limits

on devices. This policy is implemented in generic block layer and can be

used on leaf nodes as well as higher level logical devices like device mapper.

One IO control policy is throttling policy which can be used to

specify upper IO rate limits on devices. This policy is implemented in

generic block layer and can be used on leaf nodes as well as higher

level logical devices like device mapper.

HOWTO

=====

Proportional Weight division of bandwidth

-----------------------------------------

You can do a very simple testing of running two dd threads in two different

cgroups. Here is what you can do.

- Enable Block IO controller

	CONFIG_BLK_CGROUP=y

- Enable group scheduling in CFQ

	CONFIG_CFQ_GROUP_IOSCHED=y

- Compile and boot into kernel and mount IO controller (blkio); see

  cgroups.txt, Why are cgroups needed?.

	mount -t tmpfs cgroup_root /sys/fs/cgroup

	mkdir /sys/fs/cgroup/blkio

	mount -t cgroup -o blkio none /sys/fs/cgroup/blkio

- Create two cgroups

	mkdir -p /sys/fs/cgroup/blkio/test1/ /sys/fs/cgroup/blkio/test2

- Set weights of group test1 and test2

	echo 1000 > /sys/fs/cgroup/blkio/test1/blkio.weight

	echo 500 > /sys/fs/cgroup/blkio/test2/blkio.weight

- Create two same size files (say 512MB each) on same disk (file1, file2) and

  launch two dd threads in different cgroup to read those files.

	sync

	echo 3 > /proc/sys/vm/drop_caches

	dd if=/mnt/sdb/zerofile1 of=/dev/null &

	echo $! > /sys/fs/cgroup/blkio/test1/tasks

	cat /sys/fs/cgroup/blkio/test1/tasks

	dd if=/mnt/sdb/zerofile2 of=/dev/null &

	echo $! > /sys/fs/cgroup/blkio/test2/tasks

	cat /sys/fs/cgroup/blkio/test2/tasks

- At macro level, first dd should finish first. To get more precise data, keep

  on looking at (with the help of script), at blkio.disk_time and

  blkio.disk_sectors files of both test1 and test2 groups. This will tell how

  much disk time (in milliseconds), each group got and how many sectors each

  group dispatched to the disk. We provide fairness in terms of disk time, so

  ideally io.disk_time of cgroups should be in proportion to the weight.

Throttling/Upper Limit policy

-----------------------------

- Enable Block IO controller

Hierarchical Cgroups

====================

Both CFQ and throttling implement hierarchy support; however,

Throttling implements hierarchy support; however,

throttling's hierarchy support is enabled iff "sane_behavior" is

enabled from cgroup side, which currently is a development option and

not publicly available.

			|

		     test3

CFQ by default and throttling with "sane_behavior" will handle the

hierarchy correctly.  For details on CFQ hierarchy support, refer to

Documentation/block/cfq-iosched.txt.  For throttling, all limits apply

Throttling with "sane_behavior" will handle the

hierarchy correctly. For throttling, all limits apply

to the whole subtree while all statistics are local to the IOs

directly generated by tasks in that cgroup.

	- Debug help. Right now some additional stats file show up in cgroup

	  if this option is enabled.

CONFIG_CFQ_GROUP_IOSCHED

	- Enables group scheduling in CFQ. Currently only 1 level of group

	  creation is allowed.

CONFIG_BLK_DEV_THROTTLING

	- Enable block device throttling support in block layer.

- blkio.reset_stats

	- Writing an int to this file will result in resetting all the stats

	  for that cgroup.

CFQ sysfs tunable

=================

/sys/block/<disk>/queue/iosched/slice_idle

------------------------------------------

On a faster hardware CFQ can be slow, especially with sequential workload.

This happens because CFQ idles on a single queue and single queue might not

drive deeper request queue depths to keep the storage busy. In such scenarios

one can try setting slice_idle=0 and that would switch CFQ to IOPS

(IO operations per second) mode on NCQ supporting hardware.

That means CFQ will not idle between cfq queues of a cfq group and hence be

able to driver higher queue depth and achieve better throughput. That also

means that cfq provides fairness among groups in terms of IOPS and not in

terms of disk time.

/sys/block/<disk>/queue/iosched/group_idle

------------------------------------------

If one disables idling on individual cfq queues and cfq service trees by

setting slice_idle=0, group_idle kicks in. That means CFQ will still idle

on the group in an attempt to provide fairness among groups.

By default group_idle is same as slice_idle and does not do anything if

slice_idle is enabled.

One can experience an overall throughput drop if you have created multiple

groups and put applications in that group which are not driving enough

IO to keep disk busy. In that case set group_idle=0, and CFQ will not idle

on individual groups and throughput should improve.