cpufreq: Documentation: Updates based on current code

		    Dominik Brodowski  <linux@brodo.de>

		     David Kimdon <dwhedon@debian.org>

		Rafael J. Wysocki <rafael.j.wysocki@intel.com>

		   Viresh Kumar <viresh.kumar@linaro.org>

kernel "constant" loops_per_jiffy is updated on frequency changes

here.

Reference counting is done by cpufreq_get_cpu and cpufreq_put_cpu,

which make sure that the cpufreq processor driver is correctly

registered with the core, and will not be unloaded until

cpufreq_put_cpu is called.

Reference counting of the cpufreq policies is done by cpufreq_cpu_get

and cpufreq_cpu_put, which make sure that the cpufreq driver is

correctly registered with the core, and will not be unloaded until

cpufreq_put_cpu is called. That also ensures that the respective cpufreq

policy doesn't get freed while being used.

2. CPUFreq notifiers

====================

The phase is specified in the second argument to the notifier.

The third argument, a void *pointer, points to a struct cpufreq_policy

consisting of five values: cpu, min, max, policy and max_cpu_freq. min 

and max are the lower and upper frequencies (in kHz) of the new

policy, policy the new policy, cpu the number of the affected CPU; and 

max_cpu_freq the maximum supported CPU frequency. This value is given 

for informational purposes only.

consisting of several values, including min, max (the lower and upper

frequencies (in kHz) of the new policy).

2.2 CPUFreq transition notifiers

--------------------------------

These are notified twice when the CPUfreq driver switches the CPU core

frequency and this change has any external implications.

These are notified twice for each online CPU in the policy, when the

CPUfreq driver switches the CPU core frequency and this change has no

any external implications.

The second argument specifies the phase - CPUFREQ_PRECHANGE or

CPUFREQ_POSTCHANGE.

cpu	- number of the affected CPU

old	- old frequency

new	- new frequency

flags	- flags of the cpufreq driver

3. CPUFreq Table Generation with Operating Performance Point (OPP)

==================================================================

		    Dominik Brodowski  <linux@brodo.de>

		Rafael J. Wysocki <rafael.j.wysocki@intel.com>

		   Viresh Kumar <viresh.kumar@linaro.org>

What shall this struct cpufreq_driver contain? 

cpufreq_driver.name -		The name of this driver.

 .name - The name of this driver.

cpufreq_driver.init -		A pointer to the per-CPU initialization 

				function.

 .init - A pointer to the per-policy initialization function.

cpufreq_driver.verify -		A pointer to a "verification" function.

 .verify - A pointer to a "verification" function.

cpufreq_driver.setpolicy _or_ 

cpufreq_driver.target/

target_index		-	See below on the differences.

 .setpolicy _or_ .fast_switch _or_ .target _or_ .target_index - See

 below on the differences.

And optionally

cpufreq_driver.exit -		A pointer to a per-CPU cleanup

				function called during CPU_POST_DEAD

				phase of cpu hotplug process.

 .flags - Hints for the cpufreq core.

cpufreq_driver.stop_cpu -	A pointer to a per-CPU stop function

				called during CPU_DOWN_PREPARE phase of

				cpu hotplug process.

 .driver_data - cpufreq driver specific data.

cpufreq_driver.resume -		A pointer to a per-CPU resume function

				which is called with interrupts disabled

				and _before_ the pre-suspend frequency

				and/or policy is restored by a call to

				->target/target_index or ->setpolicy.

 .resolve_freq - Returns the most appropriate frequency for a target

 frequency. Doesn't change the frequency though.

cpufreq_driver.attr -		A pointer to a NULL-terminated list of

				"struct freq_attr" which allow to

				export values to sysfs.

 .get_intermediate and target_intermediate - Used to switch to stable

 frequency while changing CPU frequency.

cpufreq_driver.get_intermediate

and target_intermediate		Used to switch to stable frequency while

				changing CPU frequency.

 .get - Returns current frequency of the CPU.

 .bios_limit - Returns HW/BIOS max frequency limitations for the CPU.

 .exit - A pointer to a per-policy cleanup function called during

 CPU_POST_DEAD phase of cpu hotplug process.

 .stop_cpu - A pointer to a per-policy stop function called during

 CPU_DOWN_PREPARE phase of cpu hotplug process.

 .suspend - A pointer to a per-policy suspend function which is called

 with interrupts disabled and _after_ the governor is stopped for the

 policy.

 .resume - A pointer to a per-policy resume function which is called

 with interrupts disabled and _before_ the governor is started again.

 .ready - A pointer to a per-policy ready function which is called after

 the policy is fully initialized.

 .attr - A pointer to a NULL-terminated list of "struct freq_attr" which

 allow to export values to sysfs.

 .boost_enabled - If set, boost frequencies are enabled.

 .set_boost - A pointer to a per-policy function to enable/disable boost

 frequencies.

1.2 Per-CPU Initialization

--------------------------

Whenever a new CPU is registered with the device model, or after the

cpufreq driver registers itself, the per-CPU initialization function 

cpufreq_driver.init is called. It takes a struct cpufreq_policy

*policy as argument. What to do now?

cpufreq driver registers itself, the per-policy initialization function

cpufreq_driver.init is called if no cpufreq policy existed for the CPU.

Note that the .init() and .exit() routines are called only once for the

policy and not for each CPU managed by the policy. It takes a struct

cpufreq_policy *policy as argument. What to do now?

If necessary, activate the CPUfreq support on your CPU.

				cpufreq_driver.setpolicy or

				cpufreq_driver.target/target_index is called

				with these values.

policy->cpus			Update this with the masks of the

				(online + offline) CPUs that do DVFS

				along with this CPU (i.e.  that share

				clock/voltage rails with it).

For setting some of these values (cpuinfo.min[max]_freq, policy->min[max]), the

frequency table helpers might be helpful. See the section 2 for more information

on them.

SMP systems normally have same clock source for a group of cpus. For these the

.init() would be called only once for the first online cpu. Here the .init()

routine must initialize policy->cpus with mask of all possible cpus (Online +

Offline) that share the clock. Then the core would copy this mask onto

policy->related_cpus and will reset policy->cpus to carry only online cpus.

1.3 verify

------------

----------

When the user decides a new policy (consisting of

"policy,governor,min,max") shall be set, this policy must be validated

so that incompatible values can be corrected. For verifying these

values, a frequency table helper and/or the

cpufreq_verify_within_limits(struct cpufreq_policy *policy, unsigned

int min_freq, unsigned int max_freq) function might be helpful. See

section 2 for details on frequency table helpers.

values cpufreq_verify_within_limits(struct cpufreq_policy *policy,

unsigned int min_freq, unsigned int max_freq) function might be helpful.

See section 2 for details on frequency table helpers.

You need to make sure that at least one valid frequency (or operating

range) is within policy->min and policy->max. If necessary, increase

policy->max first, and only if this is no solution, decrease policy->min.

1.4 target/target_index or setpolicy?

----------------------------

1.4 target or target_index or setpolicy or fast_switch?

-------------------------------------------------------

Most cpufreq drivers or even most cpu frequency scaling algorithms 

only allow the CPU to be set to one frequency. For these, you use the

->target/target_index call.

only allow the CPU frequency to be set to predefined fixed values. For

these, you use the ->target(), ->target_index() or ->fast_switch()

callbacks.

Some cpufreq-capable processors switch the frequency between certain

limits on their own. These shall use the ->setpolicy call

Some cpufreq capable processors switch the frequency between certain

limits on their own. These shall use the ->setpolicy() callback.

1.5. target/target_index

-------------

------------------------

The target_index call has two arguments: struct cpufreq_policy *policy,

and unsigned int index (into the exposed frequency table).

Here again the frequency table helper might assist you - see section 2

for details.

1.6. fast_switch

----------------

1.6 setpolicy

---------------

This function is used for frequency switching from scheduler's context.

Not all drivers are expected to implement it, as sleeping from within

this callback isn't allowed. This callback must be highly optimized to

do switching as fast as possible.

This function has two arguments: struct cpufreq_policy *policy and

unsigned int target_frequency.

1.7 setpolicy

-------------

The setpolicy call only takes a struct cpufreq_policy *policy as

argument. You need to set the lower limit of the in-processor or

powersaving-oriented setting when CPUFREQ_POLICY_POWERSAVE. Also check

the reference implementation in drivers/cpufreq/longrun.c

1.7 get_intermediate and target_intermediate

1.8 get_intermediate and target_intermediate

--------------------------------------------

Only for drivers with target_index() and CPUFREQ_ASYNC_NOTIFICATION unset.

As most cpufreq processors only allow for being set to a few specific

frequencies, a "frequency table" with some functions might assist in

some work of the processor driver. Such a "frequency table" consists

of an array of struct cpufreq_frequency_table entries, with any value in

"driver_data" you want to use, and the corresponding frequency in

"frequency". At the end of the table, you need to add a

cpufreq_frequency_table entry with frequency set to CPUFREQ_TABLE_END. And

if you want to skip one entry in the table, set the frequency to 

CPUFREQ_ENTRY_INVALID. The entries don't need to be in ascending

order.

By calling cpufreq_table_validate_and_show(struct cpufreq_policy *policy,

					struct cpufreq_frequency_table *table);

the cpuinfo.min_freq and cpuinfo.max_freq values are detected, and

policy->min and policy->max are set to the same values. This is

helpful for the per-CPU initialization stage.

int cpufreq_frequency_table_verify(struct cpufreq_policy *policy,

                                   struct cpufreq_frequency_table *table);

assures that at least one valid frequency is within policy->min and

policy->max, and all other criteria are met. This is helpful for the

->verify call.

int cpufreq_frequency_table_target(struct cpufreq_policy *policy,

                                   unsigned int target_freq,

                                   unsigned int relation);

is the corresponding frequency table helper for the ->target

stage. Just pass the values to this function, and this function

returns the number of the frequency table entry which contains

the frequency the CPU shall be set to.

some work of the processor driver. Such a "frequency table" consists of

an array of struct cpufreq_frequency_table entries, with driver specific

values in "driver_data", the corresponding frequency in "frequency" and

flags set. At the end of the table, you need to add a

cpufreq_frequency_table entry with frequency set to CPUFREQ_TABLE_END.

And if you want to skip one entry in the table, set the frequency to

CPUFREQ_ENTRY_INVALID. The entries don't need to be in sorted in any

particular order, but if they are cpufreq core will do DVFS a bit

quickly for them as search for best match is faster.

By calling cpufreq_table_validate_and_show(), the cpuinfo.min_freq and

cpuinfo.max_freq values are detected, and policy->min and policy->max

are set to the same values. This is helpful for the per-CPU

initialization stage.

cpufreq_frequency_table_verify() assures that at least one valid

frequency is within policy->min and policy->max, and all other criteria

are met. This is helpful for the ->verify call.

cpufreq_frequency_table_target() is the corresponding frequency table

helper for the ->target stage. Just pass the values to this function,

and this function returns the of the frequency table entry which

contains the frequency the CPU shall be set to.

The following macros can be used as iterators over cpufreq_frequency_table:

cpufreq_for_each_entry(pos, table) - iterates over all entries of frequency

table.

cpufreq-for_each_valid_entry(pos, table) - iterates over all entries,

cpufreq_for_each_valid_entry(pos, table) - iterates over all entries,

excluding CPUFREQ_ENTRY_INVALID frequencies.

Use arguments "pos" - a cpufreq_frequency_table * as a loop cursor and

"table" - the cpufreq_frequency_table * you want to iterate over.

-  trans_table

All the statistics will be from the time the stats driver has been inserted

to the time when a read of a particular statistic is done. Obviously, stats 

driver will not have any information about the frequency transitions before

the stats driver insertion.

(or the time the stats were reset) to the time when a read of a particular

statistic is done. Obviously, stats driver will not have any information

about the frequency transitions before the stats driver insertion.

--------------------------------------------------------------------------------

<mysystem>:/sys/devices/system/cpu/cpu0/cpufreq/stats # ls -l

		CPU Frequency scaling  --->

			[*] CPU Frequency scaling

			[*]   CPU frequency translation statistics

			[*]     CPU frequency translation statistics details

"CPU Frequency scaling" (CONFIG_CPU_FREQ) should be enabled to configure

cpufreq-stats.

"CPU frequency translation statistics" (CONFIG_CPU_FREQ_STAT) provides the

basic statistics which includes time_in_state and total_trans.

statistics which includes time_in_state, total_trans and trans_table.

"CPU frequency translation statistics details" (CONFIG_CPU_FREQ_STAT_DETAILS)

provides fine grained cpufreq stats by trans_table. The reason for having a

separate config option for trans_table is:

- trans_table goes against the traditional /sysfs rule of one value per

  interface. It provides a whole bunch of value in a 2 dimensional matrix

  form.

Once these two options are enabled and your CPU supports cpufrequency, you

Once this option is enabled and your CPU supports cpufrequency, you

will be able to see the CPU frequency statistics in /sysfs.

		    Dominik Brodowski  <linux@brodo.de>

            some additions and corrections by Nico Golde <nico@ngolde.de>

		Rafael J. Wysocki <rafael.j.wysocki@intel.com>

		   Viresh Kumar <viresh.kumar@linaro.org>

2.3  Userspace

2.4  Ondemand

2.5  Conservative

2.6  Schedutil

3.   The Governor Interface in the CPUfreq Core

4.   References

1. What Is A CPUFreq Governor?

==============================

Most cpufreq drivers (except the intel_pstate and longrun) or even most

cpu frequency scaling algorithms only offer the CPU to be set to one

frequency. In order to offer dynamic frequency scaling, the cpufreq

core must be able to tell these drivers of a "target frequency". So

these specific drivers will be transformed to offer a "->target/target_index"

call instead of the existing "->setpolicy" call. For "longrun", all

stays the same, though.

cpu frequency scaling algorithms only allow the CPU frequency to be set

to predefined fixed values.  In order to offer dynamic frequency

scaling, the cpufreq core must be able to tell these drivers of a

"target frequency". So these specific drivers will be transformed to

offer a "->target/target_index/fast_switch()" call instead of the

"->setpolicy()" call. For set_policy drivers, all stays the same,

though.

How to decide what frequency within the CPUfreq policy should be used?

That's done using "cpufreq governors". Two are already in this patch

-- they're the already existing "powersave" and "performance" which

set the frequency statically to the lowest or highest frequency,

respectively. At least two more such governors will be ready for

addition in the near future, but likely many more as there are various

different theories and models about dynamic frequency scaling

around. Using such a generic interface as cpufreq offers to scaling

governors, these can be tested extensively, and the best one can be

selected for each specific use.

That's done using "cpufreq governors".

Basically, it's the following flow graph:

		    /			       the limits of policy->{min,max}

		   /			            \

		  /				     \

	Using the ->setpolicy call,		 Using the ->target/target_index call,

	Using the ->setpolicy call,		 Using the ->target/target_index/fast_switch call,

	    the limits and the			  the frequency closest

	     "policy" is set.			  to target_freq is set.

						  It is assured that it

2.4 Ondemand

------------

The CPUfreq governor "ondemand" sets the CPU depending on the

current usage. To do this the CPU must have the capability to

switch the frequency very quickly.

The CPUfreq governor "ondemand" sets the CPU frequency depending on the

current system load. Load estimation is triggered by the scheduler

through the update_util_data->func hook; when triggered, cpufreq checks

the CPU-usage statistics over the last period and the governor sets the

CPU accordingly.  The CPU must have the capability to switch the

frequency very quickly.

Sysfs files:

----------------

The CPUfreq governor "conservative", much like the "ondemand"

governor, sets the CPU depending on the current usage.  It differs in

behaviour in that it gracefully increases and decreases the CPU speed

rather than jumping to max speed the moment there is any load on the

CPU.  This behaviour more suitable in a battery powered environment.

The governor is tweaked in the same manner as the "ondemand" governor

through sysfs with the addition of:

governor, sets the CPU frequency depending on the current usage.  It

differs in behaviour in that it gracefully increases and decreases the

CPU speed rather than jumping to max speed the moment there is any load

on the CPU. This behaviour is more suitable in a battery powered

environment.  The governor is tweaked in the same manner as the

"ondemand" governor through sysfs with the addition of:

* freq_step:

  decision on when to decrease the frequency while running in any speed.

  Load for frequency increase is still evaluated every sampling rate.

2.6 Schedutil

-------------

The "schedutil" governor aims at better integration with the Linux

kernel scheduler.  Load estimation is achieved through the scheduler's

Per-Entity Load Tracking (PELT) mechanism, which also provides

information about the recent load [1].  This governor currently does

load based DVFS only for tasks managed by CFS. RT and DL scheduler tasks

are always run at the highest frequency.  Unlike all the other

governors, the code is located under the kernel/sched/ directory.

Sysfs files:

* rate_limit_us:

  This contains a value in microseconds. The governor waits for

  rate_limit_us time before reevaluating the load again, after it has

  evaluated the load once.

For an in-depth comparison with the other governors refer to [2].

3. The Governor Interface in the CPUfreq Core

=============================================

"cpufreq_register_governor". The struct cpufreq_governor, which has to

be passed to that function, must contain the following values:

governor->name -	    A unique name for this governor

governor->governor -	    The governor callback function

governor->owner	-	    .THIS_MODULE for the governor module (if 

			    appropriate)

The governor->governor callback is called with the current (or to-be-set)

cpufreq_policy struct for that CPU, and an unsigned int event. The

following events are currently defined:

CPUFREQ_GOV_START:   This governor shall start its duty for the CPU

		     policy->cpu

CPUFREQ_GOV_STOP:    This governor shall end its duty for the CPU

		     policy->cpu

CPUFREQ_GOV_LIMITS:  The limits for CPU policy->cpu have changed to

		     policy->min and policy->max.

If you need other "events" externally of your driver, _only_ use the

cpufreq_governor_l(unsigned int cpu, unsigned int event) call to the

CPUfreq core to ensure proper locking.

governor->name - A unique name for this governor.

governor->owner - .THIS_MODULE for the governor module (if appropriate).

plus a set of hooks to the functions implementing the governor's logic.

The CPUfreq governor may call the CPU processor driver using one of

these two functions:

                                   unsigned int relation);

target_freq must be within policy->min and policy->max, of course.

What's the difference between these two functions? When your governor

still is in a direct code path of a call to governor->governor, the

per-CPU cpufreq lock is still held in the cpufreq core, and there's

no need to lock it again (in fact, this would cause a deadlock). So

use __cpufreq_driver_target only in these cases. In all other cases 

(for example, when there's a "daemonized" function that wakes up 

every second), use cpufreq_driver_target to lock the cpufreq per-CPU

lock before the command is passed to the cpufreq processor driver.

What's the difference between these two functions? When your governor is

in a direct code path of a call to governor callbacks, like

governor->start(), the policy->rwsem is still held in the cpufreq core,

and there's no need to lock it again (in fact, this would cause a

deadlock). So use __cpufreq_driver_target only in these cases. In all

other cases (for example, when there's a "daemonized" function that

wakes up every second), use cpufreq_driver_target to take policy->rwsem

before the command is passed to the cpufreq driver.

4. References

=============

[1] Per-entity load tracking: https://lwn.net/Articles/531853/

[2] Improvements in CPU frequency management: https://lwn.net/Articles/682391/

Documents in this directory:

----------------------------

amd-powernow.txt -	AMD powernow driver specific file.

boost.txt -		Frequency boosting support.

core.txt	-	General description of the CPUFreq core and

			of CPUFreq notifiers

			of CPUFreq notifiers.

cpu-drivers.txt -	How to implement a new cpufreq processor driver.

cpu-drivers.txt -	How to implement a new cpufreq processor driver

cpufreq-nforce2.txt -	nVidia nForce2 platform specific file.

cpufreq-stats.txt -	General description of sysfs cpufreq stats.

governors.txt	-	What are cpufreq governors and how to

			implement them?

index.txt	-	File index, Mailing list and Links (this document)

intel-pstate.txt -	Intel pstate cpufreq driver specific file.

pcc-cpufreq.txt -	PCC cpufreq driver specific file.

user-guide.txt	-	User Guide to CPUFreq

------------

There is a CPU frequency changing CVS commit and general list where

you can report bugs, problems or submit patches. To post a message,

send an email to linux-pm@vger.kernel.org, to subscribe go to

http://vger.kernel.org/vger-lists.html#linux-pm and follow the

instructions there.

send an email to linux-pm@vger.kernel.org.

Links

-----

* http://cvs.arm.linux.org.uk/

the CPUFreq Mailing list:

* http://vger.kernel.org/vger-lists.html#cpufreq

* http://vger.kernel.org/vger-lists.html#linux-pm

Clock and voltage scaling for the SA-1100:

* http://www.lartmaker.nl/projects/scaling