Rebase locking/kcsan to locking/urgent

   kasan

   ubsan

   kmemleak

   kcsan

   gdb-kernel-debugging

   kgdb

   kselftest

The Kernel Concurrency Sanitizer (KCSAN)

========================================

The Kernel Concurrency Sanitizer (KCSAN) is a dynamic race detector, which

relies on compile-time instrumentation, and uses a watchpoint-based sampling

approach to detect races. KCSAN's primary purpose is to detect `data races`_.

Usage

-----

KCSAN is supported in both GCC and Clang. With GCC it requires version 7.3.0 or

later. With Clang it requires version 7.0.0 or later.

To enable KCSAN configure the kernel with::

    CONFIG_KCSAN = y

KCSAN provides several other configuration options to customize behaviour (see

the respective help text in ``lib/Kconfig.kcsan`` for more info).

Error reports

~~~~~~~~~~~~~

A typical data race report looks like this::

    ==================================================================

    BUG: KCSAN: data-race in generic_permission / kernfs_refresh_inode

    write to 0xffff8fee4c40700c of 4 bytes by task 175 on cpu 4:

     kernfs_refresh_inode+0x70/0x170

     kernfs_iop_permission+0x4f/0x90

     inode_permission+0x190/0x200

     link_path_walk.part.0+0x503/0x8e0

     path_lookupat.isra.0+0x69/0x4d0

     filename_lookup+0x136/0x280

     user_path_at_empty+0x47/0x60

     vfs_statx+0x9b/0x130

     __do_sys_newlstat+0x50/0xb0

     __x64_sys_newlstat+0x37/0x50

     do_syscall_64+0x85/0x260

     entry_SYSCALL_64_after_hwframe+0x44/0xa9

    read to 0xffff8fee4c40700c of 4 bytes by task 166 on cpu 6:

     generic_permission+0x5b/0x2a0

     kernfs_iop_permission+0x66/0x90

     inode_permission+0x190/0x200

     link_path_walk.part.0+0x503/0x8e0

     path_lookupat.isra.0+0x69/0x4d0

     filename_lookup+0x136/0x280

     user_path_at_empty+0x47/0x60

     do_faccessat+0x11a/0x390

     __x64_sys_access+0x3c/0x50

     do_syscall_64+0x85/0x260

     entry_SYSCALL_64_after_hwframe+0x44/0xa9

    Reported by Kernel Concurrency Sanitizer on:

    CPU: 6 PID: 166 Comm: systemd-journal Not tainted 5.3.0-rc7+ #1

    Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014

    ==================================================================

The header of the report provides a short summary of the functions involved in

the race. It is followed by the access types and stack traces of the 2 threads

involved in the data race.

The other less common type of data race report looks like this::

    ==================================================================

    BUG: KCSAN: data-race in e1000_clean_rx_irq+0x551/0xb10

    race at unknown origin, with read to 0xffff933db8a2ae6c of 1 bytes by interrupt on cpu 0:

     e1000_clean_rx_irq+0x551/0xb10

     e1000_clean+0x533/0xda0

     net_rx_action+0x329/0x900

     __do_softirq+0xdb/0x2db

     irq_exit+0x9b/0xa0

     do_IRQ+0x9c/0xf0

     ret_from_intr+0x0/0x18

     default_idle+0x3f/0x220

     arch_cpu_idle+0x21/0x30

     do_idle+0x1df/0x230

     cpu_startup_entry+0x14/0x20

     rest_init+0xc5/0xcb

     arch_call_rest_init+0x13/0x2b

     start_kernel+0x6db/0x700

    Reported by Kernel Concurrency Sanitizer on:

    CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.3.0-rc7+ #2

    Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014

    ==================================================================

This report is generated where it was not possible to determine the other

racing thread, but a race was inferred due to the data value of the watched

memory location having changed. These can occur either due to missing

instrumentation or e.g. DMA accesses. These reports will only be generated if

``CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN=y`` (selected by default).

Selective analysis

~~~~~~~~~~~~~~~~~~

It may be desirable to disable data race detection for specific accesses,

functions, compilation units, or entire subsystems.  For static blacklisting,

the below options are available:

* KCSAN understands the ``data_race(expr)`` annotation, which tells KCSAN that

  any data races due to accesses in ``expr`` should be ignored and resulting

  behaviour when encountering a data race is deemed safe.

* Disabling data race detection for entire functions can be accomplished by

  using the function attribute ``__no_kcsan``::

    __no_kcsan

    void foo(void) {

        ...

  To dynamically limit for which functions to generate reports, see the

  `DebugFS interface`_ blacklist/whitelist feature.

  For ``__always_inline`` functions, replace ``__always_inline`` with

  ``__no_kcsan_or_inline`` (which implies ``__always_inline``)::

    static __no_kcsan_or_inline void foo(void) {

        ...

  Note: Older compiler versions (GCC < 9) also do not always honor the

  ``__no_kcsan`` attribute on regular ``inline`` functions. If false positives

  with these compilers cannot be tolerated, for small functions where

  ``__always_inline`` would be appropriate, ``__no_kcsan_or_inline`` should be

  preferred instead.

* To disable data race detection for a particular compilation unit, add to the

  ``Makefile``::

    KCSAN_SANITIZE_file.o := n

* To disable data race detection for all compilation units listed in a

  ``Makefile``, add to the respective ``Makefile``::

    KCSAN_SANITIZE := n

Furthermore, it is possible to tell KCSAN to show or hide entire classes of

data races, depending on preferences. These can be changed via the following

Kconfig options:

* ``CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY``: If enabled and a conflicting write

  is observed via a watchpoint, but the data value of the memory location was

  observed to remain unchanged, do not report the data race.

* ``CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC``: Assume that plain aligned writes

  up to word size are atomic by default. Assumes that such writes are not

  subject to unsafe compiler optimizations resulting in data races. The option

  causes KCSAN to not report data races due to conflicts where the only plain

  accesses are aligned writes up to word size.

DebugFS interface

~~~~~~~~~~~~~~~~~

The file ``/sys/kernel/debug/kcsan`` provides the following interface:

* Reading ``/sys/kernel/debug/kcsan`` returns various runtime statistics.

* Writing ``on`` or ``off`` to ``/sys/kernel/debug/kcsan`` allows turning KCSAN

  on or off, respectively.

* Writing ``!some_func_name`` to ``/sys/kernel/debug/kcsan`` adds

  ``some_func_name`` to the report filter list, which (by default) blacklists

  reporting data races where either one of the top stackframes are a function

  in the list.

* Writing either ``blacklist`` or ``whitelist`` to ``/sys/kernel/debug/kcsan``

  changes the report filtering behaviour. For example, the blacklist feature

  can be used to silence frequently occurring data races; the whitelist feature

  can help with reproduction and testing of fixes.

Tuning performance

~~~~~~~~~~~~~~~~~~

Core parameters that affect KCSAN's overall performance and bug detection

ability are exposed as kernel command-line arguments whose defaults can also be

changed via the corresponding Kconfig options.

* ``kcsan.skip_watch`` (``CONFIG_KCSAN_SKIP_WATCH``): Number of per-CPU memory

  operations to skip, before another watchpoint is set up. Setting up

  watchpoints more frequently will result in the likelihood of races to be

  observed to increase. This parameter has the most significant impact on

  overall system performance and race detection ability.

* ``kcsan.udelay_task`` (``CONFIG_KCSAN_UDELAY_TASK``): For tasks, the

  microsecond delay to stall execution after a watchpoint has been set up.

  Larger values result in the window in which we may observe a race to

  increase.

* ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For

  interrupts, the microsecond delay to stall execution after a watchpoint has

  been set up. Interrupts have tighter latency requirements, and their delay

  should generally be smaller than the one chosen for tasks.

They may be tweaked at runtime via ``/sys/module/kcsan/parameters/``.

Data Races

----------

In an execution, two memory accesses form a *data race* if they *conflict*,

they happen concurrently in different threads, and at least one of them is a

*plain access*; they *conflict* if both access the same memory location, and at

least one is a write. For a more thorough discussion and definition, see `"Plain

Accesses and Data Races" in the LKMM`_.

.. _"Plain Accesses and Data Races" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/explanation.txt#n1922

Relationship with the Linux-Kernel Memory Consistency Model (LKMM)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The LKMM defines the propagation and ordering rules of various memory

operations, which gives developers the ability to reason about concurrent code.

Ultimately this allows to determine the possible executions of concurrent code,

and if that code is free from data races.

KCSAN is aware of *marked atomic operations* (``READ_ONCE``, ``WRITE_ONCE``,

``atomic_*``, etc.), but is oblivious of any ordering guarantees and simply

assumes that memory barriers are placed correctly. In other words, KCSAN

assumes that as long as a plain access is not observed to race with another

conflicting access, memory operations are correctly ordered.

This means that KCSAN will not report *potential* data races due to missing

memory ordering. Developers should therefore carefully consider the required

memory ordering requirements that remain unchecked. If, however, missing

memory ordering (that is observable with a particular compiler and

architecture) leads to an observable data race (e.g. entering a critical

section erroneously), KCSAN would report the resulting data race.

Race Detection Beyond Data Races

--------------------------------

For code with complex concurrency design, race-condition bugs may not always

manifest as data races. Race conditions occur if concurrently executing

operations result in unexpected system behaviour. On the other hand, data races

are defined at the C-language level. The following macros can be used to check

properties of concurrent code where bugs would not manifest as data races.

.. kernel-doc:: include/linux/kcsan-checks.h

    :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_WRITER_SCOPED

                ASSERT_EXCLUSIVE_ACCESS ASSERT_EXCLUSIVE_ACCESS_SCOPED

                ASSERT_EXCLUSIVE_BITS

Implementation Details

----------------------

KCSAN relies on observing that two accesses happen concurrently. Crucially, we

want to (a) increase the chances of observing races (especially for races that

manifest rarely), and (b) be able to actually observe them. We can accomplish

(a) by injecting various delays, and (b) by using address watchpoints (or

breakpoints).

If we deliberately stall a memory access, while we have a watchpoint for its

address set up, and then observe the watchpoint to fire, two accesses to the

same address just raced. Using hardware watchpoints, this is the approach taken

in `DataCollider

<http://usenix.org/legacy/events/osdi10/tech/full_papers/Erickson.pdf>`_.

Unlike DataCollider, KCSAN does not use hardware watchpoints, but instead

relies on compiler instrumentation and "soft watchpoints".

In KCSAN, watchpoints are implemented using an efficient encoding that stores

access type, size, and address in a long; the benefits of using "soft

watchpoints" are portability and greater flexibility. KCSAN then relies on the

compiler instrumenting plain accesses. For each instrumented plain access:

1. Check if a matching watchpoint exists; if yes, and at least one access is a

   write, then we encountered a racing access.

2. Periodically, if no matching watchpoint exists, set up a watchpoint and

   stall for a small randomized delay.

3. Also check the data value before the delay, and re-check the data value

   after delay; if the values mismatch, we infer a race of unknown origin.

To detect data races between plain and marked accesses, KCSAN also annotates

marked accesses, but only to check if a watchpoint exists; i.e. KCSAN never

sets up a watchpoint on marked accesses. By never setting up watchpoints for

marked operations, if all accesses to a variable that is accessed concurrently

are properly marked, KCSAN will never trigger a watchpoint and therefore never

report the accesses.

Key Properties

~~~~~~~~~~~~~~

1. **Memory Overhead:**  The overall memory overhead is only a few MiB

   depending on configuration. The current implementation uses a small array of

   longs to encode watchpoint information, which is negligible.

2. **Performance Overhead:** KCSAN's runtime aims to be minimal, using an

   efficient watchpoint encoding that does not require acquiring any shared

   locks in the fast-path. For kernel boot on a system with 8 CPUs:

   - 5.0x slow-down with the default KCSAN config;

   - 2.8x slow-down from runtime fast-path overhead only (set very large

     ``KCSAN_SKIP_WATCH`` and unset ``KCSAN_SKIP_WATCH_RANDOMIZE``).

3. **Annotation Overheads:** Minimal annotations are required outside the KCSAN

   runtime. As a result, maintenance overheads are minimal as the kernel

   evolves.

4. **Detects Racy Writes from Devices:** Due to checking data values upon

   setting up watchpoints, racy writes from devices can also be detected.

5. **Memory Ordering:** KCSAN is *not* explicitly aware of the LKMM's ordering

   rules; this may result in missed data races (false negatives).

6. **Analysis Accuracy:** For observed executions, due to using a sampling

   strategy, the analysis is *unsound* (false negatives possible), but aims to

   be complete (no false positives).

Alternatives Considered

-----------------------

An alternative data race detection approach for the kernel can be found in the

`Kernel Thread Sanitizer (KTSAN) <https://github.com/google/ktsan/wiki>`_.

KTSAN is a happens-before data race detector, which explicitly establishes the

happens-before order between memory operations, which can then be used to

determine data races as defined in `Data Races`_.

To build a correct happens-before relation, KTSAN must be aware of all ordering

rules of the LKMM and synchronization primitives. Unfortunately, any omission

leads to large numbers of false positives, which is especially detrimental in

the context of the kernel which includes numerous custom synchronization

mechanisms. To track the happens-before relation, KTSAN's implementation

requires metadata for each memory location (shadow memory), which for each page

corresponds to 4 pages of shadow memory, and can translate into overhead of

tens of GiB on a large system.

F:	scripts/Kconfig.include

F:	scripts/kconfig/

KCSAN

M:	Marco Elver <elver@google.com>

R:	Dmitry Vyukov <dvyukov@google.com>

L:	kasan-dev@googlegroups.com

S:	Maintained

F:	Documentation/dev-tools/kcsan.rst

F:	include/linux/kcsan*.h

F:	kernel/kcsan/

F:	lib/Kconfig.kcsan

F:	scripts/Makefile.kcsan

KDUMP

M:	Dave Young <dyoung@redhat.com>

M:	Baoquan He <bhe@redhat.com>

export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS

export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE

export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN

export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN CFLAGS_KCSAN

export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE

export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE

export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL

include scripts/Makefile.kasan

include scripts/Makefile.extrawarn

include scripts/Makefile.ubsan

include scripts/Makefile.kcsan

# Add user supplied CPPFLAGS, AFLAGS and CFLAGS as the last assignments

KBUILD_CPPFLAGS += $(KCPPFLAGS)

	select THREAD_INFO_IN_TASK

	select USER_STACKTRACE_SUPPORT

	select VIRT_TO_BUS

	select HAVE_ARCH_KCSAN			if X86_64

	select X86_FEATURE_NAMES		if PROC_FS

	select PROC_PID_ARCH_STATUS		if PROC_FS

	imply IMA_SECURE_AND_OR_TRUSTED_BOOT    if EFI