asm-generic/atomic: Use __always_inline for fallback wrappers (765dcd20) · Commits · 戴 / test

Use __always_inline for atomic fallback wrappers. When building for size (CC_OPTIMIZE_FOR_SIZE), some compilers appear to be less inclined to inline even relatively small static inline functions that are assumed to be inlinable such as atomic ops. This can cause problems, for example in UACCESS regions. While the fallback wrappers aren't pure wrappers, they are trivial nonetheless, and the function they wrap should determine the final inlining policy. For x86 tinyconfig we observe: - vmlinux baseline: 1315988 - vmlinux with patch: 1315928 (-60 bytes) [ tglx: Cherry-picked from KCSAN ] Suggested-by:

Mark Rutland <mark.rutland@arm.com> Signed-off-by:

Marco Elver <elver@google.com> Acked-by:

Mark Rutland <mark.rutland@arm.com> Signed-off-by:

Paul E. McKenney <paulmck@kernel.org> Signed-off-by:

Thomas Gleixner <tglx@linutronix.de>

include/linux/atomic-fallback.h

+171 −169

File changed.

Preview size limit exceeded, changes collapsed.

scripts/atomic/fallbacks/acquire

+1 −1

Original line number	Diff line number	Diff line
		cat <<EOF
		static inline ${ret}
		static __always_inline ${ret}
		${atomic}_${pfx}${name}${sfx}_acquire(${params})
		{
		${ret} ret = ${atomic}_${pfx}${name}${sfx}_relaxed(${args});

scripts/atomic/fallbacks/add_negative

+1 −1

Original line number	Diff line number	Diff line
		@@ -8,7 +8,7 @@ cat <<EOF
		* if the result is negative, or false when
		* result is greater than or equal to zero.
		*/
		static inline bool
		static __always_inline bool
		${atomic}_add_negative(${int} i, ${atomic}_t *v)
		{
		return ${atomic}_add_return(i, v) < 0;

scripts/atomic/fallbacks/add_unless

+1 −1

Original line number	Diff line number	Diff line
		@@ -8,7 +8,7 @@ cat << EOF
		* Atomically adds @a to @v, if @v was not already @u.
		* Returns true if the addition was done.
		*/
		static inline bool
		static __always_inline bool
		${atomic}_add_unless(${atomic}_t *v, ${int} a, ${int} u)
		{
		return ${atomic}_fetch_add_unless(v, a, u) != u;

scripts/atomic/fallbacks/andnot

+1 −1

Original line number	Diff line number	Diff line
		cat <<EOF
		static inline ${ret}
		static __always_inline ${ret}
		${atomic}_${pfx}andnot${sfx}${order}(${int} i, ${atomic}_t *v)
		{
		${retstmt}${atomic}_${pfx}and${sfx}${order}(~i, v);

Admin message