mm/slub.c: init_on_free=1 should wipe freelist ptr for bulk allocations (0f181f9f) · Commits · 戴 / test

slab_alloc_node() already zeroed out the freelist pointer if init_on_free was on. Thibaut Sautereau noticed that the same needs to be done for kmem_cache_alloc_bulk(), which performs the allocations separately. kmem_cache_alloc_bulk() is currently used in two places in the kernel, so this change is unlikely to have a major performance impact. SLAB doesn't require a similar change, as auto-initialization makes the allocator store the freelist pointers off-slab. Link: http://lkml.kernel.org/r/20191007091605.30530-1-glider@google.com Fixes: ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options") Signed-off-by:

Alexander Potapenko <glider@google.com> Reported-by:

Thibaut Sautereau <thibaut@sautereau.fr> Reported-by:

Kees Cook <keescook@chromium.org> Cc: Christoph Lameter <cl@linux.com> Cc: Laura Abbott <labbott@redhat.com> Signed-off-by:

Andrew Morton <akpm@linux-foundation.org> Signed-off-by:

Linus Torvalds <torvalds@linux-foundation.org>

mm/slub.c

+16 −6

Original line number	Diff line number	Diff line
		@@ -2671,6 +2671,17 @@ static void __slab_alloc(struct kmem_cache s, gfp_t gfpflags, int node,
		return p;
		}

		/*
		* If the object has been wiped upon free, make sure it's fully initialized by
		* zeroing out freelist pointer.
		*/
		static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s,
		void *obj)
		{
		if (unlikely(slab_want_init_on_free(s)) && obj)
		memset((void )((char )obj + s->offset), 0, sizeof(void *));
		}

		/*
		* Inlined fastpath so that allocation functions (kmalloc, kmem_cache_alloc)
		* have the fastpath folded into their functions. So no function call
		@@ -2759,12 +2770,8 @@ redo:
		prefetch_freepointer(s, next_object);
		stat(s, ALLOC_FASTPATH);
		}
		/*
		* If the object has been wiped upon free, make sure it's fully
		* initialized by zeroing out freelist pointer.
		*/
		if (unlikely(slab_want_init_on_free(s)) && object)
		memset(object + s->offset, 0, sizeof(void *));

		maybe_wipe_obj_freeptr(s, object);

		if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object)
		memset(object, 0, s->object_size);
		@@ -3178,10 +3185,13 @@ int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size,
		goto error;

		c = this_cpu_ptr(s->cpu_slab);
		maybe_wipe_obj_freeptr(s, p[i]);

		continue; /* goto for-loop */
		}
		c->freelist = get_freepointer(s, object);
		p[i] = object;
		maybe_wipe_obj_freeptr(s, p[i]);
		}
		c->tid = next_tid(c->tid);
		local_irq_enable();

Admin message