Commit fda497e5 authored Jan 16, 2019 by Vakul Garg Committed by David S. Miller Jan 17, 2019

Optimize sk_msg_clone() by data merge to end dst sg entry

Function sk_msg_clone has been modified to merge the data from source sg
entry to destination sg entry if the cloned data resides in same page
and is contiguous to the end entry of destination sk_msg. This improves
kernel tls throughput to the tune of 10%.

When the user space tls application calls sendmsg() with MSG_MORE, it leads
to calling sk_msg_clone() with new data being cloned placed continuous to
previously cloned data. Without this optimization, a new SG entry in
the destination sk_msg i.e. rec->msg_plaintext in tls_clone_plaintext_msg()
gets used. This leads to exhaustion of sg entries in rec->msg_plaintext
even before a full 16K of allowable record data is accumulated. Hence we
lose oppurtunity to encrypt and send a full 16K record.

With this patch, the kernel tls can accumulate full 16K of record data
irrespective of the size of data passed in sendmsg() with MSG_MORE.

Signed-off-by: Vakul Garg <vakul.garg@nxp.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

parent 4559dd24

net/core/skmsg.c

+17 −8

Original line number	Diff line number	Diff line
		@@ -78,11 +78,9 @@ int sk_msg_clone(struct sock sk, struct sk_msg dst, struct sk_msg *src,
		{
		int i = src->sg.start;
		struct scatterlist *sge = sk_msg_elem(src, i);
		struct scatterlist *sgd = NULL;
		u32 sge_len, sge_off;

		if (sk_msg_full(dst))
		return -ENOSPC;

		while (off) {
		if (sge->length > off)
		break;
		@@ -94,16 +92,27 @@ int sk_msg_clone(struct sock sk, struct sk_msg dst, struct sk_msg *src,
		}

		while (len) {
		if (sk_msg_full(dst))
		return -ENOSPC;

		sge_len = sge->length - off;
		sge_off = sge->offset + off;
		if (sge_len > len)
		sge_len = len;

		if (dst->sg.end)
		sgd = sk_msg_elem(dst, dst->sg.end - 1);

		if (sgd &&
		(sg_page(sge) == sg_page(sgd)) &&
		(sg_virt(sge) + off == sg_virt(sgd) + sgd->length)) {
		sgd->length += sge_len;
		dst->sg.size += sge_len;
		} else if (!sk_msg_full(dst)) {
		sge_off = sge->offset + off;
		sk_msg_page_add(dst, sg_page(sge), sge_len, sge_off);
		} else {
		return -ENOSPC;
		}

		off = 0;
		len -= sge_len;
		sk_msg_page_add(dst, sg_page(sge), sge_len, sge_off);
		sk_mem_charge(sk, sge_len);
		sk_msg_iter_var_next(i);
		if (i == src->sg.end && len)

Admin message