xsk, net: Make sock_def_readable() have external linkage (43a825af) · Commits · 戴 / test

XDP sockets use the default implementation of struct sock's sk_data_ready callback, which is sock_def_readable(). This function is called in the XDP socket fast-path, and involves a retpoline. By letting sock_def_readable() have external linkage, and being called directly, the retpoline can be avoided. Signed-off-by:

Björn Töpel <bjorn.topel@intel.com> Signed-off-by:

Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20200120092917.13949-1-bjorn.topel@gmail.com

include/net/sock.h

+2 −0

Original line number	Diff line number	Diff line
		@@ -2612,4 +2612,6 @@ static inline bool sk_dev_equal_l3scope(struct sock *sk, int dif)
		return false;
		}

		void sock_def_readable(struct sock *sk);

		#endif /* _SOCK_H */

net/core/sock.c

+1 −1

Original line number	Diff line number	Diff line
		@@ -2786,7 +2786,7 @@ static void sock_def_error_report(struct sock *sk)
		rcu_read_unlock();
		}

		static void sock_def_readable(struct sock *sk)
		void sock_def_readable(struct sock *sk)
		{
		struct socket_wq *wq;

net/xdp/xsk.c

+1 −1

Original line number	Diff line number	Diff line
		@@ -217,7 +217,7 @@ static int xsk_rcv(struct xdp_sock xs, struct xdp_buff xdp)
		static void xsk_flush(struct xdp_sock *xs)
		{
		xskq_prod_submit(xs->rx);
		xs->sk.sk_data_ready(&xs->sk);
		sock_def_readable(&xs->sk);
		}

		int xsk_generic_rcv(struct xdp_sock xs, struct xdp_buff xdp)

Admin message