Commit 6ba8a3b1 authored by Nandita Dukkipati's avatar Nandita Dukkipati Committed by David S. Miller
Browse files

tcp: Tail loss probe (TLP) 

This patch series implement the Tail loss probe (TLP) algorithm described
in http://tools.ietf.org/html/draft-dukkipati-tcpm-tcp-loss-probe-01

. The
first patch implements the basic algorithm.

TLP's goal is to reduce tail latency of short transactions. It achieves
this by converting retransmission timeouts (RTOs) occuring due
to tail losses (losses at end of transactions) into fast recovery.
TLP transmits one packet in two round-trips when a connection is in
Open state and isn't receiving any ACKs. The transmitted packet, aka
loss probe, can be either new or a retransmission. When there is tail
loss, the ACK from a loss probe triggers FACK/early-retransmit based
fast recovery, thus avoiding a costly RTO. In the absence of loss,
there is no change in the connection state.

PTO stands for probe timeout. It is a timer event indicating
that an ACK is overdue and triggers a loss probe packet. The PTO value
is set to max(2*SRTT, 10ms) and is adjusted to account for delayed
ACK timer when there is only one oustanding packet.

TLP Algorithm

On transmission of new data in Open state:
  -> packets_out > 1: schedule PTO in max(2*SRTT, 10ms).
  -> packets_out == 1: schedule PTO in max(2*RTT, 1.5*RTT + 200ms)
  -> PTO = min(PTO, RTO)

Conditions for scheduling PTO:
  -> Connection is in Open state.
  -> Connection is either cwnd limited or no new data to send.
  -> Number of probes per tail loss episode is limited to one.
  -> Connection is SACK enabled.

When PTO fires:
  new_segment_exists:
    -> transmit new segment.
    -> packets_out++. cwnd remains same.

  no_new_packet:
    -> retransmit the last segment.
       Its ACK triggers FACK or early retransmit based recovery.

ACK path:
  -> rearm RTO at start of ACK processing.
  -> reschedule PTO if need be.

In addition, the patch includes a small variation to the Early Retransmit
(ER) algorithm, such that ER and TLP together can in principle recover any
N-degree of tail loss through fast recovery. TLP is controlled by the same
sysctl as ER, tcp_early_retrans sysctl.
tcp_early_retrans==0; disables TLP and ER.
		 ==1; enables RFC5827 ER.
		 ==2; delayed ER.
		 ==3; TLP and delayed ER. [DEFAULT]
		 ==4; TLP only.

The TLP patch series have been extensively tested on Google Web servers.
It is most effective for short Web trasactions, where it reduced RTOs by 15%
and improved HTTP response time (average by 6%, 99th percentile by 10%).
The transmitted probes account for <0.5% of the overall transmissions.

Signed-off-by: default avatarNandita Dukkipati <nanditad@google.com>
Acked-by: default avatarNeal Cardwell <ncardwell@google.com>
Acked-by: default avatarYuchung Cheng <ycheng@google.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parent 83e519b6

Documentation/networking/ip-sysctl.txt

+6 −2
Original line number Diff line number Diff line
@@ -190,7 +190,9 @@ tcp_early_retrans - INTEGER 
	Enable Early Retransmit (ER), per RFC 5827. ER lowers the threshold

	for triggering fast retransmit when the amount of outstanding data is

	small and when no previously unsent data can be transmitted (such

	that limited transmit could be used).

	that limited transmit could be used). Also controls the use of

	Tail loss probe (TLP) that converts RTOs occuring due to tail

	losses into fast recovery (draft-dukkipati-tcpm-tcp-loss-probe-01).

	Possible values:

		0 disables ER

		1 enables ER
@@ -198,7 +200,9 @@ tcp_early_retrans - INTEGER 
		  by a fourth of RTT. This mitigates connection falsely

		  recovers when network has a small degree of reordering

		  (less than 3 packets).

	Default: 2

		3 enables delayed ER and TLP.

		4 enables TLP only.

	Default: 3



tcp_ecn - INTEGER

	Control use of Explicit Congestion Notification (ECN) by TCP.

include/linux/tcp.h

+0 −1
Original line number Diff line number Diff line
@@ -201,7 +201,6 @@ struct tcp_sock { 
		unused      : 1;

	u8	repair_queue;

	u8	do_early_retrans:1,/* Enable RFC5827 early-retransmit  */

		early_retrans_delayed:1, /* Delayed ER timer installed */

		syn_data:1,	/* SYN includes data */

		syn_fastopen:1,	/* SYN includes Fast Open option */

		syn_data_acked:1;/* data in SYN is acked by SYN-ACK */

include/net/inet_connection_sock.h

+4 −1
Original line number Diff line number Diff line
@@ -133,6 +133,8 @@ struct inet_connection_sock { 
#define ICSK_TIME_RETRANS	1	/* Retransmit timer */

#define ICSK_TIME_DACK		2	/* Delayed ack timer */

#define ICSK_TIME_PROBE0	3	/* Zero window probe timer */

#define ICSK_TIME_EARLY_RETRANS 4	/* Early retransmit timer */

#define ICSK_TIME_LOSS_PROBE	5	/* Tail loss probe timer */



static inline struct inet_connection_sock *inet_csk(const struct sock *sk)

{
@@ -222,7 +224,8 @@ static inline void inet_csk_reset_xmit_timer(struct sock *sk, const int what, 
		when = max_when;

	}



	if (what == ICSK_TIME_RETRANS || what == ICSK_TIME_PROBE0) {

	if (what == ICSK_TIME_RETRANS || what == ICSK_TIME_PROBE0 ||

	    what == ICSK_TIME_EARLY_RETRANS || what ==  ICSK_TIME_LOSS_PROBE) {

		icsk->icsk_pending = what;

		icsk->icsk_timeout = jiffies + when;

		sk_reset_timer(sk, &icsk->icsk_retransmit_timer, icsk->icsk_timeout);

include/net/tcp.h

+4 −2
Original line number Diff line number Diff line
@@ -543,6 +543,8 @@ extern bool tcp_syn_flood_action(struct sock *sk, 
extern void tcp_push_one(struct sock *, unsigned int mss_now);

extern void tcp_send_ack(struct sock *sk);

extern void tcp_send_delayed_ack(struct sock *sk);

extern void tcp_send_loss_probe(struct sock *sk);

extern bool tcp_schedule_loss_probe(struct sock *sk);



/* tcp_input.c */

extern void tcp_cwnd_application_limited(struct sock *sk);
@@ -873,8 +875,8 @@ static inline void tcp_enable_fack(struct tcp_sock *tp) 
static inline void tcp_enable_early_retrans(struct tcp_sock *tp)

{

	tp->do_early_retrans = sysctl_tcp_early_retrans &&

		!sysctl_tcp_thin_dupack && sysctl_tcp_reordering == 3;

	tp->early_retrans_delayed = 0;

		sysctl_tcp_early_retrans < 4 && !sysctl_tcp_thin_dupack &&

		sysctl_tcp_reordering == 3;

}



static inline void tcp_disable_early_retrans(struct tcp_sock *tp)

include/uapi/linux/snmp.h

+1 −0
Original line number Diff line number Diff line
@@ -202,6 +202,7 @@ enum 
	LINUX_MIB_TCPFORWARDRETRANS,		/* TCPForwardRetrans */

	LINUX_MIB_TCPSLOWSTARTRETRANS,		/* TCPSlowStartRetrans */

	LINUX_MIB_TCPTIMEOUTS,			/* TCPTimeouts */

	LINUX_MIB_TCPLOSSPROBES,		/* TCPLossProbes */

	LINUX_MIB_TCPRENORECOVERYFAIL,		/* TCPRenoRecoveryFail */

	LINUX_MIB_TCPSACKRECOVERYFAIL,		/* TCPSackRecoveryFail */

	LINUX_MIB_TCPSCHEDULERFAILED,		/* TCPSchedulerFailed */

CRA Git | Maintained and supported by SUSTech CRA and CCSE