Commit 1a84d7fd authored by David S. Miller's avatar David S. Miller
Browse files

Merge branch 'mlxsw-Add-resource-scale-tests' 



Petr Machata says:

====================
mlxsw: Add resource scale tests

There are a number of tests that check features of the Linux networking
stack. By running them on suitable interfaces, one can exercise the
mlxsw offloading code. However none of these tests attempts to push
mlxsw to the limits supported by the ASIC.

As an additional wrinkle, the "limits supported by the ASIC" themselves
may not be a set of fixed numbers, but rather depend on a profile that
determines how the ASIC resources are allocated for different purposes.

This patchset introduces several tests that verify capability of mlxsw
to offload amounts of routes, flower rules, and mirroring sessions that
match predicted ASIC capacity, at different configuration profiles.
Additionally they verify that amounts exceeding the predicted capacity
can *not* be offloaded.

These are not generic tests, but ones that are tailored for mlxsw
specifically. For that reason they are not added to net/forwarding
selftests subdirectory, but rather to a newly-added drivers/net/mlxsw.

Patches #1, #2 and #3 tweak the generic forwarding/lib.sh to support the
new additions.

In patches #4 and #5, new libraries for interfacing with devlink are
introduced, first a generic one, then a Spectrum-specific one.

In patch #6, a devlink resource test is introduced.

Patches #7 and #8, #9 and #10, and #11 and #12 introduce three scale
tests: router, flower and mirror-to-gretap. The first of each pair of
patches introduces a generic portion of the test (mlxsw-specific), the
second introduces a Spectrum-specific wrapper.

Patch #13 then introduces a scale test driver that runs (possibly a
subset of) the tests introduced by patches from previous paragraph.
====================

Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parents a8477261 1b6130df

MAINTAINERS

+1 −0
Original line number Diff line number Diff line
@@ -9154,6 +9154,7 @@ S: Supported 
W:	http://www.mellanox.com

Q:	http://patchwork.ozlabs.org/project/netdev/list/

F:	drivers/net/ethernet/mellanox/mlxsw/

F:	tools/testing/selftests/drivers/net/mlxsw/



MELLANOX FIRMWARE FLASH LIBRARY (mlxfw)

M:	mlxsw@mellanox.com

tools/testing/selftests/drivers/net/mlxsw/mirror_gre_scale.sh

0 → 100644
+197 −0
Original line number Diff line number Diff line 
# SPDX-License-Identifier: GPL-2.0



# Test offloading a number of mirrors-to-gretap. The test creates a number of

# tunnels. Then it adds one flower mirror for each of the tunnels, matching a

# given host IP. Then it generates traffic at each of the host IPs and checks

# that the traffic has been mirrored at the appropriate tunnel.

#

#   +--------------------------+                   +--------------------------+

#   | H1                       |                   |                       H2 |

#   |     + $h1                |                   |                $h2 +     |

#   |     | 2001:db8:1:X::1/64 |                   | 2001:db8:1:X::2/64 |     |

#   +-----|--------------------+                   +--------------------|-----+

#         |                                                             |

#   +-----|-------------------------------------------------------------|-----+

#   | SW  o--> mirrors                                                  |     |

#   | +---|-------------------------------------------------------------|---+ |

#   | |   + $swp1                    BR                           $swp2 +   | |

#   | +---------------------------------------------------------------------+ |

#   |                                                                         |

#   |     + $swp3                          + gt6-<X> (ip6gretap)              |

#   |     | 2001:db8:2:X::1/64             : loc=2001:db8:2:X::1              |

#   |     |                                : rem=2001:db8:2:X::2              |

#   |     |                                : ttl=100                          |

#   |     |                                : tos=inherit                      |

#   |     |                                :                                  |

#   +-----|--------------------------------:----------------------------------+

#         |                                :

#   +-----|--------------------------------:----------------------------------+

#   | H3  + $h3                            + h3-gt6-<X> (ip6gretap)           |

#   |       2001:db8:2:X::2/64               loc=2001:db8:2:X::2              |

#   |                                        rem=2001:db8:2:X::1              |

#   |                                        ttl=100                          |

#   |                                        tos=inherit                      |

#   |                                                                         |

#   +-------------------------------------------------------------------------+



source ../../../../net/forwarding/mirror_lib.sh



MIRROR_NUM_NETIFS=6



mirror_gre_ipv6_addr()

{

	local net=$1; shift

	local num=$1; shift



	printf "2001:db8:%x:%x" $net $num

}



mirror_gre_tunnels_create()

{

	local count=$1; shift

	local should_fail=$1; shift



	MIRROR_GRE_BATCH_FILE="$(mktemp)"

	for ((i=0; i < count; ++i)); do

		local match_dip=$(mirror_gre_ipv6_addr 1 $i)::2

		local htun=h3-gt6-$i

		local tun=gt6-$i



		((mirror_gre_tunnels++))



		ip address add dev $h1 $(mirror_gre_ipv6_addr 1 $i)::1/64

		ip address add dev $h2 $(mirror_gre_ipv6_addr 1 $i)::2/64



		ip address add dev $swp3 $(mirror_gre_ipv6_addr 2 $i)::1/64

		ip address add dev $h3 $(mirror_gre_ipv6_addr 2 $i)::2/64



		tunnel_create $tun ip6gretap \

			      $(mirror_gre_ipv6_addr 2 $i)::1 \

			      $(mirror_gre_ipv6_addr 2 $i)::2 \

			      ttl 100 tos inherit allow-localremote



		tunnel_create $htun ip6gretap \

			      $(mirror_gre_ipv6_addr 2 $i)::2 \

			      $(mirror_gre_ipv6_addr 2 $i)::1

		ip link set $htun vrf v$h3

		matchall_sink_create $htun



		cat >> $MIRROR_GRE_BATCH_FILE <<-EOF

			filter add dev $swp1 ingress pref 1000 \

				protocol ipv6 \

				flower $tcflags dst_ip $match_dip \

				action mirred egress mirror dev $tun

		EOF

	done



	tc -b $MIRROR_GRE_BATCH_FILE

	check_err_fail $should_fail $? "Mirror rule insertion"

}



mirror_gre_tunnels_destroy()

{

	local count=$1; shift



	for ((i=0; i < count; ++i)); do

		local htun=h3-gt6-$i

		local tun=gt6-$i



		ip address del dev $h3 $(mirror_gre_ipv6_addr 2 $i)::2/64

		ip address del dev $swp3 $(mirror_gre_ipv6_addr 2 $i)::1/64



		ip address del dev $h2 $(mirror_gre_ipv6_addr 1 $i)::2/64

		ip address del dev $h1 $(mirror_gre_ipv6_addr 1 $i)::1/64



		tunnel_destroy $htun

		tunnel_destroy $tun

	done

}



__mirror_gre_test()

{

	local count=$1; shift

	local should_fail=$1; shift



	mirror_gre_tunnels_create $count $should_fail

	if ((should_fail)); then

	    return

	fi



	sleep 5



	for ((i = 0; i < count; ++i)); do

		local dip=$(mirror_gre_ipv6_addr 1 $i)::2

		local htun=h3-gt6-$i

		local message



		icmp6_capture_install $htun

		mirror_test v$h1 "" $dip $htun 100 10

		icmp6_capture_uninstall $htun

	done

}



mirror_gre_test()

{

	local count=$1; shift

	local should_fail=$1; shift



	if ! tc_offload_check $TC_FLOWER_NUM_NETIFS; then

		check_err 1 "Could not test offloaded functionality"

		return

	fi



	tcflags="skip_sw"

	__mirror_gre_test $count $should_fail

}



mirror_gre_setup_prepare()

{

	h1=${NETIFS[p1]}

	swp1=${NETIFS[p2]}



	swp2=${NETIFS[p3]}

	h2=${NETIFS[p4]}



	swp3=${NETIFS[p5]}

	h3=${NETIFS[p6]}



	mirror_gre_tunnels=0



	vrf_prepare



	simple_if_init $h1

	simple_if_init $h2

	simple_if_init $h3



	ip link add name br1 type bridge vlan_filtering 1

	ip link set dev br1 up



	ip link set dev $swp1 master br1

	ip link set dev $swp1 up

	tc qdisc add dev $swp1 clsact



	ip link set dev $swp2 master br1

	ip link set dev $swp2 up



	ip link set dev $swp3 up

}



mirror_gre_cleanup()

{

	mirror_gre_tunnels_destroy $mirror_gre_tunnels



	ip link set dev $swp3 down



	ip link set dev $swp2 down



	tc qdisc del dev $swp1 clsact

	ip link set dev $swp1 down



	ip link del dev br1



	simple_if_fini $h3

	simple_if_fini $h2

	simple_if_fini $h1



	vrf_cleanup

}

tools/testing/selftests/drivers/net/mlxsw/router_scale.sh

0 → 100644
+167 −0
Original line number Diff line number Diff line 
#!/bin/bash

# SPDX-License-Identifier: GPL-2.0



ROUTER_NUM_NETIFS=4



router_h1_create()

{

	simple_if_init $h1 192.0.1.1/24

	ip route add 193.0.0.0/8 via 192.0.1.2 dev $h1

}



router_h1_destroy()

{

	ip route del 193.0.0.0/8 via 192.0.1.2 dev $h1

	simple_if_fini $h1 192.0.1.1/24

}



router_h2_create()

{

	simple_if_init $h2 192.0.2.1/24

	tc qdisc add dev $h2 handle ffff: ingress

}



router_h2_destroy()

{

	tc qdisc del dev $h2 handle ffff: ingress

	simple_if_fini $h2 192.0.2.1/24

}



router_create()

{

	ip link set dev $rp1 up

	ip link set dev $rp2 up



	ip address add 192.0.1.2/24 dev $rp1

	ip address add 192.0.2.2/24 dev $rp2

}



router_destroy()

{

	ip address del 192.0.2.2/24 dev $rp2

	ip address del 192.0.1.2/24 dev $rp1



	ip link set dev $rp2 down

	ip link set dev $rp1 down

}



router_setup_prepare()

{

	h1=${NETIFS[p1]}

	rp1=${NETIFS[p2]}



	rp2=${NETIFS[p3]}

	h2=${NETIFS[p4]}



	h1mac=$(mac_get $h1)

	rp1mac=$(mac_get $rp1)



	vrf_prepare



	router_h1_create

	router_h2_create



	router_create

}



router_offload_validate()

{

	local route_count=$1

	local offloaded_count



	offloaded_count=$(ip route | grep -o 'offload' | wc -l)

	[[ $offloaded_count -ge $route_count ]]

}



router_routes_create()

{

	local route_count=$1

	local count=0



	ROUTE_FILE="$(mktemp)"



	for i in {0..255}

	do

		for j in {0..255}

		do

			for k in {0..255}

			do

				if [[ $count -eq $route_count ]]; then

					break 3

				fi



				echo route add 193.${i}.${j}.${k}/32 via \

				       192.0.2.1 dev $rp2  >> $ROUTE_FILE

				((count++))

			done

		done

	done



	ip -b $ROUTE_FILE &> /dev/null

}



router_routes_destroy()

{

	if [[ -v ROUTE_FILE ]]; then

		rm -f $ROUTE_FILE

	fi

}



router_test()

{

	local route_count=$1

	local should_fail=$2

	local count=0



	RET=0



	router_routes_create $route_count



	router_offload_validate $route_count

	check_err_fail $should_fail $? "Offload of $route_count routes"

	if [[ $RET -ne 0 ]] || [[ $should_fail -eq 1 ]]; then

		return

	fi



	tc filter add dev $h2 ingress protocol ip pref 1 flower \

		skip_sw dst_ip 193.0.0.0/8 action drop



	for i in {0..255}

	do

		for j in {0..255}

		do

			for k in {0..255}

			do

				if [[ $count -eq $route_count ]]; then

					break 3

				fi



				$MZ $h1 -c 1 -p 64 -a $h1mac -b $rp1mac \

					-A 192.0.1.1 -B 193.${i}.${j}.${k} \

					-t ip -q

				((count++))

			done

		done

	done



	tc_check_packets "dev $h2 ingress" 1 $route_count

	check_err $? "Offload mismatch"



	tc filter del dev $h2 ingress protocol ip pref 1 flower \

		skip_sw dst_ip 193.0.0.0/8 action drop



	router_routes_destroy

}



router_cleanup()

{

	pre_cleanup



	router_routes_destroy

	router_destroy



	router_h2_destroy

	router_h1_destroy



	vrf_cleanup

}

tools/testing/selftests/drivers/net/mlxsw/spectrum/devlink_lib_spectrum.sh

0 → 100644
+119 −0
Original line number Diff line number Diff line 
#!/bin/bash

# SPDX-License-Identifier: GPL-2.0



source "../../../../net/forwarding/devlink_lib.sh"



if [ "$DEVLINK_VIDDID" != "15b3:cb84" ]; then

	echo "SKIP: test is tailored for Mellanox Spectrum"

	exit 1

fi



# Needed for returning to default

declare -A KVD_DEFAULTS



KVD_CHILDREN="linear hash_single hash_double"

KVDL_CHILDREN="singles chunks large_chunks"



devlink_sp_resource_minimize()

{

	local size

	local i



	for i in $KVD_CHILDREN; do

		size=$(devlink_resource_get kvd "$i" | jq '.["size_min"]')

		devlink_resource_size_set "$size" kvd "$i"

	done



	for i in $KVDL_CHILDREN; do

		size=$(devlink_resource_get kvd linear "$i" | \

		       jq '.["size_min"]')

		devlink_resource_size_set "$size" kvd linear "$i"

	done

}



devlink_sp_size_kvd_to_default()

{

	local need_reload=0

	local i



	for i in $KVD_CHILDREN; do

		local size=$(echo "${KVD_DEFAULTS[kvd_$i]}" | jq '.["size"]')

		current_size=$(devlink_resource_size_get kvd "$i")



		if [ "$size" -ne "$current_size" ]; then

			devlink_resource_size_set "$size" kvd "$i"

			need_reload=1

		fi

	done



	for i in $KVDL_CHILDREN; do

		local size=$(echo "${KVD_DEFAULTS[kvd_linear_$i]}" | \

			     jq '.["size"]')

		current_size=$(devlink_resource_size_get kvd linear "$i")



		if [ "$size" -ne "$current_size" ]; then

			devlink_resource_size_set "$size" kvd linear "$i"

			need_reload=1

		fi

	done



	if [ "$need_reload" -ne "0" ]; then

		devlink_reload

	fi

}



devlink_sp_read_kvd_defaults()

{

	local key

	local i



	KVD_DEFAULTS[kvd]=$(devlink_resource_get "kvd")

	for i in $KVD_CHILDREN; do

		key=kvd_$i

		KVD_DEFAULTS[$key]=$(devlink_resource_get kvd "$i")

	done



	for i in $KVDL_CHILDREN; do

		key=kvd_linear_$i

		KVD_DEFAULTS[$key]=$(devlink_resource_get kvd linear "$i")

	done

}



KVD_PROFILES="default scale ipv4_max"



devlink_sp_resource_kvd_profile_set()

{

	local profile=$1



	case "$profile" in

	scale)

		devlink_resource_size_set 64000 kvd linear

		devlink_resource_size_set 15616 kvd linear singles

		devlink_resource_size_set 32000 kvd linear chunks

		devlink_resource_size_set 16384 kvd linear large_chunks

		devlink_resource_size_set 128000 kvd hash_single

		devlink_resource_size_set 48000 kvd hash_double

		devlink_reload

		;;

	ipv4_max)

		devlink_resource_size_set 64000 kvd linear

		devlink_resource_size_set 15616 kvd linear singles

		devlink_resource_size_set 32000 kvd linear chunks

		devlink_resource_size_set 16384 kvd linear large_chunks

		devlink_resource_size_set 144000 kvd hash_single

		devlink_resource_size_set 32768 kvd hash_double

		devlink_reload

		;;

	default)

		devlink_resource_size_set 98304 kvd linear

		devlink_resource_size_set 16384 kvd linear singles

		devlink_resource_size_set 49152 kvd linear chunks

		devlink_resource_size_set 32768 kvd linear large_chunks

		devlink_resource_size_set 87040 kvd hash_single

		devlink_resource_size_set 60416 kvd hash_double

		devlink_reload

		;;

	*)

		check_err 1 "Unknown profile $profile"

	esac

}

tools/testing/selftests/drivers/net/mlxsw/spectrum/devlink_resources.sh

0 → 100755
+117 −0
Original line number Diff line number Diff line 
#!/bin/bash

# SPDX-License-Identifier: GPL-2.0



NUM_NETIFS=1

source devlink_lib_spectrum.sh



setup_prepare()

{

	devlink_sp_read_kvd_defaults

}



cleanup()

{

	pre_cleanup

	devlink_sp_size_kvd_to_default

}



trap cleanup EXIT



setup_prepare



profiles_test()

{

	local i



	log_info "Running profile tests"



	for i in $KVD_PROFILES; do

		RET=0

		devlink_sp_resource_kvd_profile_set $i

		log_test "'$i' profile"

	done



	# Default is explicitly tested at end to ensure it's actually applied

	RET=0

	devlink_sp_resource_kvd_profile_set "default"

	log_test "'default' profile"

}



resources_min_test()

{

	local size

	local i

	local j



	log_info "Running KVD-minimum tests"



	for i in $KVD_CHILDREN; do

		RET=0

		size=$(devlink_resource_get kvd "$i" | jq '.["size_min"]')

		devlink_resource_size_set "$size" kvd "$i"



		# In case of linear, need to minimize sub-resources as well

		if [[ "$i" == "linear" ]]; then

			for j in $KVDL_CHILDREN; do

				devlink_resource_size_set 0 kvd linear "$j"

			done

		fi



		devlink_reload

		devlink_sp_size_kvd_to_default

		log_test "'$i' minimize [$size]"

	done

}



resources_max_test()

{

	local min_size

	local size

	local i

	local j



	log_info "Running KVD-maximum tests"

	for i in $KVD_CHILDREN; do

		RET=0

		devlink_sp_resource_minimize



		# Calculate the maximum possible size for the given partition

		size=$(devlink_resource_size_get kvd)

		for j in $KVD_CHILDREN; do

			if [ "$i" != "$j" ]; then

				min_size=$(devlink_resource_get kvd "$j" | \

					   jq '.["size_min"]')

				size=$((size - min_size))

			fi

		done



		# Test almost maximum size

		devlink_resource_size_set "$((size - 128))" kvd "$i"

		devlink_reload

		log_test "'$i' almost maximize [$((size - 128))]"



		# Test above maximum size

		devlink resource set "$DEVLINK_DEV" \

			path "kvd/$i" size $((size + 128)) &> /dev/null

		check_fail $? "Set kvd/$i to size $((size + 128)) should fail"

		log_test "'$i' Overflow rejection [$((size + 128))]"



		# Test maximum size

		if [ "$i" == "hash_single" ] || [ "$i" == "hash_double" ]; then

			echo "SKIP: Observed problem with exact max $i"

			continue

		fi



		devlink_resource_size_set "$size" kvd "$i"

		devlink_reload

		log_test "'$i' maximize [$size]"



		devlink_sp_size_kvd_to_default

	done

}



profiles_test

resources_min_test

resources_max_test



exit "$RET"

CRA Git | Maintained and supported by SUSTech CRA and CCSE