scripts: ci: differentiate warnings from failures in compliance_check (5a1065c3) · Commits · Wenxi XU / zephyr

We have a few checks that report warnings, and for some of them it's actually the _only_ type of failure they report, so it can be confusing when the output of check_compliance.py says the "check failed" and lists all the things that are effectively warnings as "ERROR"s. This commit makes the output more clear by differentiating between warnings and actual errors, both when printing out individual failures, but also when printing the summary for each test case (ie. a test case is said to be "failing" only when it has at least one error, and is only showing up as having warnings if it only contains... warnings). Signed-off-by:

Benjamin Cabé <benjamin@zephyrproject.org>

scripts/ci/check_compliance.py

+15 −2

Changes for scripts/ci/check_compliance.py: 15 added lines, 2 removed lines.
Original line number	Diff line number	Diff line
		@@ -2288,6 +2288,7 @@ def _main(args):
		xml.write(args.output, pretty=True)

		failed_cases = []
		warning_cases = []
		name2doc = {testcase.name: testcase.doc
		for testcase in inheritors(ComplianceTest)}

		@@ -2296,19 +2297,31 @@ def _main(args):
		if case.is_skipped:
		logging.warning(f"Skipped {case.name}")
		else:
		if any(res.type in ('error', 'failure') for res in case.result):
		failed_cases.append(case)
		else:
		warning_cases.append(case)
		else:
		# Some checks can produce no .result
		logging.info(f"No JUnit result for {case.name}")

		n_fails = len(failed_cases)
		n_warnings = len(warning_cases)

		if n_fails or n_warnings:
		if n_fails:
		print(f"{n_fails} checks failed")
		for case in failed_cases:
		print(f"{n_fails} check(s) failed")
		if n_warnings:
		print(f"{n_warnings} check(s) with warnings only")

		for case in failed_cases + warning_cases:
		for res in case.result:
		errmsg = res.text.strip()
		if res.type in ('error', 'failure'):
		logging.error(f"Test {case.name} failed: \n{errmsg}")
		else:
		logging.warning(f"Test {case.name} warning: \n{errmsg}")

		if args.no_case_output:
		continue
		with open(f"{case.name}.txt", "w") as f:

Admin message