twister: ztest: short test case names on --no-detailed-test-id

a software program to ensure it meets certain requirements. The Test Cases in a

Test Suite are either related or meant to be executed together.

The name of each Test Scenario needs to be unique in the context of the overall

test application and has to follow basic rules:

Test Scenario, Test Suite, and Test Case names must follow to these basic rules:

#. The format of the Test Scenario identifier shall be a string without any spaces or

   special characters (allowed characters: alphanumeric and [\_=]) consisting

   subsection names delimited with a dot (``.``). For example, a test scenario

   that covers semaphores in the kernel shall start with ``kernel.semaphore``.

#. All Test Scenario identifiers within a ``testcase.yaml`` file need to be unique.

#. All Test Scenario identifiers within a Test Configuration (``testcase.yaml`` file)

   need to be unique.

   For example a ``testcase.yaml`` file covering semaphores in the kernel can have:

   * ``kernel.semaphore``: For general semaphore tests

     Test Case name, for example: ``debug.coredump.logging_backend``.

The ``--no-detailed-test-id`` command line option modifies the above rules in this way:

#. A Test Suite name has only ``<Test Scenario identifier>`` component.

   Its Application Project path can be found in ``twister.json`` report as ``path:`` property.

#. With short Test Suite names in this mode, all corresponding Test Scenario names

   must be unique for the Twister execution scope.

#. **Ztest** Test Case names have only Ztest components ``<Ztest suite name>.<Ztest test name>``.

   Its parent Test Suite name equals to the corresponding Test Scenario identifier.

The following is an example test configuration with a few options that are

explained in this document.

  them can use the :zephyr_file:`scripts/utils/twister_to_list.py` script to

  automatically migrate Twister configuration files.

* Twister

  * Test Case names for Ztest now include Ztest suite name, so the resulting identifier has

    three sections and looks like: ``<test_scenario_name>.<ztest_suite_name>.<ztest_name>``.

    These extended identifiers are used in log output, twister.json and testplan.json,

    as well as for ``--sub-test`` command line parameters (:github:`80088`).

  * The ``--no-detailed-test-id`` command line option also shortens Ztest Test Case names excluding

    its Test Scenario name prefix which is the same as the parent Test Suite id (:github:`82302`).

Drivers and Sensors

*******************

    test_plan_report_xor.add_argument("--list-tests", action="store_true",

                             help="""List of all sub-test functions recursively found in

        all --testsuite-root arguments. Note different sub-tests can share

        the same test scenario identifier (section.subsection)

        and come from different directories.

        The output is flattened and reports --sub-test names only,

        not their directories. For instance net.socket.getaddrinfo_ok

        and net.socket.fd_set belong to different directories.

        all --testsuite-root arguments. The output is flattened and reports detailed

        sub-test names without their directories.

        Note: sub-test names can share the same test scenario identifier prefix

        (section.subsection) even if they are from different test projects.

        """)

    test_plan_report_xor.add_argument("--test-tree", action="store_true",

        functions. Sub-tests are named by:

        'section.subsection_in_testcase_yaml.ztest_suite.ztest_without_test_prefix'.

        Example_1: 'kernel.fifo.fifo_api_1cpu.fifo_loop' where 'kernel.fifo' is a test scenario

        name (section.subsection) and 'fifo_api_1cpu.fifo_loop' is

        a Ztest suite_name.test_name identificator.

        name (section.subsection) and 'fifo_api_1cpu.fifo_loop' is a Ztest 'suite_name.test_name'.

        Example_2: 'debug.coredump.logging_backend' is a standalone test scenario name.

        Note: This selection mechanism works only for Ztest suite and test function names in

        the source files which are not generated by macro-substitutions.

        Note: With --no-detailed-test-id use only Ztest names without scenario name.

        """)

    parser.add_argument(

    parser.add_argument(

        '--detailed-test-id', action='store_true',

        help="Include paths to tests' locations in tests' names. Names will follow "

             "PATH_TO_TEST/SCENARIO_NAME schema "

             "e.g. samples/hello_world/sample.basic.helloworld")

        help="Compose each test Suite name from its configuration path (relative to root) and "

             "the appropriate Scenario name using PATH_TO_TEST_CONFIG/SCENARIO_NAME schema. "

             "Also (for Ztest only), prefix each test Case name with its Scenario name. "

             "For example: 'kernel.common.timing' Scenario with test Suite name "

             "'tests/kernel/sleep/kernel.common.timing' and 'kernel.common.timing.sleep.usleep' "

             "test Case (where 'sleep' is its Ztest suite name and 'usleep' is Ztest test name.")

    parser.add_argument(

        "--no-detailed-test-id", dest='detailed_test_id', action="store_false",

        help="Don't put paths into tests' names. "

             "With this arg a test name will be a scenario name "

             "e.g. sample.basic.helloworld.")

        help="Don't prefix each test Suite name with its configuration path, "

             "so it is the same as the appropriate Scenario name. "

             "Also (for Ztest only), don't prefix each Ztest Case name with its Scenario name. "

             "For example: 'kernel.common.timing' Scenario name, the same Suite name, "

             "and 'sleep.usleep' test Case (where 'sleep' is its Ztest suite name "

             "and 'usleep' is Ztest test name.")

    # Include paths in names by default.

    parser.set_defaults(detailed_test_id=True)

        for ts_name_ in ts_names:

            if self.started_suites[ts_name_]['count'] < (0 if phase == 'TS_SUM' else 1):

                continue

            tc_fq_id = f"{self.id}.{ts_name_}.{tc_name}"

            tc_fq_id = self.instance.compose_case_name(f"{ts_name_}.{tc_name}")

            if tc := self.instance.get_case_by_name(tc_fq_id):

                if self.trace:

                    logger.debug(f"On {phase}: Ztest case '{tc_name}' matched to '{tc_fq_id}")

            f"On {phase}: Ztest case '{tc_name}' is not known"

            f" in {self.started_suites} running suite(s)."

        )

        tc_id = f"{self.id}.{tc_name}"

        tc_id = self.instance.compose_case_name(tc_name)

        return self.instance.get_case_or_create(tc_id)

    def start_suite(self, suite_name):

        return symbol_name

    def determine_testcases(self, results):

        yaml_testsuite_name = self.instance.testsuite.id

        logger.debug(f"Determine test cases for test suite: {yaml_testsuite_name}")

        logger.debug(f"Determine test cases for test suite: {self.instance.testsuite.id}")

        logger.debug(

            f"Test instance {self.instance.name} already has {len(self.instance.testcases)} cases."

        )

        new_ztest_unit_test_regex = re.compile(r"z_ztest_unit_test__([^\s]+?)__([^\s]*)")

        detected_cases = []

                                f"not present in: {self.instance.testsuite.ztest_suite_names}"

                            )

                        test_func_name = m_[2].replace("test_", "", 1)

                        testcase_id = f"{yaml_testsuite_name}.{new_ztest_suite}.{test_func_name}"

                        testcase_id = self.instance.compose_case_name(

                            f"{new_ztest_suite}.{test_func_name}"

                        )

                        detected_cases.append(testcase_id)

        logger.debug(

            f"Test instance {self.instance.name} already has {len(self.instance.testcases)} cases."

        )

        if detected_cases:

            logger.debug(f"Detected Ztest cases: [{', '.join(detected_cases)}] in {elf_file}")

            tc_keeper = {

            self.instance.testcases.clear()

            self.instance.testsuite.testcases.clear()

            # When the old regex-based test case collection is fully deprecated,

            # this will be the sole place where test cases get added to the test instance.

            # Then we can further include the new_ztest_suite info in the testcase_id.

            for testcase_id in detected_cases:

                testcase = self.instance.add_testcase(name=testcase_id)

                self.instance.testsuite.add_testcase(name=testcase_id)

                # Keep previous statuses and reasons

                tc_info = tc_keeper.get(testcase_id, {})

                if not tc_info and self.trace:

                    # Also happens when Ztest uses macroses, eg. DEFINE_TEST_VARIANT

                    logger.debug(f"Ztest case '{testcase_id}' discovered for "

                                 f"'{self.instance.testsuite.source_dir_rel}' "

                                 f"with {list(tc_keeper)}")

                testcase.status = tc_info.get('status', TwisterStatus.NONE)

                testcase.reason = tc_info.get('reason')