doc: Rework Development Process section

.. _api_lifecycle:

API Lifecycle

#############

Developers using Zephyr's APIs need to know how long they can trust that a

given API will not change in future releases. At the same time, developers

maintaining and extending Zephyr's APIs need to be able to introduce

new APIs that aren't yet fully proven, and to potentially retire old APIs when they're

no longer optimal or supported by the underlying platforms.

.. figure:: api_lifecycle.png

    :align: center

    :alt: API Life Cycle

    :figclass: align-center

    API Life Cycle

Experimental

*************

Experimental APIs denote that a feature was introduced recently, and may change

or be removed in future versions. Try it out and provide feedback

to the community via the `Developer mailing list <https://lists.zephyrproject.org/g/devel>`_.

Peripheral APIs (Hardware Related)

==================================

When introducing an API (public header file with documentation) for a new

peripheral or driver subsystem, review of the API is enforced and is driven by

the API working group consisting of representatives from different vendors.

The following requirements apply to new APIs:

- Documentation of the API (usage)

  explaining its design and assumptions, how it is to be used, current

  implementation limitations, and future potential, if appropriate.

- The API introduction should be accompanied by at least one driver

  implementing this API

- At least one sample using the new API

The API shall be promoted to ``unstable`` when it has at least two

implementations on different hardware platforms.

Hardware Agnostic APIs

=======================

For hardware agnostic APIs, multiple applications are required to promote an

API from ``experimental`` to ``unstable``.

Unstable

********

The API is in the process of settling, but has not yet had sufficient real-world

testing to be considered stable. The API is considered generic in nature and can

be used on different hardware platforms.

.. note::

   Changes will not be announced.

Stable

*******

The API has proven satisfactory, but cleanup in the underlying code may cause

minor changes. Backwards-compatibility will be maintained if reasonable.

An API can be declared ``stable`` after fullfilling the following requirements:

- Test cases for the new API with 100% coverage

- Complete documentation in code. All public interfaces shall be documented

  and available in online documentation.

- The API has been in-use and was available in at least 2 development releases

- Stable APIs can get backward compatible updates, bug fixes and security fixes

  at any time.

.. note::

   Incompatible changes will be announced in the release notes.

Deprecated

***********

.. note::

   Unstable APIs can be removed without deprecation at any time.

The following are the requirements for deprecating an existing API:

- Deprecation Time (stable APIs): 2 Releases

  The API needs to be marked as deprecated in at least two full releases.

  For example, if an API was first deprecated in release 1.14,

  it will be ready to be removed in 1.16 at the earliest.

  There may be special circumstances, determined by the API working group,

  where an API is deprecated sooner.

- What is required when deprecating:

  - Mark as deprecated

  - Document the deprecation

  - All usage in Zephyr for deprecated API needs to be modified

  - Change needs to be atomic and bisectable

  - Code using the deprecated API needs to be modified to remove usage of said

    API

During the deprecation waiting period, the API will be in the ``deprecated``

state. The Zephyr maintainers will track usage of deprecated APIs on

``docs.zephyrproject.org`` and support developers migrating their code. Zephyr

will continue to provide warnings:

- API documentation will inform users that the API is deprecated.

- Attempts to use a deprecated API at build time will log a warning to the

  console.

Retired

*******

In this phase, the API is removed.

The target removal date is 2 releases after deprecation is announced.

The Zephyr maintainers will decide when to actually remove the API: this

will depend on how many developers have successfully migrated from the

deprecated API, and on how urgently the API needs to be removed.

If it's OK to remove the API, it will be removed. The maintainers will remove

the corresponding documentation, and communicate the removal in the usual ways:

the release notes, mailing lists, Github issues and pull-requests.

If it's not OK to remove the API, the maintainers will continue to support

migration and aim to remove the API in the next release.

Code Flow and Branches

######################

Introduction

************

The zephyr Git repository has three types of branches:

master

  Which contains the latest state of development

topic-\*

  Topic branches that are used for shared development of a new feature

vx.y-branch

  Branches which track maintenance releases based on a major

  release

Development in topic branches before features go to mainline allows teams to

work independently on a subsystem or a feature, improves efficiency and

turnaround time, and encourages collaboration and streamlines communication

between developers.

Changes submitted to a development topic branch can evolve and improve

incrementally in a branch, before they are submitted to the mainline tree for

final integration.

By dedicating an isolated branch to complex features, it’s

possible to initiate in-depth discussions around new additions before

integrating them into the official project.

Roles and Responsibilities

**************************

Development topic branch owners have the following responsibilities:

- Use the infrastructure and tools provided by the project (GitHub, Git)

- Review changes coming from team members and request review from branch owners

  when submitting changes.

- Keep the branch in sync with upstream and update on a regular basis.

- Push changes frequently to upstream using the following methods:

  - GitHub pull requests: for example, when reviews have not been done in the local

    branch (one-man branch).

  - Merge requests: When a set of changes has been done in a local branch and

    has been reviewed and tested in a topic branch.

Communication and Collaboration

################################

The `Zephyr project mailing lists <https://lists.zephyrproject.org/g/main/subgroups>`_ are used as the primary

communication tool by project members, contributors, and the community. The

mailing list is open for topics related to the project and should be used for

collaboration among team members working on the same feature or subsystem or for

discussing project direction and daily development of the code base. In general,

bug reports and issues should be entered and tracked in the bug tracking system

(`GitHub Issues <https://github.com/zephyrproject-rtos/zephyr/issues>`_) and not broadcasted to the mailing list, the same applies to

code reviews. Code should be submitted to GitHub using the appropriate tools.

Code Documentation

###################

API Documentation

******************

Well documented APIs enhance the experience for developers and are an essential

requirement for defining an API's success. Doxygen is a general purpose

documentation tool that the zephyr project uses for documenting APIs. It

generates either an on-line documentation browser (in HTML) and/or providess

input for other tools that is used to generate a reference manual from

documented source files. In particular, doxygen's XML output is used as an input

when producing the Zephyr project's online documentation.

Reference to Requirements

**************************

APIs for the most part document the implementation of requirements or advertised

features and can be traced back to features. We use the API documentation as the

main interface to trace implementation back to documented features. This is done

using custom _doxygen_ tags that reference requirements maintained somewhere

else in a requirement catalogue.

Test Documentation

*******************

To help understand what each test does and which functionality it tests we also

document all test code using the same tools and in the same context and generate

documentation for all unit and integration tests maintained in the same

environment. Tests are documented using references to the APIs or functionality

they validate by creating a link back to the APIs and by adding a reference to

the original resquirements.

Documentation Guidelines

*************************

Test Code

**********

The Zephyr project uses several test methodologies, the most common being the

:ref:`Ztest framework <testing>`. The documentation of tests should only be done

on the entry test functions (usually prefixed with test\_) and those that are

called directly by the Ztest framework. Those tests are going to appear in test

reports and using their name and identifier is the best way to identify them and

trace back to them to requirements.

Test documentation should not interfere with the actual API documentation and

needs to follow a new structure to avoid confusion. Using a consistent naming

scheme and following a well-defined structure we will be able to group this

documentation in its own module and identify it uniquely when parsing test data

for traceability reports. Here are a few guidelines to be followed:

- All test code documentation should be grouped under the ``all_tests`` doxygen

  group

- All test documentation should be under doxygen groups that are prefixed

  with tests\_

The doxygen ``@see`` directive is used to link features and APIs under test,

like so::

    /**

    * @brief Tests for the Semaphore kernel object

    * @defgroup kernel_semaphore_tests Semaphore

    * @ingroup all_tests

    * @{

    */

    ...

    /**

    * @brief A brief description of the tests

    * Some details about the test

    * more details

    *

    * @see k_sem_init(), #K_SEM_DEFINE(x)

    */

    void test_sema_thread2thread(void)

    {

    ...

    }

    ...

    /**

    * @}

    */