Theory of Operation

This chapter contains technical details about the operation of the CMSIS Toolbox.

Build Process Overview details the build process.
Adding a Toolchain to CMSIS-Toolbox contains details about the Compiler support.
CMake Integration describes the integration of CMake.
West Integration describes the integration of the West build system for Zephyr applications.
Generator Integration explains the integration of device configuration tools.
Debug Adapter Integration explains the integration of debuggers.

Build Process Overview

This section contains details of the build process. For a high-level description, refer to Overall Workflow.

Note

The tool options --verbose and --debug enable detailed output about the build process for analysis.
Refer to the chapter Build Tools for more information on the command line syntax.

`cbuild` Build Invocation

The cbuild utility controls the overall build process. It has two operating modes:

build mode generates the application and is the default command (no explicit command required).
setup mode generates the setup information for an IDE to populate dialogues, IntelliSense, and project outline views.

Details of the build mode

Without a specific cbuild command, the application is built. The typical invocation is:

cbuild <name>.csolution.yml [--packs] [--context-set] [--update-rte] [--frozen-packs]

It generates the application program that is described with the <name>.csolution.yml project.

When option --packs is used, it downloads missing software packs using cpackget.
It calls csolution to process the the <name>.csolution.yml project.
- With option --update-rte new configuration files from software packs are updated and the RTE_Components.h file is recreated.
- With option --context-set the file *.cbuild-set.yml specifies the context configuration of the application.
- With option --frozen-packs the file *.cbuild-pack.yml is used as input to issue an error when a pack version changes.
The output are build information files with all relevant project information for the build process.
The option --toolchain can be used to explicitly select a compiler.

Note

By default, the cbuild invocation does not update or change configuration files in the RTE Directory. Use the option --update-rte if this is required.

Details of the setup mode

The cbuild setup command prepares the data for an IDE environment. This command is called at the start of an IDE or whenever a csolution project file is modified. The typical invocation is:

cbuild setup <name>.csolution.yml [--packs] [--context-set] [--update-rte] [--frozen-packs]

Typical IDE environments use a --context-set that specifies the context configuration of the application. For an application described by a <name>.csolution.yml file, these steps are executed:

Check YML file syntax against schema for all files specified by <name>.csolution.yml.
Check the correctness of all files specified by <name>.csolution.yml.
Evaluate the potential software layers for Reference Applications using variables: to refer to layers (if the value is undefined). All projects are considered in this step.
Evaluate the selectable compiler toolchains when the csolution project does not contain a compiler: selection or the --toolchain option is not applied. The available toolchains are based on the compiler registration and the select-compiler: node in the file <name>.csolution.yml or cdefault.yml.
Create the file compile_commands.json in the output directory for the context defined in *.cbuild-set.yml.

Note

The file compile_commands.json is generated by CMake with the option --target <context>-database and contains all source files of the context along with compiler options. It is used by various code analyzer tools and the IntelliSense system of an IDE.

The operation is further controlled by options:

The option --packs downloads missing software packs.
The option --active selects a target-set or build variant that combines various projects with defined build-types.
The option --update-rte updates the configuration files of the application.
With the option --frozen-packs the file *.cbuild-pack.yml is used as input. An error is issued when a pack version changes.
The option --toolchain can be used to explicitly select a compiler.

`csolution` Project Manager

csolution processes the csolution project files (in YAML format) and the *.pdsc metadata files of software packs and performs the following operations:

In the Project Area:
- Generate build information files *.cbuild-idx.yml and *.cbuild.yml with all relevant project information for the build process.
In the RTE Directory:
- Generate for each context the RTE_components.h file and pre-include files from the software pack (*.pdsc) metadata.
- Copy the configuration files from selected software componentsand provide PLM information.
In the base directory of the solution:
- Generate the file *.cbuild-pack.yml that records all used software packs. With the option --frozen-packs, this file is used as input.
- With the option --context-set the file *.cbuild-set.yml specifies the context configuration of the application. When --context names are specified this file is updated with this selection.

The picture below outlines the operation.

Operation of csolution tool

`cbuild2cmake` Generate CMakeLists Files

cbuild2cmake reads the build information files *.cbuild-idx.yml and *.cbuild.yml to get all relevant project information for the build process. It generates the following output files for CMake build system:

Output Directory/File	Description
`./<tmp-dir>/CMakeList.txt`	Describes the overall application build process with the current context configuration.
`./<tmp-dir>/<context>`	Each context has a separate sub-directory with the following files:
`./<tmp-dir>/<context>/CMakeList.txt`	Describes the build process for this context.
`./<tmp-dir>/<context>/toolchain.cmake`	Describes the toolchain used for this context.
`./<tmp-dir>/<context>/groups.cmake`	Contains all definitions and source files that are related to file groups.
`./<tmp-dir>/<context>/components.cmake`	Contains all definitions and source files that are related to components.

`CMake` Invocation

The CMake build system is invoked with the following commands:

CMake configuration command defines the build generator, source, and build directory with:

cmake -G Ninja -S <tmpdir> -B <tmpdir> -Wnodev

CMake build command to build the application program for each context with:

cmake --build <tmpdir> -j <n> --target <context>

CMake build command to generate the database file compile_commands.json (used by the command cbuild setup) for each context with:

cmake --build <tmpdir> -j <n> --target <context>-database

Adding a Toolchain to CMSIS-Toolbox

The following section explains how to add a compiler toolchain to the CMSIS-Toolbox.

Steps

The section below describes the steps to add a new compiler toolchain to the CMSIS-Toolbox.

Define a compiler_name for the new compiler toolchain, i.e. CLang.
Add this compiler_name to the "CompilerType": in the schema file ./tools/projmgr/schemas/common.schema.json.
Create a new CMake file in ./tools/buildmgr/cbuildgen/config with the naming convention compiler_name.<version>.cmake.
Map with the file compiler_name.<version>.cmake. the CMake input variables to the CMake toolchain variables.
Use an existing *.cmake file, i.e. GCC.<version>.cmake as a reference.

CMake Variables

The CMakeLists.txt file sets the following CMake input variables that should be processed by compiler_name.<version>.cmake.

CMake Variable	Possible Values	Description
`BYTE_ORDER`	Little-endian, Big-endian	Endian processor configuration
`CPU`	DCoreEnum	Processor core selection
`FPU`	DfpuEnum	Floating point unit support
`DSP`	DdspEnum	DSP instruction set support
`TZ`	DtzEnum	TrustZone support
`SECURE`	DsecureEnum	Software model selection
`MVE`	DmveEnum	MVE instruction set support
`BRANCHPROT`	`BRANCHPROT` values	[Branch protection
`OPTIMIZE`	Optimize values	Generic optimize levels for code generation
`WARNINGS`	Warnings values	Control warning level for compiler diagnostic
`DEBUG`	Debug values	Control the generation of debug information
`DEFINES`	Define symbols	List of symbol #define statements

`BRANCHPROT` Values

The following table lists the possible values for the CMake variable BRANCHPROT.

Values	Description
NO_BRANCHPROT	Branch protection not used
BTI	Using BTI (Branch Target ID)
BTI_SIGNRET	Using BTI + Sign Return

The compiler_name.<version>.cmake sets the following CMake variables to specify the toolchain and select toolchain options.

CMake Variable	Description
`ASM_CPU`, `CC_CPU`, `CXX_CPU`	Device selection set according to the combination of device attributes (`CPU`, `FPU`, `DSP`, `MVE`, etc.)
`AS_LEG_CPU`, `AS_ARM_CPU`, `AS_GNU_CPU`	Similar to the previous item but for assembly dialect variants (if applicable)
`ASM_FLAGS`, `CC_FLAGS`, `CXX_FLAGS`, `LD_FLAGS`	Flags applicable to all modules of the given language
`CC_SECURE`, `LD_SECURE`	Flags applicable only for secure projects
`_PI`	Pre-include option
`_ISYS`	system include option
`LIB_PREFIX`	Generated library name prefix
`LIB_SUFFIX`	Generated library name suffix
`EXE_SUFFIX`	Generated executable name suffix
`ELF2HEX`	Flags for ELF to HEX conversion
`ELF2BIN`	Flags for ELF to BIN conversion
`CMAKE_C_COMPILER_ID`	CMake compiler identifier
`CMAKE_C_COMPILER_VERSION`	CMake compiler version

CMake Integration

The executes: node in the csolution project files allows the integration of other CMake projects or scripts.

Example

The following CMakeLists.txt file integrates the FCARM file converter that is part of the MDK-Middleware. The FCARM file converter reformats all web files into a single C-file which is then included and compiled into the project.

# CMakeLists.txt for calling FCARM
# Find input files in the input base directory and in its subfolders using recursive scanning
# Format arguments and generate a steering command file, overcoming any command line length limitation
# Call FCARM using the steering command file, generating the source file in the expected output
#
# Configuration Step: ${CMAKE_COMMAND} -G <generator> -S <source directory> -B <build directory> -DINPUT_DIRECTORY=<input base directory> -DOUTPUT=<output source file>
# Build Step: ${CMAKE_COMMAND} --build <build directory>
#
# <generator>: underlying generator build system, e.g. Ninja
# <source directory>: directory where this CMakeLists.txt resides
# <build directory>: directory for temp files
# <input base directory>: directory where input data is located
# <output source file>: path and filename of source file to be generated

cmake_minimum_required(VERSION 3.22)
include(ExternalProject)

project("FCARM" NONE)

file(GLOB_RECURSE INPUT ${INPUT_DIRECTORY}/*)

foreach(ITEM ${INPUT})
  cmake_path(RELATIVE_PATH ITEM BASE_DIRECTORY ${INPUT_DIRECTORY} OUTPUT_VARIABLE FILE)
  list(APPEND FILES ${FILE})
endforeach()
string(REPLACE ";" ",\n" FILES "${FILES}")

cmake_path(RELATIVE_PATH OUTPUT BASE_DIRECTORY ${INPUT_DIRECTORY} OUTPUT_VARIABLE RELATIVE_OUTPUT)
cmake_path(GET INPUT_DIRECTORY FILENAME INPUT_DIRECTORY_NAME)
cmake_path(GET INPUT_DIRECTORY PARENT_PATH WORKING_DIRECTORY)

set(COMMAND_FILE "${CMAKE_CURRENT_BINARY_DIR}/Auto_FcArm_Cmd.inp")
file(WRITE ${COMMAND_FILE} "${FILES}\nTO ${RELATIVE_OUTPUT} RTE NOPRINT ROOT(${INPUT_DIRECTORY_NAME})\n")

add_custom_target(FCARM ALL DEPENDS ${OUTPUT})
add_custom_command(OUTPUT ${OUTPUT} DEPENDS ${INPUT}
  COMMAND fcarm @${COMMAND_FILE}
  WORKING_DIRECTORY ${WORKING_DIRECTORY}
)

Integration in a csolution project. In this case, it is part of the *.csolution.yml, but it may also be part of the *.cproject.yml file that uses the source file Web.c as input. The CMake build system checks for project dependencies and schedules the overall build process.

solution:
  :

  executes:
    - execute: Run-FCARM
      run: ${CMAKE_COMMAND} -G Ninja -S ${INPUT_0} -B ${CMAKE_CURRENT_BINARY_DIR}/fcarm-cmake -DINPUT_DIRECTORY=${INPUT_1} -DOUTPUT=${OUTPUT} && ${CMAKE_COMMAND} --build ${CMAKE_CURRENT_BINARY_DIR}/fcarm-cmake -- --quiet
      always:
      input:
        - fcarm-cmake    # CMake script directory
        - Web            # Input directory with "web files" for FCARM
      output:
        - project/Web.c  # Output file for FCARM

West Integration

The west build system uses two environment variables to configure the toolchain. These variables are set when the *.csolution.yml contains the west: node. It is created by the VS Code CMSIS Solution extension when a terminal is opened (ToDo) or by the cbuild orchestration when west build is called during the setup and build process.

The west build command is called for each app-path: that is specified in the west: node.

Environment Variable	Description
`ZEPHYR_TOOLCHAIN_VARIANT`	Toolchain selection depending on the selected `compiler:` in `csolution.yml`. Values are: armclang (for AC6), gnuarmemb (for GCC), llvm (for CLANG), iar (for IAR).
`<TOOLCHAIN>_TOOLCHAIN_PATH`	Path to selected compiler executable. `<TOOLCHAIN>` is the capitalized string specified with `ZEPHYR_TOOLCHAIN_VARIANT`. The value is copied from the compiler registration environment variable.

The various build operations of the cbuild tool map as shown below to the west tool.

Operation	Cbuild Invocation	West Invocation
Build	`cbuild <name>.csolution.yml`	`west build -p auto -d <dir> -b <board> <app>`
Database	`cbuild setup <name>.csolution.yml`	`west build -p auto -d <dir> -b <board> --cmake-only <app> -- -DCMAKE_EXPORT_COMPILE_COMMANDS=ON`

The cbuild Clean operation is performed as usual by cleaning temporary and output files, and the cbuild Rebuild triggers a normal build after cleaning. No other west build option is required.

Generator Integration

The diagram below shows how the STM32CubeMX generator is integrated into the CMSIS build process. The data flow is exemplified on STM32CubeMX (Generator ID for this example is CubeMX). The information about the project is delivered to the generator using the Generator Information files (<solution-name>.cbuild-gen-idx.yml and <context>.cbuild-gen.yml). This information provides CubeMX with the project context, such as the selected board or device, and CPU mode, such as TrustZone, disabled/enabled.

Generator Integration

The utility cbridge gets as parameter the <solution-name>.cbuild-gen-idx.yml and calls the generator. For the CubeMX generator example, these files are created:

*.ioc CubeMX project file with current project settings
*.c/.h source files, i.e. for interfacing with drivers
<project-name>.cgen.yml (created by cbridge) provides the data for project import into the csolution build process.

Note

The NXP MCUXpresso generator has a direct interface to the csolution project format. The utility cbridge is in this case not required as the build information files are the input and the <project-name>.cgen.yml is directly generated.

Generator Start via `component`

A <component> element with a generator attribute in a *.PDSC file is used to start the generator. Typically this component is provided in a Device Family Pack (DFP) or a Board Support Pack (BSP).

Example component for CubeMX in DFP:

  <component generator="CubeMX" Cclass="Device" Cgroup="CubeMX" Cversion="0.9.0">
      <description>Configure device or board with STM32CubeMX</description>
  </component>

Note

No <generator> element in the *.PDSC file is required when the Global Generator Registry File is used. The generator="id" attribute of the <component> element in the *.PDSC file is the reference to the - id: list node in the global.generator.yml file.

Global Generator Registry File

For generators with no <generator> element in the *.PDSC file, the global.generator.yml in the CMSIS-Toolbox ./etc directory contains is used. The generator: node in this YAML file registers the supported generators with the following keys:

`generator:`		Content
`- id:`	Required	`<generator-id>` referred in the `*.PDSC` file
`download-url:`	Optional	URL for downloading the generator
`run:`	Required	Name and location of the utility that starts the generator
`path:`	Required	Output directory of the generator. Contains the file `*.cgen.yml`.

generator:
  - id: CubeMX
    download-url: https://www.st.com/en/development-tools/stm32cubemx.html#st-get-software
    run: ../bin/cbridge
    path: $SolutionDir()$/STM32CubeMX/$TargetType$

Note

The only argument to the run: command is the path to the Generator Information Index File. There are no configurable parameters for this utility. The invocation is: cbrige <csolution-name>.cbuild-gen-idx.yml

Add Configuration Generator

The following steps add a new generator to the CMSIS-Toolbox:

Add a Generator Start Component to a software pack (for device configuration typically the DFP).
Add the configuration utility to the Global Generator Registry File with a pull request to the devtools repository.
The generator tool specified with run: is called with the *.cbuild-gen-idx.yml file as command line parameter. This file also specifies the output directory for the generated files.
The generator tool should generate a *.cgen.yml file and other source files that are included in the project.

An example project where you can explore this process is CubeMX. It contains an example for a *.cgen.yml file.

Debug Adapter Integration

The file debug-adapters.yml in the CMSIS-Toolbox ./etc directory contains the list of supported debug adapters. If no debugger is specified for a csolution project, the first debug adapter ("CMSIS-DAP@pyOCD") contained in this file will be used.

The debug-adapters: node in this YAML file registers the supported debug adapters with the following keys:

`debug-adapters:`		Content
`- name:`	Required	`<generator-id>` referred in the `*.PDSC` file
`alias-name:`	Optional	List of names (in input node or BSP) that map to this debug adapter.
`template:`	Optional	Used only by the VS Code CMSIS Solution extension for configuration.
`gdbserver:`	Optional	Add the `gdbserver:` node in the `cbuild-run.yml` file.
`defaults:`	Optional	List of default options to use when not specified in the `target-set:` node.
`user-interface:`	Optional	Defines the user interface for VS Code CMSIS Solution extension.

Note

As the file is shared with the VS Code CMSIS Solution extension, also template files are references. However these template files are not part of the CMSIS-Toolbox.
The debugger name: <none> identifies no debugger configuration. With this setting, no template: is applied and the .vscode configuration files are not modified.

Example debug-adapters.yml

debug-adapters:
  - name: "CMSIS-DAP@pyOCD"
    alias-name: ["CMSIS-DAP", "DAP-Link"]     # alternative names that map to this debug adapter
    template: CMSIS-DAP-pyOCD.adapter.json    # template file
    gdbserver:                                # add gdbserver: node under debugger: in cbuild-run.yml
    defaults:                                 # default values to use when nowhere specified
      port: 3333                              # default value of first gdbserver port
      protocol: swd
      clock: 10000000

  - name: "ULINKplus"
    template: CMSIS-DAP-pyOCD.adapter.json    # template file (initally same as CMSIS-DAP@pyOCD)
    gdbserver:                                # add gdbserver: node under debugger: in cbuild-run.yml
    defaults:                                 # default values to use when nowhere specified
      port: 3333                              # default value of first gdbserver port
      protocol: swd
      clock: 10000000

  - name: "ST-Link@pyOCD"
    alias-name: ["ST-LINK"]                   # alternative names that map to this debug adapter
    template: STLink-pyOCD.adapter.json       # template file
    gdbserver:                                # add gdbserver: node under debugger: in cbuild-run.yml
    defaults:                                 # default values to use when nowhere specified
      port: 3333                              # default value of first gdbserver port
      protocol: swd
      clock: 10000000

  - name: "JLink Server"
    template: jlink.adapter.json              # template file
    defaults:                                 # default values to use when nowhere specified
      port: 3333                              # default value of first gdbserver port
      protocol: swd
      clock: auto

  - name: "AVH-FVP"
    template: FVP.adapter.json                # template file

  - name: "Keil uVision"
    template: uVision.adapter.json            # template file

User Interface

The file debug-adapters.yml also contains the information for the user interface that is available in the VS Code CMSIS Solution extension. The options are added

`user-interface:`		Content
`- section:`	Required	Label text for configuration section.
`description:`	Optional	Hover over text.
`yml-node:`	Optional	If present, options are under this group node in the `debugger:` section.
`select:`	Optional	If present, the section can be enabled. Applies to all options.
`options:`	Optional	List of available options.
`pname-options:`	Optional	List of available options that are repeated for each pname.

Note

When a section: is disabled all nodes are removed from the csolution.yml file.

`options:`		Content
`- name:`	Required	Label text for the option in the user interface.
`description:`	Optional	Hover over text.
`yml-node:`	Required	Name of the node in the csolution file under `debugger:` section.
`type:`	Required	Type (enum: value list, number: value, string: name, file: name).
`range:`	Optional	Value range for type int.
`values:`	Optional	Value list for type enum.
`default:`	Optional	Default value (or enum name) for user interface when no value given in csolution.yml.
`scale:`	Optional	The value in csolution.yml value is multiplied by the scale factor.

`values:`		Content
`- name:`	Required	Label text for the option in the user interface.
`value:`	Required	Value used for the enum name in file csolution.yml file.
`description:`	Optional	Descriptive text (hover over or sub-text in dialog).

Example

Debug Adapter User Interface

solution:
   :
      target-set:
        - set:
          debugger:0
            name: CMSIS-DAP@pyOCD
            protocol: swd
            clock: 10000000      # SWD clock = 10MHz
            trace:
              mode: UART
              clock: 120000000   # Trace clock = 120 MHz
            telnet:              # Enable telnet with defaults

Example debug-adapters.yml

debug-adapters:
  - name: "CMSIS-DAP@pyOCD"
    alias-name: ["CMSIS-DAP", "DAP-Link"] # alternative names that map to this debug adapter
    template: CMSIS-DAP-pyOCD.adapter.json # template file
    gdbserver: # add gdbserver: node under debugger: in cbuild-run.yml
    defaults: # this section is only used by csolution to provide default values when settings are missing
      port: 3333 # default value of first gdbserver port
      protocol: swd
      clock: 1000
    user-interface:  # this section is only used by the UI to display and edit settings
      - section: Debug Interface
        description: Interface configuration for the debug port
        options:
        - name: Clock (kHz)       # UI display
          description: Trace configuration   # hover over text
          yml-node: clock          # node entry in csolution.yml
          type: int                # type of value
          range: [10, 5000]        # valid range
          default: 1000            # default value if not specified anywhere
        - name: Protocol
          yml-node: protocol
          type: enum
          values: 
            - name: JTAG
              value: jtag
            - name: SWD
              value: swd
          default: SWD
      - section: Trace
        description: Trace configuration   # hover over text
        yml-node: trace                    # when a yml node is given options are under this section
        select: off
        options:
          - name: Clock (kHz)
            yml-node: trace-clock
            type: number
            range: [10, 200000] # 10 kHz .. 200 MHz
            default: 12000
          - name: Mode
            yml-node: trace-port
            type: enum
            values: 
              - name: UART
                value: UART
                description: SWO configured for UART protocol
              - name: Manchester
                value: Man
                description: SWO configured for Manchester protocol
              - name: Trace Port 4
                value: TP4
                description: Connection via 4-bit Trace port
            default: UART
      - section: Telnet
        description: Telnet server configuration   # hover over text
        yml-node: telnet
        select: off
        pname-option:
          - title: Port
            yml-node: port
            type: inumber
            range: [1, 100000]
            default: 4444

  - name: "JLink Server"
      - section: Debug Interface
        description: Interface configuration for the debug port
        options:
        - name: Clock (kHz)       # UI display
          description: JTAG/SWO clock frequency   # hover over text
          yml-node: clock          # node entry in csolution.yml
          type: number             # type of value
          range: [10, 5000]        # valid range
          default: 4000            # default value if not specified anywhere
        - name: Protocol
          yml-node: protocol
          type: enum
          values: 
            - name: SWD
              value: swd
          default: SWD
      - section: Trace
        description: Trace configuration   # hover over text
        yml-node: trace                    # only on/off option
        select: off
        options:
          - name: Mode
            yml-node: trace-port
            type: enum
            values:
              name: UART
              value: UART
            default: UART

  - name: "Keil uVision"
    template: uVision.adapter.json # template file
    defaults:
      uv4: "C:\\Keil_v5\\UV4\\UV4.exe"
      args: []
    user-interface:  # this section is only used by the UI to display and edit settings
      - section: Setup
        description: Configure uVision settings
        options:
        - name: Path
          description: Absolute path to uVision executable
          yml-node: uv4            # node entry in csolution.yml
          type: file               # type of value
          default: "C:\\Keil_v5\\UV4\\UV4.exe"

Theory of Operation

Build Process Overview

cbuild Build Invocation

Details of the build mode

Details of the setup mode

csolution Project Manager

cbuild2cmake Generate CMakeLists Files

CMake Invocation

Adding a Toolchain to CMSIS-Toolbox

Steps

CMake Variables

BRANCHPROT Values

CMake Integration

Example

West Integration

Generator Integration

Generator Start via component

Global Generator Registry File

Add Configuration Generator

Debug Adapter Integration

User Interface

`cbuild` Build Invocation

`csolution` Project Manager

`cbuild2cmake` Generate CMakeLists Files

`CMake` Invocation

`BRANCHPROT` Values

Generator Start via `component`