Ethos-U (rm_ethosu)

Functions
fsp_err_t	RM_ETHOSU_Open (rm_ethosu_ctrl_t *p_api_ctrl, rm_ethosu_cfg_t const *const p_cfg)
fsp_err_t	RM_ETHOSU_CallbackSet (rm_ethosu_ctrl_t *const p_api_ctrl, void(*p_callback)(rm_ethosu_callback_args_t *), void *const p_context, rm_ethosu_callback_args_t *const p_callback_memory)
fsp_err_t	RM_ETHOSU_Close (rm_ethosu_ctrl_t *p_api_ctrl)

Detailed Description

Middleware to support Ethos-U NPU driver. This module implements the Ethos-U Interface.

Overview

RM_ETHOSU module is a wrapper of the Ethos-U core driver.

Features

• Initialization Ethos-U core driver
• NPU secure and privilege level can be configured
• Callback Events
  • NPU finishes inference job

Configuration

Build Time Configurations for rm_ethosu

The following build time configurations are defined in fsp_cfg/rm_ethosu_cfg.h:

Configuration	Options	Default	Description
Parameter Checking	Default (BSP) Enabled Disabled	Default (BSP)	Selects if code for parameter checking is to be included in the build.
Debug log level	RM_ETHOSU_LOG_LEVEL_OFF RM_ETHOSU_LOG_LEVEL_ERROR RM_ETHOSU_LOG_LEVEL_WARNING RM_ETHOSU_LOG_LEVEL_INFO RM_ETHOSU_LOG_LEVEL_DEBUG	RM_ETHOSU_LOG_LEVEL_OFF	Defines ethosu log level.

Configurations for AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu)

This module can be added to the Stacks tab via New Stack > AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu).

Configuration	Options	Default	Description
Name	Name must be a valid C symbol	g_rm_ethosu0	Module name.
Callback	Name must be a valid C symbol	NULL	A user callback function that will be called from the interrupt service routine (ISR).
NPU Interrupt Priority	MCU Specific Options		Select the NPU interrupt priority.
Enable Secure Mode	Disabled Enabled	Enabled	Set Ethos-U NPU secure mode.
Enable Privilege Mode	Disabled Enabled	Enabled	Set Ethos-U NPU privilege mode.

Clock Configuration

The NPU clock is configurable on the clocks tab.

Usage Notes

Here are the steps to create an AI application in e2studio. Please refer to the example code for details.

Getting Started: Creating an AI Project with NPU stacks

Start by creating a new c++ project in e² studio. On the Stacks tab, add New Stack > AI > TFLM Core Lib. On the BSP tab, stack and heap need to be configured. On the Clocks tab, NPU clock needs to be configured. Generate the project content. Please note an Ethos-U core driver instance will be automatically created. It will be used by RM_ETHOSU API to initialize/deinitialize Ethos-U driver. Application also must use the same instance to call other Ethos-U core driver API if needed.

Getting Started: Confirming operators used in the model

Users can use tools like Netron to find out what operators are used by the model (.tflite). If there are opeartors not yet supported by Ethos-U55, then they need to be run on MCU. There are two types of kernels: optimized kernel based on CMSIS-NN, and reference kernel. Optimized kernel (CMSIS-NN kernel)should always be preferred on ARM MCUs. Please refer to https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/micro/docs/optimized_kernel_implementations.md for more information.

Getting Started: Learning inference process framework

Users can refer to inference process https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-core-software/-/tree/main/applications for how to integrate TFLM inference functions, e.g parsing model, creating interpreter, allocating tensors and running inference. Please note by default lots of operators are registered to micro_op_resolver in micro_mutable_all_ops_resolver.h. What opreators are needed should be determined by your model.

Getting Started: Defining functions and PMU events for MCU performance profiling

ticks_per_second and GetCurrentTimeTicks need to be defined for MCU profiling. Here is an example

namespace tflite {

    uint32_t ticks_per_second();
    uint32_t GetCurrentTimeTicks();

    uint32_t ticks_per_second() { return US_IN_SECOND; }

    uint32_t GetCurrentTimeTicks() {

        static bool is_initialized = false;

        if (!is_initialized) {
            ARM_PMU_Enable();
            DCB->DEMCR |= DCB_DEMCR_TRCENA_Msk;

            ARM_PMU_CYCCNT_Reset();
            ARM_PMU_CNTR_Enable(PMU_CNTENSET_CCNTR_ENABLE_Msk);

            is_initialized = true;
        }

        return ARM_PMU_Get_CCNTR();
    }
}

There are 4 PMU event counters. Each can be used to monitor an event, e.g

#define ETHOSU_PMU_EVENT_0 ETHOSU_PMU_CYCLE
#define ETHOSU_PMU_EVENT_1 ETHOSU_PMU_NPU_ACTIVE

Getting Started: Overriding weak functions in application

There are weak functions need to be overridden if needed

Module	Function	Description
NPU Driver	ethosu_flush_dcache	Flush/clean the data cache by address and size. Passing NULL as p argument expects the whole cache to be flushed.
NPU Driver	ethosu_invalidate_dcache	Invalidate the data cache by address and size. Passing NULL as p argument expects the whole cache to be invalidated.
NPU Driver	ethosu_mutex_create	define mutex API based on RTOS
NPU Driver	ethosu_mutex_destroy	define mutex API based on RTOS
NPU Driver	ethosu_mutex_lock	define mutex API based on RTOS
NPU Driver	ethosu_mutex_unlock	define mutex API based on RTOS
NPU Driver	ethosu_semaphore_create	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_destroy	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_take	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_give	define semaphore API based on RTOS
NPU Core	ethosu_inference_begin	Callback invoked just before the inference is started.
NPU Core	ethosu_inference_end	Callback invoked just after the inference has completed.

Getting Started: Creating AI inference

Create inference job and run, e.g

InferenceJob job(std::string(modelName),
                DataPtr(networkModelData, networkModelDataLen),
                {DataPtr(inputData, inputDataLen)},
                {DataPtr(outputData, outputDataLen)},
                {DataPtr(expectedOutputData, expectedOutputDataLen)},
                numBytesToPrint);

// Run job
bool failed = inferenceProcess.runJob(job);

Limitations

Users have to implement weak functions if needed. Please refer to https://git.mlplatform.org/ml/ethos-u/ethos-u-core-driver.git/tree/README.md for further information.

Users have to update linker script if they want to save AI model and data to external memories.

Examples

Basic Example

This is a basic example of Ethos-U NPU application.

/* These variables are auto generated. */
extern struct ethosu_driver g_ethosu_dev;
extern rm_ethosu_cfg_t g_ethosu_cfg;
extern rm_ethosu_instance_ctrl_t g_ethosu_ctrl;

extern "C"
{
    /* Override the SysTick Handler. */
    extern void SysTick_Handler(void);
    void SysTick_Handler(void)
    {
        // Disable systick, preventing new systick interrupt to fire while we call the Ethos-U monitor
        SysTick->CTRL = 0;

        if (ethosuDrv != NULL) {
            ethosuMonitor.monitorSample(ethosuDrv);

            // Restart the systick timer
            SysTick_Config(usDelay);
        }
    }

    void ethosu_inference_begin(struct ethosu_driver *drv, void *)
    {
        ethosuDrv = drv;
        ethosuMonitor.configure(drv, pmuEventConfig);
        SysTick_Config(usDelay);
    }

    void ethosu_inference_end(struct ethosu_driver *drv, void *)
    {
        // Disable polling
        SysTick->CTRL = 0;

        ethosuDrv = NULL;
        ethosuMonitor.monitorSample(drv);
        ethosuMonitor.release(drv);
    }
}

void rm_ethosu_basic_example (void)
{
    fsp_err_t err = FSP_SUCCESS;


    err = RM_ETHOSU_Open(&g_ethosu_ctrl, &g_ethosu_cfg);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


    /* Initialize event recoder */
    EventRecorderInitialize(EventRecordAll, 1);

    InferenceJob job(std::string(modelName),
                    DataPtr(networkModelData, networkModelDataLen),
                    {DataPtr(inputData, inputDataLen)},
                    {DataPtr(outputData, outputDataLen)},
                    {DataPtr(expectedOutputData, expectedOutputDataLen)},
                    numBytesToPrint);

    // Run job
    bool failed = inferenceProcess.runJob(job);
    printf("Status of executed job: %s\n", failed ? "Failed\n" : "Success\n");


    err = RM_ETHOSU_Close(&g_ethosu_ctrl);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


}

Data Structures
struct	rm_ethosu_instance_ctrl_t

---

Data Structure Documentation

rm_ethosu_instance_ctrl_t

struct rm_ethosu_instance_ctrl_t

Instance control block. This is private to the FSP and should not be used or modified by the application.

Function Documentation

RM_ETHOSU_Open()

fsp_err_t RM_ETHOSU_Open	(	rm_ethosu_ctrl_t *	p_api_ctrl,
		rm_ethosu_cfg_t const *const	p_cfg
	)

Enable Ethos-U NPU and initialize driver. Implements rm_ethosu_api_t::open.

This function performs the following tasks:

• Performs parameter checking and processes error conditions.
• Enables NPU module
• Initializes Ethos-U NPU driver.
• Provides the handle for use with other API functions.

Parameters

p_api_ctrl	Pointer to the control structure.
p_cfg	Pointer to a configuration structure.

Return values

FSP_SUCCESS	NPU initialized successfully.
FSP_ERR_ASSERTION	An input parameter is invalid or NULL.
FSP_ERR_ALREADY_OPEN	The instance has already been opened.
FSP_ERR_INVALID_ARGUMENT	Failed to initialize ethosu driver

RM_ETHOSU_CallbackSet()

fsp_err_t RM_ETHOSU_CallbackSet	(	rm_ethosu_ctrl_t *const	p_api_ctrl,
		void(*)(rm_ethosu_callback_args_t *)	p_callback,
		void *const	p_context,
		rm_ethosu_callback_args_t *const	p_callback_memory
	)

Updates the user callback and has option of providing memory for callback structure. Implements rm_ethosu_api_t::callbackSet

Return values

FSP_SUCCESS	Callback updated successfully.
FSP_ERR_ASSERTION	A required pointer is NULL.
FSP_ERR_NOT_OPEN	The control block has not been opened.
FSP_ERR_NO_CALLBACK_MEMORY	p_callback is non-secure and p_callback_memory is either secure or NULL.

RM_ETHOSU_Close()

fsp_err_t RM_ETHOSU_Close

(

rm_ethosu_ctrl_t *

p_api_ctrl

)

Disable NPU and set the instance as not open. Implements rm_ethosu_api_t::close.

Parameters

p_api_ctrl

Pointer to an opened instance.

Return values

FSP_SUCCESS	Channel successfully closed.
FSP_ERR_ASSERTION	The parameter p_api_ctrl is NULL.
FSP_ERR_NOT_OPEN	The channel has not been opened. Open the channel first.