Ethos-U (rm_ethosu)

Functions
fsp_err_t	RM_ETHOSU_Open (rm_ethosu_ctrl_t *p_api_ctrl, rm_ethosu_cfg_t const *const p_cfg)
fsp_err_t	RM_ETHOSU_CallbackSet (rm_ethosu_ctrl_t *const p_api_ctrl, void(*p_callback)(rm_ethosu_callback_args_t *), void *const p_context, rm_ethosu_callback_args_t *const p_callback_memory)
fsp_err_t	RM_ETHOSU_Close (rm_ethosu_ctrl_t *p_api_ctrl)

Detailed Description

Middleware to support Ethos-U NPU driver. This module implements the Ethos-U Interface.

Overview

RM_ETHOSU module is a wrapper of the Ethos-U core driver.

Features

• Initialization Ethos-U core driver
• NPU secure and privilege level can be configured
• Callback Events
  • NPU finishes inference job

Configuration

Build Time Configurations for rm_ethosu

The following build time configurations are defined in fsp_cfg/rm_ethosu_cfg.h:

Configuration	Options	Default	Description
Parameter Checking	Default (BSP) Enabled Disabled	Default (BSP)	Selects if code for parameter checking is to be included in the build.
Ethos-U log level	Disabled Error Warning Info Debug	Disabled	Defines Ethos-U log level.

Configurations for AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu)

This module can be added to the Stacks tab via New Stack > AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu).

Configuration	Options	Default	Description
Name	Name must be a valid C symbol	g_rm_ethosu0	Module name.
Callback	Name must be a valid C symbol	NULL	A user callback function that will be called from the interrupt service routine (ISR).
NPU Interrupt Priority	MCU Specific Options		Select Ethos-U interrupt priority.
Enable Secure Mode	Disabled Enabled	Enabled	Set Ethos-U NPU secure mode.
Enable Privilege Mode	Disabled Enabled	Enabled	Set Ethos-U NPU privilege mode.

Clock Configuration

The NPU clock is configurable on the clocks tab.

Usage Notes

Below are the steps to create an AI application that uses a tflite model. If you have a model geneated by RUHMI framework, the TFLM Core lib is not required, and the project type doesn't need to be C++. Please refer to RHUMI framework for more information https://github.com/renesas/ruhmi-framework-mcu.

Getting Started: Create an AI project

Start by creating a new C++ project in e² studio. On the BSP tab, stack and heap need to be configured. On the Clocks tab, NPU clock needs to be configured. On the Stacks tab, add New Stack > AI > Google TFLM Core Lib.

Getting Started: Configure C++ Build Settings

To build Google TFLM Core Lib, C++17 is recommended. Lower versions may not be supported. Also Run-Time Type Information and exceptions need to be disabled.

Toolchain	C++ Settings
GCC/LLVM	-fno-rtti, -fno-exceptions
ARMCC	-D__ARMCC_USING_STANDARDLIB=1
IAR	-DARM_MATH_AUTOVECTORIZE=1, -DFLATBUFFERS_USE_STD_OPTIONAL=0

Please note

1. -DTF_LITE_STATIC_MEMORY need to be defined if dynamic memory allocation is not supported.

Getting Started: Know your model

Users can use tools like Netron to find out what operators are used by the model. If there are operators not yet supported by Ethos-U55, they must be run on MCU. It is also possible to run a model on MCU only if the model is not optimized for NPU support.

There are two types of MCU kernels: optimized kernel based on CMSIS-NN, and reference kernel. Optimized kernel (CMSIS-NN kernel) should always be preferred on ARM MCUs. Please refer to https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/micro/docs/optimized_kernel_implementations.md for more information.

Getting Started: Implement inference process framework

Users can refer to https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-core-software/-/tree/main/applications for how to integrate TFLM inference functions, e.g parsing model, creating interpreter, allocating tensors and running inference. Please note by default lots of operators are registered to micro_op_resolver in micro_mutable_all_ops_resolver.h. Only operators used in your model need to be registered.

Getting Started: Creating AI inference

Create inference job and run, e.g

InferenceJob job(std::string(modelName),
                DataPtr(networkModelData, networkModelDataLen),
                {DataPtr(inputData, inputDataLen)},
                {DataPtr(outputData, outputDataLen)},
                {DataPtr(expectedOutputData, expectedOutputDataLen)},
                numBytesToPrint);

// Run job
bool failed = inferenceProcess.runJob(job);

Getting Started: Profile performance

Ethos-U Profiler stack has the components for performance profiling, e.g ethos-u monitor, event recoder. Users will have to implement ticks_per_second() and GetCurrentTimeTicks() for this purpose. Here is an example

namespace tflite {

    uint32_t ticks_per_second();
    uint32_t GetCurrentTimeTicks();

    uint32_t ticks_per_second() { return US_IN_SECOND; }

    uint32_t GetCurrentTimeTicks() {

        static bool is_initialized = false;

        if (!is_initialized) {
            ARM_PMU_Enable();
            DCB->DEMCR |= DCB_DEMCR_TRCENA_Msk;

            ARM_PMU_CYCCNT_Reset();
            ARM_PMU_CNTR_Enable(PMU_CNTENSET_CCNTR_ENABLE_Msk);

            is_initialized = true;
        }

        return ARM_PMU_Get_CCNTR();
    }
}

Note

Ethos-U Profiler stack is not required. Users can use a timer for performance profiling.

Getting Started: Overwrite weak functions

Below weak functions shall be overwritten by the application as needed.

Function	Description
ethosu_mutex_create	define mutex API based on RTOS
ethosu_mutex_destroy	define mutex API based on RTOS
ethosu_mutex_lock	define mutex API based on RTOS
ethosu_mutex_unlock	define mutex API based on RTOS
ethosu_semaphore_create	define semaphore API based on RTOS
ethosu_semaphore_destroy	define semaphore API based on RTOS
ethosu_semaphore_take	define semaphore API based on RTOS
ethosu_semaphore_give	define semaphore API based on RTOS
ethosu_inference_begin	Callback invoked just before the inference is started.
ethosu_inference_end	Callback invoked just after the inference is completed.

Below weak functions are overwritten in the rm_ethosu driver

Function	Description
ethosu_flush_dcache	Flush/clean the data cache by address and size. Passing NULL as p argument expects the whole cache to be flushed.
ethosu_invalidate_dcache	Invalidate the data cache by address and size. Passing NULL as p argument expects the whole cache to be invalidated.

Please refer to https://git.mlplatform.org/ml/ethos-u/ethos-u-core-driver.git/tree/README.md for more information.

Limitations

• e2studio project type must be C++ when tflite-micro is used
• Stack Ethos-U Profiler can only be used with the tflite-micro
• OFS2 must be enabled to configure NPU secure and privilege attributes. When used as non-secure, please make sure OFS2 NPU attributes are the same in secure and non-secure projects.

Examples

Basic Example

This is a basic example of Ethos-U NPU application.

/* These variables are auto generated. */
extern struct ethosu_driver g_ethosu_dev;
extern rm_ethosu_cfg_t g_ethosu_cfg;
extern rm_ethosu_instance_ctrl_t g_ethosu_ctrl;

extern "C"
{
    /* Override the SysTick Handler. */
    extern void SysTick_Handler(void);
    void SysTick_Handler(void)
    {
        // Disable systick, preventing new systick interrupt to fire while we call the Ethos-U monitor
        SysTick->CTRL = 0;

        if (ethosuDrv != NULL) {
            ethosuMonitor.monitorSample(ethosuDrv);

            // Restart the systick timer
            SysTick_Config(usDelay);
        }
    }

    void ethosu_inference_begin(struct ethosu_driver *drv, void *)
    {
        ethosuDrv = drv;
        ethosuMonitor.configure(drv, pmuEventConfig);
        SysTick_Config(usDelay);
    }

    void ethosu_inference_end(struct ethosu_driver *drv, void *)
    {
        // Disable polling
        SysTick->CTRL = 0;

        ethosuDrv = NULL;
        ethosuMonitor.monitorSample(drv);
        ethosuMonitor.release(drv);
    }
}

void rm_ethosu_basic_example (void)
{
    fsp_err_t err = FSP_SUCCESS;


    err = RM_ETHOSU_Open(&g_ethosu_ctrl, &g_ethosu_cfg);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


    /* Initialize event recorder */
    EventRecorderInitialize(EventRecordAll, 1);

    InferenceJob job(std::string(modelName),
                    DataPtr(networkModelData, networkModelDataLen),
                    {DataPtr(inputData, inputDataLen)},
                    {DataPtr(outputData, outputDataLen)},
                    {DataPtr(expectedOutputData, expectedOutputDataLen)},
                    numBytesToPrint);

    // Run job
    bool failed = inferenceProcess.runJob(job);
    printf("Status of executed job: %s\n", failed ? "Failed\n" : "Success\n");


    err = RM_ETHOSU_Close(&g_ethosu_ctrl);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


}

Data Structures
struct	rm_ethosu_instance_ctrl_t

---

Data Structure Documentation

rm_ethosu_instance_ctrl_t

struct rm_ethosu_instance_ctrl_t

Instance control block. This is private to the FSP and should not be used or modified by the application.

Function Documentation

RM_ETHOSU_Open()

fsp_err_t RM_ETHOSU_Open	(	rm_ethosu_ctrl_t *	p_api_ctrl,
		rm_ethosu_cfg_t const *const	p_cfg
	)

Enable Ethos-U NPU and initialize driver. Implements rm_ethosu_api_t::open.

This function performs the following tasks:

• Performs parameter checking and processes error conditions.
• Enables NPU module
• Initializes Ethos-U NPU driver.
• Provides the handle for use with other API functions.

Parameters

p_api_ctrl	Pointer to the control structure.
p_cfg	Pointer to a configuration structure.

Return values

FSP_SUCCESS	NPU initialized successfully.
FSP_ERR_ASSERTION	An input parameter is invalid or NULL.
FSP_ERR_ALREADY_OPEN	The instance has already been opened.
FSP_ERR_INVALID_ARGUMENT	Failed to initialize ethosu driver

RM_ETHOSU_CallbackSet()

fsp_err_t RM_ETHOSU_CallbackSet	(	rm_ethosu_ctrl_t *const	p_api_ctrl,
		void(*)(rm_ethosu_callback_args_t *)	p_callback,
		void *const	p_context,
		rm_ethosu_callback_args_t *const	p_callback_memory
	)

Updates the user callback and has option of providing memory for callback structure. Implements rm_ethosu_api_t::callbackSet

Return values

FSP_SUCCESS	Callback updated successfully.
FSP_ERR_ASSERTION	A required pointer is NULL.
FSP_ERR_NOT_OPEN	The control block has not been opened.
FSP_ERR_NO_CALLBACK_MEMORY	p_callback is non-secure and p_callback_memory is either secure or NULL.

RM_ETHOSU_Close()

fsp_err_t RM_ETHOSU_Close

(

rm_ethosu_ctrl_t *

p_api_ctrl

)

Disable NPU and set the instance as not open. Implements rm_ethosu_api_t::close.

Parameters

p_api_ctrl

Pointer to an opened instance.

Return values

FSP_SUCCESS	Channel successfully closed.
FSP_ERR_ASSERTION	The parameter p_api_ctrl is NULL.
FSP_ERR_NOT_OPEN	The channel has not been opened. Open the channel first.