Ethos-U NPU (rm_ethosu)

Functions
fsp_err_t	RM_ETHOSU_Open (rm_ethosu_ctrl_t *p_api_ctrl, rm_ethosu_cfg_t const *const p_cfg)
fsp_err_t	RM_ETHOSU_CallbackSet (rm_ethosu_ctrl_t *const p_api_ctrl, void(*p_callback)(rm_ethosu_callback_args_t *), void *const p_context, rm_ethosu_callback_args_t *const p_callback_memory)
fsp_err_t	RM_ETHOSU_Close (rm_ethosu_ctrl_t *p_api_ctrl)

Detailed Description

Middleware to support Ethos-U NPU driver. This module implements the Ethos-U NPU (rm_ethosu).

Overview

RM_ETHOSU module is a wrapper of the Ethos-U core driver.

The Ethos-U driver and RM_ETHOSU module are not recommended for direct use from application programs. They must be used in combination with the RZ/G3E Ethos Support Package. The F/W project for the CM33 core that uses the Ethos-U driver is bundled in the RZ/G3E Ethos Support Package. For details on using the Ethos-U driver, please refer to the RZ/G3E Ethos Support Package documentation.

Features

• Initialization Ethos-U core driver
• NPU secure and privilege level can be configured
• Callback Events
  • NPU finishes inference job

Configuration

Build Time Configurations for rm_ethosu

The following build time configurations are defined in fsp_cfg/rm_ethosu_cfg.h:

Configuration	Options	Default	Description
Parameter Checking	Default (BSP) Enabled Disabled	Default (BSP)	Selects if code for parameter checking is to be included in the build.
Debug log level	RM_ETHOSU_LOG_LEVEL_OFF RM_ETHOSU_LOG_LEVEL_ERROR RM_ETHOSU_LOG_LEVEL_WARNING RM_ETHOSU_LOG_LEVEL_INFO RM_ETHOSU_LOG_LEVEL_DEBUG	RM_ETHOSU_LOG_LEVEL_OFF	Defines ethosu log level.

Configurations for AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu)

This module can be added to the Stacks tab via New Stack > AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu).

Configuration	Options	Default	Description
Name	Name must be a valid C symbol	g_rm_ethosu0	Module name.
Callback	Name must be a valid C symbol	NULL	A user callback function that will be called from the interrupt service routine (ISR).
NPU Interrupt Priority	Value must be an integer between 0 and 255	24	Select the NPU interrupt priority.
Enable Secure Mode	Disabled Enabled	Enabled	Set Ethos-U NPU secure mode.
Enable Privilege Mode	Disabled Enabled	Enabled	Set Ethos-U NPU privilege mode.

Usage Notes

Here are the steps to create an AI application in e2studio. Please refer to the example code for details.

Getting Started: Creating an AI Project with NPU stacks

Start by creating a new C++ project in e² studio. On the Stacks tab, add New Stack > AI > Arm Ethos-U Core Lib. CMSIS DSP can be removed if not needed. On the BSP tab, stack and heap need to be configured. Generate the project content. Please note an Ethos-U core driver instance will be automatically created. It will be used by RM_ETHOSU API to initialize/deinitialize Ethos-U driver. Application also must use the same instance to call other Ethos-U core driver API if needed.

Getting Started: Configure C++ Build Settings

To build Google TFLM Core Lib, C++17 is recommended. Lower versions may not be supported. Also Run-Time Type Information and exceptions need to be disabled.

Toolchain	C++ Settings
GCC	-fno-rtti, -fno-exceptions

Getting Started: Defining functions for MCU performance profiling

ticks_per_second and GetCurrentTimeTicks need to be defined for MCU profiling. Here is an exmaple

namespace tflite {

    uint32_t ticks_per_second();
    uint32_t GetCurrentTimeTicks();

    uint32_t ticks_per_second() { return US_IN_SECOND; }

    uint32_t GetCurrentTimeTicks() {

        static bool is_initialized = false;

        if (!is_initialized) {
            CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk;

            // Reset and enable DWT cycle counter.
            DWT->CYCCNT = 0;
            DWT->CTRL |= 1UL;

            is_initialized = true;
        }

        return DWT->CYCCNT;
    }
}

Getting Started: Overriding weak functions in application

There are weak functions need to be overriden if needed

Module	Function	Description
NPU Driver	ethosu_flush_dcache	Flush/clean the data cache by address and size. Passing NULL as p argument expects the whole cache to be flushed.
NPU Driver	ethosu_invalidate_dcache	Invalidate the data cache by address and size. Passing NULL as p argument expects the whole cache to be invalidated.
NPU Driver	ethosu_mutex_create	define mutex API based on RTOS
NPU Driver	ethosu_mutex_destroy	define mutex API based on RTOS
NPU Driver	ethosu_mutex_lock	define mutex API based on RTOS
NPU Driver	ethosu_mutex_unlock	define mutex API based on RTOS
NPU Driver	ethosu_semaphore_create	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_destroy	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_take	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_give	define semaphore API based on RTOS
NPU Core	ethosu_inference_begin	Callback invoked just before the inference is started.
NPU Core	ethosu_inference_end	Callback invoked just after the inference has completed.

Getting Started: Creating AI inference

Create inference job and run, e.g

InferenceJob job(std::string(modelName),
                DataPtr(networkModelData, networkModelDataLen),
                {DataPtr(inputData, inputDataLen)},
                {DataPtr(outputData, outputDataLen)},
                {DataPtr(expectedOutputData, expectedOutputDataLen)},
                numBytesToPrint);

// Run job
bool failed = inferenceProcess.runJob(job);

Limitations

Users have to implement weak functions if needed. Please refer to https://git.mlplatform.org/ml/ethos-u/ethos-u-core-driver.git/tree/README.md for further information.

Users have to update linker script if they want to save AI model and data to external memories.

Examples

Basic Example

This is a basic example of Ethos-U NPU application.

/* These variables are auto generated. */
extern struct ethosu_driver g_ethosu_dev;
extern rm_ethosu_cfg_t g_ethosu_cfg;
extern rm_ethosu_instance_ctrl_t g_ethosu_ctrl;

extern "C"
{
    /* Override the SysTick Handler. */
    extern void SysTick_Handler(void);
    void SysTick_Handler(void)
    {
        // Disable systick, preventing new systick interrupt to fire while we call the Ethos-U monitor
        SysTick->CTRL = 0;

        if (ethosuDrv != NULL) {
            ethosuMonitor.monitorSample(ethosuDrv);

            // Restart the systick timer
            SysTick_Config(usDelay);
        }
    }

    void ethosu_inference_begin(struct ethosu_driver *drv, void *)
    {
        ethosuDrv = drv;
        ethosuMonitor.configure(drv, pmuEventConfig);
        SysTick_Config(usDelay);
    }

    void ethosu_inference_end(struct ethosu_driver *drv, void *)
    {
        // Disable polling
        SysTick->CTRL = 0;

        ethosuDrv = NULL;
        ethosuMonitor.monitorSample(drv);
        ethosuMonitor.release(drv);
    }
}

void rm_ethosu_basic_example (void)
{
    fsp_err_t err = FSP_SUCCESS;


    err = RM_ETHOSU_Open(&g_ethosu_ctrl, &g_ethosu_cfg);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


    /* Initialize event recoder */
    EventRecorderInitialize(EventRecordAll, 1);

    InferenceJob job(std::string(modelName),
                    DataPtr(networkModelData, networkModelDataLen),
                    {DataPtr(inputData, inputDataLen)},
                    {DataPtr(outputData, outputDataLen)},
                    {DataPtr(expectedOutputData, expectedOutputDataLen)},
                    numBytesToPrint);

    // Run job
    bool failed = inferenceProcess.runJob(job);
    printf("Status of executed job: %s\n", failed ? "Failed\n" : "Success\n");


    err = RM_ETHOSU_Close(&g_ethosu_ctrl);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


}

Classes
struct	rm_ethosu_instance_ctrl_t

---

Class Documentation

rm_ethosu_instance_ctrl_t

struct rm_ethosu_instance_ctrl_t

Instance control block. This is private to the FSP and should not be used or modified by the application.

Function Documentation

RM_ETHOSU_Open()

fsp_err_t RZG::RM_ETHOSU_Open	(	rm_ethosu_ctrl_t *	p_api_ctrl,
		rm_ethosu_cfg_t const *const	p_cfg
	)

Enable Ethos-U NPU and initialize driver. Implements rm_ethosu_api_t::open.

This function performs the following tasks:

• Performs parameter checking and processes error conditions.
• Enables NPU module
• Initializes Ethos-U NPU driver.
• Provides the handle for use with other API functions.

Parameters

p_api_ctrl	Pointer to the control structure.
p_cfg	Pointer to a configuration structure.

Return values

FSP_SUCCESS	NPU initialized successfully.
FSP_ERR_ASSERTION	An input parameter is invalid or NULL.
FSP_ERR_ALREADY_OPEN	The instance has already been opened.
FSP_ERR_INVALID_ARGUMENT	Failed to initialize ethosu driver

RM_ETHOSU_CallbackSet()

fsp_err_t RZG::RM_ETHOSU_CallbackSet	(	rm_ethosu_ctrl_t *const	p_api_ctrl,
		void(*)(rm_ethosu_callback_args_t *)	p_callback,
		void *const	p_context,
		rm_ethosu_callback_args_t *const	p_callback_memory
	)

Updates the user callback and has option of providing memory for callback structure. Implements rm_ethosu_api_t::callbackSet

Return values

FSP_SUCCESS	Callback updated successfully.
FSP_ERR_ASSERTION	A required pointer is NULL.
FSP_ERR_NOT_OPEN	The control block has not been opened.
FSP_ERR_NO_CALLBACK_MEMORY	p_callback is non-secure and p_callback_memory is either secure or NULL.

RM_ETHOSU_Close()

fsp_err_t RZG::RM_ETHOSU_Close

(

rm_ethosu_ctrl_t *

p_api_ctrl

)

Disable NPU and set the instance as not open. Implements rm_ethosu_api_t::close.

Parameters

p_api_ctrl

Pointer to an opened instance.

Return values

FSP_SUCCESS	Channel successfully closed.
FSP_ERR_ASSERTION	The parameter p_api_ctrl is NULL.
FSP_ERR_NOT_OPEN	The channel has not been opened. Open the channel first.