Ethos-U NPU (rm_ethosu)

Middleware to support Ethos-U NPU driver. This module implements the Ethos-U NPU (rm_ethosu).

Overview

RM_ETHOSU module is a wrapper of the Ethos-U core driver.

Features

• Initialization Ethos-U core driver
• NPU secure and privilege level can be configured
• Callback Events
  • NPU finishes inference job

Configuration

Build Time Configurations for rm_ethosu

The following build time configurations are defined in fsp_cfg/rm_ethosu_cfg.h:

Configuration	Options	Default	Description
Parameter Checking	Default (BSP) Enabled Disabled	Default (BSP)	Selects if code for parameter checking is to be included in the build.
Debug log level	RM_ETHOSU_LOG_LEVEL_OFF RM_ETHOSU_LOG_LEVEL_ERROR RM_ETHOSU_LOG_LEVEL_WARNING RM_ETHOSU_LOG_LEVEL_INFO RM_ETHOSU_LOG_LEVEL_DEBUG	RM_ETHOSU_LOG_LEVEL_OFF	Defines ethosu log level.

Configurations for AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu)

This module can be added to the Stacks tab via New Stack > AI > Arm Ethos-U Core Driver Wrapper (rm_ethosu).

Configuration	Options	Default	Description
Name	Name must be a valid C symbol	g_rm_ethosu0	Module name.
Callback	Name must be a valid C symbol	NULL	A user callback function that will be called from the interrupt service routine (ISR).
NPU Interrupt Priority	Value must be an integer between 0 and 255	24	Select the NPU interrupt priority.
Enable Secure Mode	Disabled Enabled	Enabled	Set Ethos-U NPU secure mode.
Enable Privilege Mode	Disabled Enabled	Enabled	Set Ethos-U NPU privilege mode.

Usage Notes

Here are the steps to create an AI application in e2studio. Please refer to the example code for details.

Getting Started: Creating an AI Project with NPU stacks

Start by creating a new C++ project in e² studio. On the Stacks tab, add New Stack > AI > Arm Ethos-U Core Lib. CMSIS DSP can be removed if not needed. On the BSP tab, stack and heap need to be configured. Generate the project content. Please note an Ethos-U core driver instance will be automatically created. It will be used by RM_ETHOSU API to initialize/deinitialize Ethos-U driver. Application also must use the same instance to call other Ethos-U core driver API if needed.

Getting Started: Configure C++ Build Settings

To build Google TFLM Core Lib, C++17 is recommended. Lower versions may not be supported. Also Run-Time Type Information and exceptions need to be disabled.

Toolchain	C++ Settings
GCC	-fno-rtti, -fno-exceptions

Getting Started: Defining functions for MCU performance profiling

ticks_per_second and GetCurrentTimeTicks need to be defined for MCU profiling. Here is an exmaple

namespace tflite {

    uint32_t ticks_per_second();
    uint32_t GetCurrentTimeTicks();

    uint32_t ticks_per_second() { return US_IN_SECOND; }

    uint32_t GetCurrentTimeTicks() {

        static bool is_initialized = false;

        if (!is_initialized) {
            CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk;

            // Reset and enable DWT cycle counter.
            DWT->CYCCNT = 0;
            DWT->CTRL |= 1UL;

            is_initialized = true;
        }

        return DWT->CYCCNT;
    }
}

Getting Started: Overriding weak functions in application

There are weak functions need to be overriden if needed

Module	Function	Description
NPU Driver	ethosu_flush_dcache	Flush/clean the data cache by address and size. Passing NULL as p argument expects the whole cache to be flushed.
NPU Driver	ethosu_invalidate_dcache	Invalidate the data cache by address and size. Passing NULL as p argument expects the whole cache to be invalidated.
NPU Driver	ethosu_mutex_create	define mutex API based on RTOS
NPU Driver	ethosu_mutex_destroy	define mutex API based on RTOS
NPU Driver	ethosu_mutex_lock	define mutex API based on RTOS
NPU Driver	ethosu_mutex_unlock	define mutex API based on RTOS
NPU Driver	ethosu_semaphore_create	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_destroy	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_take	define semaphore API based on RTOS
NPU Driver	ethosu_semaphore_give	define semaphore API based on RTOS
NPU Core	ethosu_inference_begin	Callback invoked just before the inference is started.
NPU Core	ethosu_inference_end	Callback invoked just after the inference has completed.

Getting Started: Creating AI inference

Create inference job and run, e.g

InferenceJob job(std::string(modelName),
                DataPtr(networkModelData, networkModelDataLen),
                {DataPtr(inputData, inputDataLen)},
                {DataPtr(outputData, outputDataLen)},
                {DataPtr(expectedOutputData, expectedOutputDataLen)},
                numBytesToPrint);

// Run job
bool failed = inferenceProcess.runJob(job);

Limitations

Users have to implement weak functions if needed. Please refer to https://git.mlplatform.org/ml/ethos-u/ethos-u-core-driver.git/tree/README.md for further information.

Users have to update linker script if they want to save AI model and data to external memories.

Examples

Basic Example

This is a basic example of Ethos-U NPU application.

/* These variables are auto generated. */
extern struct ethosu_driver g_ethosu_dev;
extern rm_ethosu_cfg_t g_ethosu_cfg;
extern rm_ethosu_instance_ctrl_t g_ethosu_ctrl;

extern "C"
{
    /* Override the SysTick Handler. */
    extern void SysTick_Handler(void);
    void SysTick_Handler(void)
    {
        // Disable systick, preventing new systick interrupt to fire while we call the Ethos-U monitor
        SysTick->CTRL = 0;

        if (ethosuDrv != NULL) {
            ethosuMonitor.monitorSample(ethosuDrv);

            // Restart the systick timer
            SysTick_Config(usDelay);
        }
    }

    void ethosu_inference_begin(struct ethosu_driver *drv, void *)
    {
        ethosuDrv = drv;
        ethosuMonitor.configure(drv, pmuEventConfig);
        SysTick_Config(usDelay);
    }

    void ethosu_inference_end(struct ethosu_driver *drv, void *)
    {
        // Disable polling
        SysTick->CTRL = 0;

        ethosuDrv = NULL;
        ethosuMonitor.monitorSample(drv);
        ethosuMonitor.release(drv);
    }
}

void rm_ethosu_basic_example (void)
{
    fsp_err_t err = FSP_SUCCESS;


    err = RM_ETHOSU_Open(&g_ethosu_ctrl, &g_ethosu_cfg);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


    /* Initialize event recoder */
    EventRecorderInitialize(EventRecordAll, 1);

    InferenceJob job(std::string(modelName),
                    DataPtr(networkModelData, networkModelDataLen),
                    {DataPtr(inputData, inputDataLen)},
                    {DataPtr(outputData, outputDataLen)},
                    {DataPtr(expectedOutputData, expectedOutputDataLen)},
                    numBytesToPrint);

    // Run job
    bool failed = inferenceProcess.runJob(job);
    printf("Status of executed job: %s\n", failed ? "Failed\n" : "Success\n");


    err = RM_ETHOSU_Close(&g_ethosu_ctrl);
    if (FSP_SUCCESS != err)
    {
        /* Handle any errors. */
    }


}