Tengine Support

Introduction

Tengine is an edge AI computing framework launched by OPEN AI LAB, dedicated to solving the problem of fragmentation of the AIoT industry chain and accelerating the industrialization of AI. In order to solve the problem of AIoT application landing, Tengine focuses on edge AI computing reasoning on embedded devices, and provides high-performance AI reasoning technology empowerment for massive AIoT applications and devices.

On the one hand, Tengine uses heterogeneous computing technology to simultaneously call different computing units such as CPU, GPU, DSP, and NPU to complete AI network calculations. On the other hand, it supports TensorFlow, Caffe, MXNet, PyTorch, MegEngine, DarkNet, ONNX, ncnn, etc. The industry’s mainstream framework simplifies and accelerates the rapid migration of scene-oriented AI algorithms on embedded edge devices, as well as the deployment of actual applications, which improves the efficiency of basic development tenfold.

Tengine is downward compatible with almost all mainstream domestic AI chips; upwardly, it supports all mainstream training frameworks and network models, including TensorFlow, PyTorch, Caffe, PaddlePaddle, etc. It is one of the few strategic partners who have passed ONNX official certification in the world. In the "China AI Application Developer Report" released by CSDN, it was rated as "Developers' favorite local AI framework tool".

Tengine and OpenCV work together to accelerate edge intelligence and become OpenCV's global partner. Tengine has also further in-depth cooperation with the established cross-platform computer vision library OpenCV to jointly accelerate edge intelligence and become its global partner.

For Tengine examples, please refer to: https://github.com/OAID/Tengine

Download

【Power enhancement package】Tengine D1-H Nezha development board Demo：https://www.aw-ol.com/downloads?cat=9

Version file directory description

$ tree ./sample
sample/
├── README.md
└── console
    ├── CMakeLists.txt
    ├── apps
    │   ├── CMakeLists.txt
    │   ├── common
    │   │   ├── common.h
    │   │   ├── compiler_fp16.h
    │   │   ├── msc_getopt.h
    │   │   ├── stb_image.h
    │   │   ├── stb_image_write.h
    │   │   ├── tengine_operations.c
    │   │   └── tengine_operations.h
    │   ├── test_classification.cpp
    │   └── test_yolofastest.cpp
    ├── cmake
    │   ├── check.cmake
    │   └── summary.cmake
    ├── data
    │   ├── cat.jpg
    │   ├── mobilenet_v1.tmfile
    │   ├── squeezenet.tmfile
    │   ├── ssd_dog.jpg
    │   └── yolo-fastest-1.1.tmfile
    ├── tengine
    │   └── linux-rv64
    │       ├── include
    │       │   └── tengine
    │       │       └── c_api.h
    │       └── lib
    │           ├── libtengine-lite-static.a
    │           └── libtengine-lite.so
    └── toolchains
        └── rv64-c906.toolchain.cmake

Running example

Cross tool chain preparation

Dependent cross tool chain compression package:

riscv64-glibc-gcc-thead_20200702.tar.gz

gcc version information：

$ riscv64-unknown-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=riscv64-unknown-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/mnt/d/ubuntu/riscv64-glibc-gcc-thead_20200702/bin/../libexec/gcc/riscv64-unknown-linux-gnu/8.1.0/lto-wrapper
Target: riscv64-unknown-linux-gnu
Configured with: /ldhome/software/toolsbuild/slave/workspace/riscv64_build_linux_x86_64/build/../source/riscv/riscv-gcc/configure --target=riscv64-unknown-linux-gnu --with-mpc=/ldhome/software/toolsbuild/slave/workspace/riscv64_build_linux_x86_64/lib-for-gcc-x86_64-linux/ --with-mpfr=/ldhome/software/toolsbuild/slave/workspace/riscv64_build_linux_x86_64/lib-for-gcc-x86_64-linux/ --with-gmp=/ldhome/software/toolsbuild/slave/workspace/riscv64_build_linux_x86_64/lib-for-gcc-x86_64-linux/ --prefix=/ldhome/software/toolsbuild/slave/workspace/riscv64_build_linux_x86_64/install --with-sysroot=/ldhome/software/toolsbuild/slave/workspace/riscv64_build_linux_x86_64/install/sysroot --with-system-zlib --enable-shared --enable-tls --enable-languages=c,c++,fortran --disable-libmudflap --disable-libssp --disable-libquadmath --disable-nls --disable-bootstrap --src=../../source/riscv/riscv-gcc --enable-checking=yes --with-pkgversion='C-SKY RISCV Tools V1.8.4 B20200702' --enable-multilib --with-abi=lp64d --with-arch=rv64gcxthead 'CFLAGS_FOR_TARGET=-O2  -mcmodel=medany' 'CXXFLAGS_FOR_TARGET=-O2  -mcmodel=medany' CC=gcc CXX=g++
Thread model: posix
gcc version 8.1.0 (C-SKY RISCV Tools V1.8.4 B20200702)

Example compilation

$ cd sample/console
$ mkdir build && cd build
$ cmake ..
$ make -j4

$ export LD_LIBRARY_PATH=<project root>/tengine/linux-rv64/lib/

Example running

$ ./apps/test_classification -m <your/sample/path>/tengine/squeezenet.tmfile -i <your/sample/path>/tengine/data/cat.jpg

$ ./apps/test_yolofastest -m <your/sample/path>/tengine/data/yolo-fastest-1.1.tmfile -i <your/sample/path>/tengine/data/ssd_dog.jpg

Precautions

Model support limitations

This release is the technical preview version of Allwinner D1-H. It is recommended to run smaller models, such as MobileNet v1, MobileNet v2, ResNet18, SqueezeNet v1.1, YOLO-Fastest.

Vector calculation support

The system pre-installed on Allwinner D1-H development board does not support the operation of RSIC-V instruction set code. You need to copy the library files in the tool chain sysroot/lib64vxthread to the D1-H root directory.

Tengine technical support

Tengine open source community: https://github.com/OAID/Tengine