333|2

37

帖子

0

TA的资源

一粒金砂(中级)

#AI挑战营第二站# 基于RKNN toolkit的模型转换与验证(续,含代码) [复制链接]

本帖最后由 hollyedward 于 2024-5-30 03:37 编辑

接上一篇文章,可能我使用的服务器的cpu并不支持某些操作,由于rknn也没有开源的sdk,不知道怎么调试,所以我直接租用别的服务器。

首先,安装环境,步骤不用赘述

image.png  

 

模型的导出

 

首先是导出脚本

# filename: onnx2rknn.py


import numpy as np
from rknn.api import RKNN


if __name__ == '__main__':

  # 模型部署平台
  platform = 'rv1106'
  #训练模拟时输入图片大小
  Width = 28
  Height = 28
  # 此处改为自己的模型地址
  MODEL_PATH = '/home/ljl/mnist/mnist_cnn_model.onnx'
  # 导出模型地址
  RKNN_MODEL_PATH = '/home/ljl/mnist/mnist_cnn_model.rknn'
  # 创建RKNN对象并在屏幕打印详细的日志信息
  rknn = RKNN(verbose=True)
  # 模型配置
  # mean_values: 输入图像像素均值
  # std_values: 输入图像像素标准差
  # target_platform: 目标部署平台
  # 本模型训练时输入图象为单通道
  rknn.config(mean_values=[0], std_values=[255], target_platform=platform)
  # 模型加载
  print('--> Loading model')
  ret = rknn.load_onnx(MODEL_PATH) 
  if ret != 0:
      print('load model failed!')
      exit(ret)
  print('done')
  # 构建 RKNN 模型
  print('--> Building model')
  #do_quantization:是否对模型进行量化。默认值为 True
  ret = rknn.build(do_quantization=True, dataset="./data.txt")
  if ret != 0:
     print('build model failed.')
     exit(ret)
  print('done')

  # 导出模型
  ret = rknn.export_rknn(RKNN_MODEL_PATH)
  #释放RKNN模型
  rknn.release()

 

然后要将测试的图片放在文件夹里,路径写在data.txt 文件中

首先要获取mnist的图片用于测试

这里提供导出脚本

# filename: generate_data.py


import torch
import torchvision
import torchvision.transforms as transforms
import torchvision.datasets as datasets
from torch.utils.data import DataLoader
import cv2
import os
import numpy as np

# Test set
test_set = datasets.MNIST('dataset/', train=False, transform=transforms.ToTensor(), download=True)
test_loader = DataLoader(dataset=test_set, batch_size=1, shuffle=True)


def mnist_save_png():
    for data, i in test_loader:
        with torch.no_grad():
            image = data.squeeze().numpy()  # Remove unnecessary transpose

            # Optional: If you need to move channel dimension to the last position
            # image = np.transpose(image, (1, 2, 0))

            image = cv2.GaussianBlur(image, (9, 9), 0)
            # image *= 255  # Scale image to 0-255 range

            index = i.numpy()[0]

            if not os.path.exists('./mnist_image/'):
                os.mkdir('./mnist_image/')
                
            # 每张图片只保存一次
            if not os.path.exists('./mnist_image/' + str(index) + '.png'):
                cv2.imwrite('./mnist_image/' + str(index) + '.png', image)


if __name__ == '__main__':
    mnist_save_png()

 

导出后的效果,分辨率为28 * 28,代码也对应前面文章的

这里只是保存了10张图片,如果要更多图片测试需要修改代码

image.png  

 

准备好后运行脚本即可

 

模型转换打印的过程。

感觉rknn至少比全志系的好用些(又不开源,还不允许个人使用),虽然不是开源的。

I rknn-toolkit2 version: 2.0.0b0+9bab5682
--> Loading model
I It is recommended onnx opset 19, but your onnx model opset is 17!
I Model converted from pytorch, 'opset_version' should be set 19 in torch.onnx.export for successful convert!
I Loading : 100%|██████████████████████████████████████████████████| 5/5 [00:00<00:00, 24966.10it/s]
done
--> Building model
D base_optimize ...
D base_optimize done.
D 
D fold_constant ...
D fold_constant done.
D 
D correct_ops ...
D correct_ops done.
D 
D fuse_ops ...
W build: Can not find 'idx' to insert, default insert to 0!
D fuse_ops results:
D     replace_reshape_gemm_by_conv: remove node = ['/Reshape', '/fc1/Gemm'], add node = ['/fc1/Gemm_2conv', '/fc1/Gemm_2conv_reshape']
D     swap_reshape_relu: remove node = ['/fc1/Gemm_2conv_reshape', '/Relu'], add node = ['/Relu', '/fc1/Gemm_2conv_reshape']
D     convert_gemm_by_conv: remove node = ['/fc2/Gemm'], add node = ['/fc2/Gemm_2conv_reshape1', '/fc2/Gemm_2conv', '/fc2/Gemm_2conv_reshape2']
D     fuse_two_reshape: remove node = ['/fc1/Gemm_2conv_reshape']
D     remove_invalid_reshape: remove node = ['/fc2/Gemm_2conv_reshape1']
D     fold_constant ...
D     fold_constant done.
D fuse_ops done.
D 
D sparse_weight ...
D sparse_weight done.
D 
I GraphPreparing : 100%|████████████████████████████████████████████| 4/4 [00:00<00:00, 5403.29it/s]
I Quantizating : 100%|███████████████████████████████████████████████| 4/4 [00:00<00:00, 283.34it/s]
D 
D quant_optimizer ...
D quant_optimizer results:
D     adjust_relu: ['/Relu']
D quant_optimizer done.
D 
W build: The default input dtype of 'onnx::Reshape_0' is changed from 'float32' to 'int8' in rknn model for performance!
                       Please take care of this change when deploy rknn model with Runtime API!
W build: The default output dtype of '15' is changed from 'float32' to 'int8' in rknn model for performance!
                      Please take care of this change when deploy rknn model with Runtime API!
I rknn building ...
I RKNN: [00:09:32.440] compress = 0, conv_eltwise_activation_fuse = 1, global_fuse = 1, multi-core-model-mode = 7, output_optimize = 1, layout_match = 1, enable_argb_group = 0
I RKNN: librknnc version: 2.0.0b0 (35a6907d79@2024-03-24T02:34:11)
D RKNN: [00:09:32.440] RKNN is invoked
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNExtractCustomOpAttrs
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNExtractCustomOpAttrs
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNSetOpTargetPass
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNSetOpTargetPass
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNBindNorm
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNBindNorm
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNAddFirstConv
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNAddFirstConv
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNEliminateQATDataConvert
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNEliminateQATDataConvert
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTileGroupConv
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTileGroupConv
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTileFcBatchFuse
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTileFcBatchFuse
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNAddConvBias
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNAddConvBias
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTileChannel
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTileChannel
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNPerChannelPrep
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNPerChannelPrep
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNBnQuant
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNBnQuant
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNFuseOptimizerPass
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNFuseOptimizerPass
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNTurnAutoPad
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNTurnAutoPad
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNInitRNNConst
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNInitRNNConst
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNInitCastConst
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNInitCastConst
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNMultiSurfacePass
D RKNN: [00:09:32.442] <<<<<<<< end: rknn::RKNNMultiSurfacePass
D RKNN: [00:09:32.442] >>>>>> start: rknn::RKNNReplaceConstantTensorPass
D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNReplaceConstantTensorPass
D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNSubgraphManager
D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNSubgraphManager
D RKNN: [00:09:32.443] >>>>>> start: OpEmit
D RKNN: [00:09:32.443] <<<<<<<< end: OpEmit
D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNLayoutMatchPass
I RKNN: [00:09:32.443] AppointLayout: t->setNativeLayout(64), tname:[/fc1/Gemm_output_0_new]
I RKNN: [00:09:32.443] AppointLayout: t->setNativeLayout(64), tname:[15_conv]
I RKNN: [00:09:32.443] AppointLayout: t->setNativeLayout(0), tname:[15]
D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNLayoutMatchPass
D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNAddSecondaryNode
D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNAddSecondaryNode
D RKNN: [00:09:32.443] >>>>>> start: OpEmit
D RKNN: [00:09:32.443] finish initComputeZoneMap
D RKNN: [00:09:32.443] <<<<<<<< end: OpEmit
D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNSubGraphMemoryPlanPass
D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNSubGraphMemoryPlanPass
D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNProfileAnalysisPass
D RKNN: [00:09:32.443] node: Reshape:/fc2/Gemm_2conv_reshape2, Target: NPU
D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNProfileAnalysisPass
D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNOperatorIdGenPass
D RKNN: [00:09:32.443] <<<<<<<< end: rknn::RKNNOperatorIdGenPass
D RKNN: [00:09:32.443] >>>>>> start: rknn::RKNNWeightTransposePass
W RKNN: [00:09:32.444] Warning: Tensor /fc2/Gemm_2conv_reshape2_shape need paramter qtype, type is set to float16 by default!
W RKNN: [00:09:32.444] Warning: Tensor /fc2/Gemm_2conv_reshape2_shape need paramter qtype, type is set to float16 by default!
D RKNN: [00:09:32.444] <<<<<<<< end: rknn::RKNNWeightTransposePass
D RKNN: [00:09:32.444] >>>>>> start: rknn::RKNNCPUWeightTransposePass
D RKNN: [00:09:32.444] <<<<<<<< end: rknn::RKNNCPUWeightTransposePass
D RKNN: [00:09:32.444] >>>>>> start: rknn::RKNNModelBuildPass
D RKNN: [00:09:32.446] <<<<<<<< end: rknn::RKNNModelBuildPass
D RKNN: [00:09:32.446] >>>>>> start: rknn::RKNNModelRegCmdbuildPass
D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
D RKNN: [00:09:32.446]                                                         Network Layer Information Table                                                     
D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
D RKNN: [00:09:32.446] ID   OpType           DataType Target InputShape                               OutputShape            Cycles(DDR/NPU/Total)    RW(KB)       FullName        
D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
D RKNN: [00:09:32.446] 0    InputOperator    INT8     CPU    \                                        (1,1,28,28)            0/0/0                    0            InputOperator:onnx::Reshape_0
D RKNN: [00:09:32.446] 1    ConvRelu         INT8     NPU    (1,1,28,28),(50,1,28,28),(50)            (1,50,1,1)             6585/12544/12544         39           Conv:/fc1/Gemm_2conv
D RKNN: [00:09:32.446] 2    Conv             INT8     NPU    (1,50,1,1),(10,50,1,1),(10)              (1,10,1,1)             138/64/138               0            Conv:/fc2/Gemm_2conv
D RKNN: [00:09:32.446] 3    Reshape          INT8     NPU    (1,10,1,1),(2)                           (1,10)                 7/0/7                    0            Reshape:/fc2/Gemm_2conv_reshape2
D RKNN: [00:09:32.446] 4    OutputOperator   INT8     CPU    (1,10)                                   \                      0/0/0                    0            OutputOperator:15
D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
D RKNN: [00:09:32.446] <<<<<<<< end: rknn::RKNNModelRegCmdbuildPass
D RKNN: [00:09:32.446] >>>>>> start: rknn::RKNNFlatcModelBuildPass
D RKNN: [00:09:32.446] Export Mini RKNN model to /tmp/tmpkbgrb68z/check.rknn
D RKNN: [00:09:32.446] >>>>>> end: rknn::RKNNFlatcModelBuildPass
D RKNN: [00:09:32.446] >>>>>> start: rknn::RKNNMemStatisticsPass
D RKNN: [00:09:32.446] ------------------------------------------------------------------------------------------------------------------------------
D RKNN: [00:09:32.446]                                           Feature Tensor Information Table                               
D RKNN: [00:09:32.446] --------------------------------------------------------------------------------------------+---------------------------------
D RKNN: [00:09:32.446] ID  User           Tensor                 DataType  DataFormat   OrigShape    NativeShape   |     [Start       End)       Size
D RKNN: [00:09:32.446] --------------------------------------------------------------------------------------------+---------------------------------
D RKNN: [00:09:32.446] 1   ConvRelu       onnx::Reshape_0        INT8      NC1HWC2      (1,1,28,28)  (1,1,28,28,1) | 0x00027500 0x00027880 0x00000380
D RKNN: [00:09:32.446] 2   Conv           /fc1/Gemm_output_0_new INT8      NC1HWC2      (1,50,1,1)   (1,4,1,1,16)  | 0x00027880 0x000278c0 0x00000040
D RKNN: [00:09:32.446] 3   Reshape        15_conv                INT8      NC1HWC2      (1,10,1,1)   (1,1,1,1,16)  | 0x00027500 0x00027510 0x00000010
D RKNN: [00:09:32.446] 4   OutputOperator 15                     INT8      UNDEFINED    (1,10)       (1,10)        | 0x00027580 0x000275c0 0x00000040
D RKNN: [00:09:32.446] --------------------------------------------------------------------------------------------+---------------------------------
D RKNN: [00:09:32.446] -----------------------------------------------------------------------------------------------------
D RKNN: [00:09:32.446]                                  Const Tensor Information Table                    
D RKNN: [00:09:32.446] -------------------------------------------------------------------+---------------------------------
D RKNN: [00:09:32.446] ID  User     Tensor                         DataType  OrigShape    |     [Start       End)       Size
D RKNN: [00:09:32.446] -------------------------------------------------------------------+---------------------------------
D RKNN: [00:09:32.446] 1   ConvRelu fc1.weight                     INT8      (50,1,28,28) | 0x00000000 0x00026480 0x00026480
D RKNN: [00:09:32.446] 1   ConvRelu fc1.bias                       INT32     (50)         | 0x00026480 0x00026680 0x00000200
D RKNN: [00:09:32.446] 2   Conv     fc2.weight                     INT8      (10,50,1,1)  | 0x00026680 0x00026900 0x00000280
D RKNN: [00:09:32.446] 2   Conv     fc2.bias                       INT32     (10)         | 0x00026900 0x00026980 0x00000080
D RKNN: [00:09:32.446] 3   Reshape  /fc2/Gemm_2conv_reshape2_shape INT64     (2)          | 0x00026980*0x000269c0 0x00000040
D RKNN: [00:09:32.446] -------------------------------------------------------------------+---------------------------------
D RKNN: [00:09:32.446] ----------------------------------------
D RKNN: [00:09:32.446] Total Internal Memory Size: 0.9375KB
D RKNN: [00:09:32.446] Total Weight Memory Size: 154.438KB
D RKNN: [00:09:32.446] ----------------------------------------
D RKNN: [00:09:32.446] <<<<<<<< end: rknn::RKNNMemStatisticsPass
I rknn buiding done.
done

 

模型验证

我们需要检验模型的输出是否正确

貌似simulator不支持rknn模型验证

 

报错日志:

image.png   官方提供的demo教程也是使用load_onnx,https://wiki.luckfox.com/zh/Luckfox-Pico/Luckfox-Pico-RKNN-Test

image.png  

 

006-e8172071e0532c4a666d00b99118efb8.png

所以其实可以在导出rknn模型的时候,也可以验证模型

 

脚本如下

# filename: rknn_mnist_test.py
import numpy as np
import cv2
from rknn.api import RKNN

# Model conversion parameters
MODEL_PATH = '/root/test/my_model.onnx'  # Path to the ONNX model
RKNN_MODEL_PATH = '/root/test/my_model.rknn'  # Path to save the RKNN model

# Model inference parameters
input_size = (28, 28)  # Define the input size (same as your model's input)
data_file = 'data.txt'  # Path to the data file (containing image paths and labels)

rknn = RKNN(verbose=True)  # Create RKNN object with verbose logging
rknn.config(mean_values=[0], std_values=[255], target_platform='rv1106')  # Set configuration parameters
ret = rknn.load_onnx(MODEL_PATH)
if ret != 0:
    print('Load ONNX model failed!')
    exit(ret)
print('done')

print('--> Building RKNN model')

ret = rknn.build(do_quantization=True, dataset="./data.txt")
if ret != 0:
    print('Build model failed.')
    exit(ret)
print('done')

# Model export (optional)  #导出rknn模型
ret = rknn.export_rknn(RKNN_MODEL_PATH)


# Model inference
print('--> Performing inference on data')
rknn.init_runtime()  # Initialize RKNN runtime
with open(data_file, 'r') as f:
    lines = f.readlines()

    for line in lines:
        # Get image path and label
        image_path = line.strip()
        
        # Read the image
        image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)

        # Preprocess the image
        image = image.astype(np.float32)
        
        # image 这时候读取出来是 (28, 28), 需要增加维度
        image = np.expand_dims(image, axis=[0,1])
        
        # Run inference
        outputs = rknn.inference([image], data_format = 'nchw') #(1, 1, 28, 28) 对应nchw,批次,通道,长,宽
        print(f"Ineference Output: {outputs}")
        # Check inference results
        if outputs is not None:
            predicted_label = np.argmax(outputs)
            print(f"Image: {image_path}")
            print(f"Predicted label: {predicted_label}")
        else:
            print(f"Inference failed for image: {image_path}")

# Release RKNN resources
rknn.release()

打印的结果

可以看到推理的结果是正确的

image.png  

 

 

images.rar

5.79 KB, 下载次数: 0

售价: 5 分芯积分  [记录]  [购买]

用于测试的mnist图片文件

my_model.onnx

155.95 KB, 下载次数: 0

售价: 5 分芯积分  [记录]  [购买]

my_model.rknn

162.7 KB, 下载次数: 0

售价: 5 分芯积分  [记录]  [购买]

此帖出自ARM技术论坛

最新回复

可以给自己电脑装一个虚拟机的。不一定要用Linux服务器。   详情 回复 发表于 2024-5-30 18:30

回复
举报

6042

帖子

0

TA的资源

五彩晶圆(高级)

knn至少比全志系的好用些,又不开源,还不允许个人使用,虽然不是开源的,感觉用这还行哈

此帖出自ARM技术论坛

回复

5858

帖子

2

TA的资源

版主

可以给自己电脑装一个虚拟机的。不一定要用Linux服务器。

此帖出自ARM技术论坛

回复
您需要登录后才可以回帖 登录 | 注册

随便看看
查找数据手册?

EEWorld Datasheet 技术支持

相关文章 更多>>
关闭
站长推荐上一条 1/10 下一条

About Us 关于我们 客户服务 联系方式 器件索引 网站地图 最新更新 手机版

站点相关: 国产芯 安防电子 汽车电子 手机便携 工业控制 家用电子 医疗电子 测试测量 网络通信 物联网

北京市海淀区中关村大街18号B座15层1530室 电话:(010)82350740 邮编:100190

电子工程世界版权所有 京B2-20211791 京ICP备10001474号-1 电信业务审批[2006]字第258号函 京公网安备 11010802033920号 Copyright © 2005-2024 EEWORLD.com.cn, Inc. All rights reserved
快速回复 返回顶部 返回列表