【玄铁杯第三届RISC-V应用创新大赛】LicheePi 4A 基于Yolov5的姿态识别模型的选择...

ccccccc@ · 发表于2023-10-25 17:10

【玄铁杯第三届RISC-V应用创新大赛】LicheePi 4A 基于Yolov5的姿态识别模型的选择... [复制链接]

<i class="pstatus"> 本帖最后由 ccccccc@ 于 2023-10-26 09:49 编辑 </i>

本次选择的是现成最佳模型：
[ Image-based-Human-Fall-Detection ](https://github.com/uttej2001/Image-based-Human-Fall-Detection " Image-based-Human-Fall-Detection ")

<ignore_js_op>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_743736" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=743736&handlekey=savephoto_743736">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

可以在电脑上对模型进行简单验证：
```shell
#首先需要安装虚拟环境，anconda或者miniconda，具体请百度
#创建虚拟环境
conda create -n sample#名字爱怎么取怎么取，简单好记
source activate#激活虚拟环境
conda activate sample#进入虚拟环境
#进入到本地yolov5路径下
cd /home/chen/work/nfs/yolov5n/yolov5
#如果没有就进行下载
#git clone https://github.com/ultralytics/yolov5.git
#cd yolov5
pip3 install -r requirements.txt#布置环境
```
将下面图片另存为本地，命名为`kite.jpg`

<ignore_js_op>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_743749" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=743749&handlekey=savephoto_743749">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

进行操作：
```shell
python detect.py --source ./kite.jpg --weights ./best.pt--device cpu
```
结果打印如下：
```shell
(sample) chen@ubuntu:~/work/nfs/yolov5n/yolov5$ python detect.py --source ./kite.jpg --weights ./best.pt--device cpu
detect: weights=['./best.pt'], source=./kite.jpg, data=data/coco128.yaml, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=cpu, view_img=False, save_txt=False, save_csv=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False, dnn=False, vid_stride=1
YOLOv5 🚀 v7.0-227-ge4df1ec Python-3.11.5 torch-2.1.0+cu121 CPU

Fusing layers... 
Model summary: 213 layers, 7018216 parameters, 0 gradients, 15.8 GFLOPs
image 1/1 /home/chen/work/nfs/yolov5n/yolov5/kite.jpg: 448x640 3 walkings, 304.4ms
Speed: 40.6ms pre-process, 304.4ms inference, 111.8ms NMS per image at shape (1, 3, 640, 640)
Results saved to runs/detect/exp2
```

<ignore_js_op>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744444" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744444&handlekey=savephoto_744444">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

模型可以正常运行，开始进行pt向onnx的转化。

参考hhb进行转化：
[HHB 用户手册](https://www.yuque.com/za4k4z/oxlbxl/xgm5bybl2v3stf0r#qbIEY "HHB 用户手册")

因为机器进行环境安装时候，很多python库安装不了，加上docker镜像中git不了，我就选择直接在电脑,直接在电脑中进行下载，再复制到docker中进行操作，再传给机器，或者也可以考虑在镜像启动时候挂载本地文件夹
```shell
#先进入电脑Docker镜像中
sudo docker start hhb2.4#激活镜像
#下载姿态识别模型
cd /home/chen/work/nfs/yolov5n/yolov5
git clone https://github.com/uttej2001/Image-based-Human-Fall-Detection fall
#将本地下载的文件复制到镜像系统中
sudo docker cp /home/chen/work/nfs/yolov5n/yolov5/fall hhb2.4:/home/example/th1520_npu/yolov5n/yolov5/fall
#进入镜像
sudo docker exec -it hhb2.4 /bin/bash
cd /home/example/th1520_npu/yolov5n/yolov5
#将pt文件转化为onnx：
python3 export.py --weights ./fall/best.pt--data ../fall/custom_data.yaml --include onnx--imgsz 384 640
```

<ignore_js_op>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744398" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744398&handlekey=savephoto_744398">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

```shell
#将以下内容写入run.sh
hhb -S --model-file ./fall/best.onnx --data-scale 0.017 --data-mean '124 117 104' --board c920 --input-name images --output-name '/model.24/m.0/Conv_output_0;/model.24/m.1/Conv_output_0;/model.24/m.2/Conv_output_0' --input-shape '2 3 384 640' --postprocess save_and_top5 --simulate-data ../kite.jpg
[2023-10-25 07:45:48] (HHB LOG): Start import model.
```

打印如下
```shell
./run.sh 
+ hhb -S --model-file ./fall/best.onnx --data-scale 0.017 --data-mean '124 117 104' --board c920 --input-name images --output-name '/model.24/m.0/Conv_output_0;/model.24/m.1/Conv_output_0;/model.24/m.2/Conv_output_0' --input-shape '2 3 384 640' --postprocess save_and_top5 --simulate-data ../kite.jpg
[2023-10-25 07:45:48] (HHB LOG): Start import model.
[2023-10-25 07:45:51] (HHB LOG): Model import completed! 
[2023-10-25 07:45:51] (HHB LOG): Start quantization.
[2023-10-25 07:45:51] (HHB LOG): Start optimization.
[2023-10-25 07:45:51] (HHB LOG): Optimization completed!
[2023-10-25 07:45:51] (HHB LOG): Start conversion to csinn.
[2023-10-25 07:45:52] (HHB LOG): Conversion completed!
[2023-10-25 07:45:52] (HHB LOG): Start operator fusion.
[2023-10-25 07:45:52] (HHB LOG): Operator fusion completed!
[2023-10-25 07:45:52] (HHB LOG): Start operator split.
[2023-10-25 07:45:52] (HHB LOG): Operator split completed!
[2023-10-25 07:45:52] (HHB LOG): Start layout convert.
[2023-10-25 07:45:52] (HHB LOG): Layout convert completed!
[2023-10-25 07:45:52] (HHB LOG): Quantization completed!
[2023-10-25 07:46:09] (HHB LOG): cd hhb_out; qemu-riscv64 -cpu c920 hhb_runtime ./hhb.bm kite.jpg.0.bin 
Run graph execution time: 199610.84375ms, FPS=0.01

=== tensor info ===
shape: 2 3 384 640 
data pointer: 0x11a92c0

=== tensor info ===
shape: 2 24 48 80 
data pointer: 0x3d9270
The max_value of output: nan
The min_value of output: nan
The mean_value of output: nan
The std_value of output: nan
 ============ top5: ===========
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000

=== tensor info ===
shape: 2 24 24 40 
data pointer: 0x2156c0
The max_value of output: nan
The min_value of output: nan
The mean_value of output: nan
The std_value of output: nan
 ============ top5: ===========
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000

=== tensor info ===
shape: 2 24 12 20 
data pointer: 0x251880
The max_value of output: nan
The min_value of output: nan
The mean_value of output: nan
The std_value of output: nan
 ============ top5: ===========
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000

```
生成文件如下：

<ignore_js_op>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744438" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744438&handlekey=savephoto_744438">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

其中 hhb_runtime 是生成的可执行文件，hhb.bm 是示例程序使用的模型权重，kite.jpg.0.bin 是经过的预处理的输入数据。其他文件的具体含义可查看官方文件定义。
[HHB 用户手册](https://www.yuque.com/za4k4z/oxlbxl/xgm5bybl2v3stf0r#qbIEY "HHB 用户手册")
将上面三个文件复制到机器上：
```shell
scp -r ../yolov5 sipeed@192.168.0.104:/home/sipeed/Desktop/udisk/nfs/yolov5n/
```
使用ssh远程登陆机器进行操作：
```shell
ssh sipeed@192.168.0.104
cd ~/Desktop/udisk/nfs/yolov5n/yolov5
./hhb_out/hhb_runtime ./hhb_out/hhb.bm ./hhb_out/kite.jpg.0.bin
```
打印如下：
```shell
(env) sipeed@lpi4a:~/Desktop/udisk/nfs/yolov5n/yolov5$ ./hhb_out/hhb_runtime ./hhb_out/hhb.bm ./hhb_out/kite.jpg.0.bin 
Run graph execution time: 1607.92810ms, FPS=0.62

=== tensor info ===
shape: 2 3 384 640 
data pointer: 0x11a96b0

=== tensor info ===
shape: 2 24 48 80 
data pointer: 0x3d9660
The max_value of output: nan
The min_value of output: nan
The mean_value of output: nan
The std_value of output: nan
 ============ top5: ===========
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000

=== tensor info ===
shape: 2 24 24 40 
data pointer: 0x215ab0
The max_value of output: nan
The min_value of output: nan
The mean_value of output: nan
The std_value of output: nan
 ============ top5: ===========
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000

=== tensor info ===
shape: 2 24 12 20 
data pointer: 0x251c70
The max_value of output: nan
The min_value of output: nan
The mean_value of output: nan
The std_value of output: nan
 ============ top5: ===========
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
 -1: -169473953031737902729750710990328037376.000000
```
数据异常，尝试使用通用模型部署方式：
#首先进行指令的更换
```shell
hhb -D --model-file ./fall/best.onnx --data-scale 0.017 --data-mean '124 117 104' --board c920 --input-name images --output-name '/model.24/m.0/Conv_output_0;/model.24/m.1/Conv_output_0;/model.24/m.2/Conv_output_0' --input-shape '2 3 384 640' --postprocess save_and_top5 --simulate-data ../kite.jpg
```
将生成的`hhb_out`复制到机器上
#进行操作
```shell
./hhb_out/hhb_runtime ./hhb_out/hhb.bm ../kite.jpg 
Run graph execution time: 1546.36108ms, FPS=0.65

=== tensor info ===
shape: 2 3 384 640 
data pointer: 0xed96a0

=== tensor info ===
shape: 2 24 48 80 
data pointer: 0x3d9660
The max_value of output: 2.798828
The min_value of output: -11.468750
The mean_value of output: -1.485657
The std_value of output: 12.376108
 ============ top5: ===========
38644: 2.798828
38564: 2.746094
38645: 2.6777
38565: 2.648438
38647: 2.617188

=== tensor info ===
shape: 2 24 24 40 
data pointer: 0x215b80
The max_value of output: 3.070312
The min_value of output: -11.781250
The mean_value of output: -1.402809
The std_value of output: 10.823801
 ============ top5: ===========
21699: 3.070312
6339: 2.812500
21659: 2.767578
21619: 2.753906
21739: 2.707031

=== tensor info ===
shape: 2 24 12 20 
data pointer: 0x251d40
The max_value of output: 5.281250
The min_value of output: -10.593750
The mean_value of output: -1.239471
The std_value of output: 7.504400
 ============ top5: ===========
1589: 5.281250
3509: 4.863281
1609: 4.378906
1590: 4.367188
1569: 4.281250
```
打印似乎没有问题

回到镜像中进行交叉编译
```shell
cd /home/example/th1520_npu/yolov5n/yolov5
riscv64-unknown-linux-gnu-gcc ../yolov5n.c -static -o fall_example hhb_out/io.c hhb_out/model.c -Wl,--gc-sections -O2 -g -mabi=lp64d -I hhb_out/ -L /usr/local/lib/python3.8/dist-packages/hhb/install_nn2/c920/lib/ -lshl -static -L /usr/local/lib/python3.8/dist-packages/hhb/prebuilt/decode/install/lib/rv -L /usr/local/lib/python3.8/dist-packages/hhb/prebuilt/runtime/riscv_linux -lprebuilt_runtime -ljpeg -lpng -lz -lstdc++ -lm -I /usr/local/lib/python3.8/dist-packages/hhb/install_nn2/c920/include/ -mabi=lp64d -march=rv64gcv0p7_zfh_xtheadc
```
在本路径下生成 `fall_example`,将其复制到机器上
```shell
scp -r ./fall_example sipeed@192.168.0.104:/home/sipeed/Desktop/udisk/nfs/yolov5n/yolov5 
```
进入机器端的页面
```shell
cd /home/sipeed/Desktop/udisk/nfs/yolov5n
修改`interface.py`第72行`yolov5_example`修改为`./yolov5/fall_example`,`hhb_out`修改为`./yolov5/hhb_out`
将下图另存为本地，命名为kite.jpg
```

<ignore_js_op>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744472" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744472&handlekey=savephoto_744472">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

```shell
#运行，且打印如下
python3 inference.py 
 ********** preprocess image **********
 ******* run yolov5 and postprocess *******
Run graph execution time: 1546.41077ms, FPS=0.65
detect num: 1
id:    label    score        x1        y1        x2        y2
[0]:    0    0.443974    193.455765    81.969955    465.592712    354.700195
 ********** draw bbox **********
[193.455765, 81.969955, 465.592712, 354.700195, 0.443974, 0]
```
结果如下：

<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744474_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>kite_result.jpg</strong> <em class="xg1">(51.15 KB, 下载次数: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >下载附件</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744474" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744474&handlekey=savephoto_744474">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

可以对比看出，由于转换某些精度发生变化，结果也发生一些错误——不能进行姿态的判断。但是在模型的移植和转换已经迈出较为成功的小半步了。

ccccccc@ · 发表于2023-10-25 17:23

但是我不确定具体什么地方操作有问题，有知道的望指点！

wangerxian · 发表于2023-10-25 19:31

可以识别实时视频吗？

nmg · 发表于2023-10-26 09:43

如果换成同一张图片呢？也是只能识别物体，不能识别姿态？

ccccccc@ · 发表于2023-10-26 09:46

nmg 发表于 2023-10-26 09:43 如果换成同一张图片呢？也是只能识别物体，不能识别姿态？

同一张图片，甚至识别物体都有问题

ccccccc@ · 发表于2023-10-26 09:47

wangerxian 发表于 2023-10-25 19:31 可以识别实时视频吗？

额，图片识别目前都还有问题

wangerxian · 发表于2023-10-26 15:11

ccccccc@ 发表于 2023-10-26 09:47 额，图片识别目前都还有问题

哈哈，慢慢找问题吧，总能解决的！

freebsder · 发表于2023-10-26 15:20

女主给个赞！

【玄铁杯第三届RISC-V应用创新大赛】LicheePi 4A 基于Yolov5的姿态识别模型的选择... [复制链接]

最新回复

点评

点评

点评

浏览过的版块