RV1106手把手教你：惊呆了！USB摄像头秒变AI助手，rknn轻松拍照做yolov5推理！
<div class='showpostmsg'> 本帖最后由 LitchiCheng 于 2024-3-12 23:56 编辑<article data-content="[{"type":"block","id":"3060-1621846615933","name":"paragraph","data":{},"nodes":[{"type":"text","id":"p5PQ-1621846617594","leaves":[{"text":"背景：官方的rknn_yolov5_demo是通过加载一张图片，然后进行推理，实际使用过程中往往需要使用摄像头capture图片后，然后再调用demo进行推理验证，很不方便。","marks":[]}]}],"state":{}},{"type":"block","id":"t1Bw-1709738910591","name":"paragraph","data":{},"nodes":[{"type":"text","id":"oNxs-1709738910590","leaves":[{"text":"故对官网demo进行改造，先将整个yolov5目录复制出来。偷懒的小伙伴可以直接clone我修改好的，github地址如下","marks":[]}]}],"state":{}},{"type":"block","id":"KW6X-1709739033391","name":"paragraph","data":{},"nodes":[{"type":"text","id":"SlCQ-1709739033390","leaves":[{"text":"","marks":[]}]},{"type":"inline","id":"XAVo-1708767960961","name":"link","data":{"href":"https://github.com/LitchiCheng/RV1106_Linux/tree/main"},"nodes":[{"type":"text","id":"Ct0g-1708767960962","leaves":[{"text":"LitchiCheng/RV1106_Linux: Linux test for RV1106 dev board (github.com)","marks":[]}]}]},{"type":"text","id":"G58s-1708767960967","leaves":[{"text":"","marks":[]}]}],"state":{}},{"type":"block","id":"2PXH-1709739047914","name":"paragraph","data":{},"nodes":[{"type":"text","id":"tJuy-1709739047913","leaves":[{"text":"","marks":[]}]},{"type":"inline","id":"p4I5-1709739048148","name":"link","data":{"href":"https://github.com/LitchiCheng/RV1106_Linux"},"nodes":[{"type":"text","id":"vytj-1709739048147","leaves":[{"text":"https://github.com/LitchiCheng/RV1106_Linux","marks":[]}]}]},{"type":"text","id":"hipH-1709739048150","leaves":[{"text":"","marks":[]}]}],"state":{}},{"type":"block","id":"HwFz-1709738954801","name":"paragraph","data":{},"nodes":[{"type":"text","id":"KkII-1709738954800","leaves":[{"text":"下面介绍几个需要特别修改的地方：","marks":[]}]}],"state":{}},{"type":"block","id":"jUfM-1709739055007","name":"paragraph","data":{},"nodes":[{"type":"text","id":"tlGC-1709739055006","leaves":[{"text":"修改工程主目录下的CMakeLists.txt","marks":[]}]}],"state":{}},{"type":"block","id":"CvIQ-1709739187161","name":"paragraph","data":{},"nodes":[{"type":"text","id":"L4jH-1709739187160","leaves":[{"text":"指定交叉编译器","marks":[]}]}],"state":{}}]">
<p><iframe allowfullscreen="true" frameborder="0" height="450" src="//player.bilibili.com/player.html?bvid=1Ci421d7q7&page=1" style="background:#eee;margin-bottom:10px;" width="700"></iframe><br />
背景：官方的rknn_yolov5_demo是通过加载一张图片，然后进行推理，实际使用过程中往往需要使用摄像头capture图片后，然后再调用demo进行推理验证，很不方便。</p>
<p>故对官网demo进行改造，先将整个yolov5目录复制出来。偷懒的小伙伴可以直接clone我修改好的，github地址如下</p>
<p><a href="https://github.com/LitchiCheng/RV1106_Linux/tree/main">LitchiCheng/RV1106_Linux: Linux test for RV1106 dev board (github.com)</a></p>
<p><a href="https://github.com/LitchiCheng/RV1106_Linux">https://github.com/LitchiCheng/RV1106_Linux</a></p>
<p>下面介绍几个需要特别修改的地方：</p>
<p>修改工程主目录下的CMakeLists.txt</p>
<p>指定交叉编译器</p>
<pre>
<code>set (CMAKE_C_COMPILER "/mnt/d/luckfox-pico/tools/linux/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-gcc")
set (CMAKE_CXX_COMPILER "/mnt/d/luckfox-pico/tools/linux/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-g++")</code></pre>
<article data-content="[{"type":"block","id":"wIDF-1709739166283","name":"paragraph","data":{},"nodes":[{"type":"text","id":"IOvO-1709739166282","leaves":[{"text":"增加预编译变量的声明","marks":[]}]}],"state":{}}]">
<p>增加预编译变量的声明</p>
<pre>
<code>add_definitions(-DRV1106_1103)</code></pre>
<article data-content="[{"type":"block","id":"47zP-1709738977976","name":"paragraph","data":{"style":{"textIndent":0}},"nodes":[{"type":"text","id":"51cE-1709738977977","leaves":[{"text":"增加我们上期分享的v4l2的工具类","marks":[]}]}],"state":{}}]">
<p>增加我们上期分享的v4l2的工具类</p>
<pre>
<code>add_executable(${PROJECT_NAME}
main.cc
postprocess.cc
../v4l2/tools/v4l2CapPicTool.cpp
${rknpu2_yolov5_file}
)</code></pre>
<article data-content="[{"type":"block","id":"Ldsh-1709739103034","name":"paragraph","data":{},"nodes":[{"type":"text","id":"AefW-1709739103035","leaves":[{"text":"修改cmake安装路径，很多，这里只放一个，具体看整个工程","marks":[]}]}],"state":{}}]">
<p>修改cmake安装路径，很多，这里只放一个，具体看整个工程</p>
<pre>
<code>install(TARGETS ${PROJECT_NAME} DESTINATION ${OUTPUTPATH})</code></pre>
<article data-content="[{"type":"block","id":"jDaA-1709739127970","name":"paragraph","data":{},"nodes":[{"type":"text","id":"sokO-1709739127971","leaves":[{"text":"修改RV1106_Linux/yolov5/3rdparty下的CMakeLists.txt","marks":[]}]}],"state":{}},{"type":"block","id":"disI-1709739277227","name":"paragraph","data":{},"nodes":[{"type":"text","id":"eGiw-1709739277226","leaves":[{"text":"直接设置架构类型为armhf_uclibc","marks":[]}]}],"state":{}}]">
<p>修改RV1106_Linux/yolov5/3rdparty下的CMakeLists.txt</p>
<p>直接设置架构类型为armhf_uclibc</p>
<pre>
<code>if (CMAKE_C_COMPILER MATCHES "uclibc")
set (TARGET_LIB_ARCH ${TARGET_LIB_ARCH}_uclibc)
endif()</code></pre>
<article data-content="[{"type":"block","id":"91cQ-1709739274913","name":"paragraph","data":{},"nodes":[{"type":"text","id":"DHyW-1709739274914","leaves":[{"text":"修改runtime库，直接用armhf_uclibc版本","marks":[]}]}],"state":{}}]">
<p>修改runtime库，直接用armhf_uclibc版本</p>
<pre>
<code># rknn runtime
set(RKNN_PATH ${CMAKE_CURRENT_SOURCE_DIR}/rknpu2)
set(LIBRKNNRT ${RKNN_PATH}/${CMAKE_SYSTEM_NAME}/armhf-uclibc/librknnmrt.so)
set(LIBRKNNRT_INCLUDES ${RKNN_PATH}/include PARENT_SCOPE)
install(PROGRAMS ${LIBRKNNRT} DESTINATION ${OUTPUTPATH}/lib)
set(LIBRKNNRT ${LIBRKNNRT} PARENT_SCOPE)</code></pre>
<article data-content="[{"type":"block","id":"7oud-1709739326214","name":"paragraph","data":{},"nodes":[{"type":"text","id":"wrFO-1709739326215","leaves":[{"text":"然后修改main.cc，如下，增加对v4l2的调用","marks":[]}]}],"state":{}}]">
<p>然后修改main.cc，如下，增加对v4l2的调用</p>
<pre>
<code>int main(int argc, char **argv)
{
if (argc != 3)
{
printf("%s <model_path> <camera_path>\n", argv);
return -1;
}
const char *model_path = argv;
const char *image_path = "tmp.jpg";
std::string camera_path = std::string(argv);
v4l2CapPicTool vt(camera_path, 1080, 960, "jpg");
vt.init();
int ret;
rknn_app_context_t rknn_app_ctx;
memset(&rknn_app_ctx, 0, sizeof(rknn_app_context_t));
init_post_process();
ret = init_yolov5_model(model_path, &rknn_app_ctx);
if (ret != 0)
{
printf("init_yolov5_model fail! ret=%d model_path=%s\n", ret, model_path);
goto out;
}
vt.capture();
vt.save("tmp.jpg");
image_buffer_t src_image;
memset(&src_image, 0, sizeof(image_buffer_t));
ret = read_image("tmp.jpg", &src_image);
#if defined(RV1106_1103)
//RV1106 rga requires that input and output bufs are memory allocated by dma
ret = dma_buf_alloc(RV1106_CMA_HEAP_PATH, src_image.size, &rknn_app_ctx.img_dma_buf.dma_buf_fd,
(void **) & (rknn_app_ctx.img_dma_buf.dma_buf_virt_addr));
memcpy(rknn_app_ctx.img_dma_buf.dma_buf_virt_addr, src_image.virt_addr, src_image.size);
dma_sync_cpu_to_device(rknn_app_ctx.img_dma_buf.dma_buf_fd);
free(src_image.virt_addr);
src_image.virt_addr = (unsigned char *)rknn_app_ctx.img_dma_buf.dma_buf_virt_addr;
#endif
//.....
return 0;
}
</code></pre>
<article data-content="[{"type":"block","id":"7d3j-1709735831615","name":"paragraph","data":{},"nodes":[{"type":"text","id":"nnA7-1709735831616","leaves":[{"text":"创建构建目录","marks":[]}]}],"state":{}}]">
<p>创建构建目录</p>
<pre>
<code>mkdir build
cd build
cmake ..
make
make install </code></pre>
<p> </p>
<article data-content="[{"type":"block","id":"0xZL-1709739374860","name":"paragraph","data":{},"nodes":[{"type":"text","id":"MSQS-1709739374859","leaves":[{"text":"生成的文件都放在主目录下的output","marks":[]}]}],"state":{}}]">
<p>生成的文件都放在主目录下的output</p>
<p> 复制到开发板中，运行测试</p>
<article data-content="[{"type":"block","id":"1dox-1709739392651","name":"paragraph","data":{},"nodes":[{"type":"text","id":"DvpV-1709739392650","leaves":[{"text":"复制到开发板中，运行测试","marks":[]}]}],"state":{}}]">
<pre>
<code>pico@luckfox:~/output$ sudo ./rknn_yolov5_realtime model/yolov5s_relu.rknn /dev/video9
init success
load lable ./model/coco_80_labels_list.txt
model input num: 1, output num: 3
input tensors:
index=0, name=images, n_dims=4, dims=, n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
index=0, name=output, n_dims=4, dims=, n_elems=1632000, size=1632000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003860
index=1, name=283, n_dims=4, dims=, n_elems=408000, size=408000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=2, name=285, n_dims=4, dims=, n_elems=102000, size=102000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003915
model is NHWC input fmt
model input height=640, width=640, channel=3
origin size=960x720 crop size=960x720
input image: 960 x 720, subsampling: 4:2:2, colorspace: YCbCr, orientation: 1
scale=0.666667 dst_box=(0 80 639 559) allow_slight_change=1 _left_offset=0 _top_offset=80 padding_w=0 padding_h=160
src width=960 height=720 fmt=0x1 virAddr=0x0x558b65c400 fd=0
dst width=640 height=640 fmt=0x1 virAddr=0x0x558b856810 fd=0
src_box=(0 0 959 719)
dst_box=(0 80 639 559)
color=0x72
fill dst image (x y w h)=(0 0 640 640) with color=0x72727272
rga_api version 1.10.0_
rknn_run
book @ (268 189 544 457) 0.581
book @ (274 4 537 255) 0.522
book @ (238 372 555 637) 0.481
book @ (216 472 502 697) 0.477
tv @ (759 0 958 586) 0.357
suitcase @ (580 48 693 126) 0.223
write_image path: out.png width=960 height=720 channel=3 data=0x558b65c400
</code></pre>
<p> </p>
</article>
</article>
</article>
</article>
</article>
</article>
</article>
</article>
</article>
</article>
</div><script> var loginstr = '<div class="locked">查看精华帖全部内容，请<a href="javascript:;" style="color:#e60000" class="loginf">登录</a>或者<a href="https://bbs.eeworld.com.cn/member.php?mod=register_eeworld.php&action=wechat" style="color:#e60000" target="_blank">注册</a></div>';
if(parseInt(discuz_uid)==0){
(function($){
var postHeight = getTextHeight(400);
$(".showpostmsg").html($(".showpostmsg").html());
$(".showpostmsg").after(loginstr);
$(".showpostmsg").css({height:postHeight,overflow:"hidden"});
})(jQuery);
}
</script><script type="text/javascript">(function(d,c){var a=d.createElement("script"),m=d.getElementsByTagName("script"),eewurl="//counter.eeworld.com.cn/pv/count/";a.src=eewurl+c;m.parentNode.insertBefore(a,m)})(document,523)</script> <p>对摄像头拍的实时画面进行推理吗？</p>
<p>我这方面不懂，楼主，加载一张图片，然后进行推理，是什么意思？ 推理什么？？能详细讲一下吗？？？</p>
怀揣少年梦 发表于 2024-3-13 10:58
对摄像头拍的实时画面进行推理吗？
<p>对，改成循环就可以了</p>
tagetage 发表于 2024-3-13 10:58
我这方面不懂，楼主，加载一张图片，然后进行推理，是什么意思？ 推理什么？？能详细讲一下吗？？？
<p>这个可以搜一下哈，DL方面的，一句两句讲不清楚</p>
<p>推理的时候用NPU了嘛。</p>
LitchiCheng 发表于 2024-3-13 11:19
对，改成循环就可以了
<p>学习一下</p>
wangerxian 发表于 2024-3-13 11:58
推理的时候用NPU了嘛。
<p>rknn，可以看视频，里面有讲</p>
<p>rknn的算力有多少？实时的话会不会卡？</p>
<p>先收下，感谢楼主</p>
cruise__tom123 发表于 2024-3-13 16:59
先收下，感谢楼主
<p>一键三连了吗，哈哈哈哈</p>
freebsder 发表于 2024-3-13 15:45
rknn的算力有多少？实时的话会不会卡？
<p> 0.5 t，应该不会卡，推理没啥问题</p>
<p>lyh@lyh-pc:~/RV1106_Linux-main/yolov5/build$ cmake ..<br />
-- The C compiler identification is unknown<br />
-- The CXX compiler identification is unknown<br />
CMake Error at CMakeLists.txt:6 (project):<br />
The CMAKE_C_COMPILER:</p>
<p> /mnt/d/luckfox-pico/tools/linux/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-gcc</p>
<p> is not a full path to an existing compiler tool.</p>
<p> Tell CMake where to find the compiler by setting either the environment<br />
variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to<br />
the compiler, or to the compiler name if it is in the PATH.</p>
<p><br />
CMake Error at CMakeLists.txt:6 (project):<br />
The CMAKE_CXX_COMPILER:</p>
<p> /mnt/d/luckfox-pico/tools/linux/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-g++</p>
<p> is not a full path to an existing compiler tool.</p>
<p> Tell CMake where to find the compiler by setting either the environment<br />
variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path<br />
to the compiler, or to the compiler name if it is in the PATH.</p>
<p><br />
-- Configuring incomplete, errors occurred!<br />
See also "/home/lyh/RV1106_Linux-main/yolov5/build/CMakeFiles/CMakeOutput.log".<br />
See also "/home/lyh/RV1106_Linux-main/yolov5/build/CMakeFiles/CMakeError.log".<br />
CMake无法找到正确的C和C++编译器<br />
请问这个怎么解决</p>
li-ai-meng 发表于 2024-4-26 13:18
lyh@lyh-pc:~/RV1106_Linux-main/yolov5/build$ cmake ..
-- The C compiler identification is unknown
...
<p>/mnt/d/luckfox-pico/tools/linux/toolchain/arm-rockchip830-linux-uclibcgnueabihf/bin/arm-rockchip830-linux-uclibcgnueabihf-gcc</p>
<p>修改你对应sdk的位置，这个是我电脑的位置</p>
本帖最后由 fimai 于 2024-10-31 17:30 编辑
<p>运行时报错，是什么问题</p>
<p>load lable ./model/coco_80_labels_list.txt<br />
E RKNN: failed to decode config data!<br />
Segmentation fault (core dumped)</p>
fimai 发表于 2024-10-31 14:26
运行时报错，是什么问题
load lable ./model/coco_80_labels_list.txt
E RKNN: failed to decode confi ...
<p>越界之类的吧</p>
<p>ubuntu@ubuntu:~/luckfox-pico/RV1106_Linux/yolov5/build$ make<br />
[ 16%] Built target imageutils<br />
[ 33%] Built target fileutils<br />
[ 50%] Built target imagedrawing<br />
make: *** No rule to make target '/home/ubuntu/luckfox-pico/RV1106_Linux/sources/3rdparty/rknpu2/Linux/armhf-uclibc/librknnmrt.so', needed by 'rknn_yolov5_realtime'. Stop.<br />
make: *** Error 2<br />
make: *** Error 2<br />
这是为啥呀</p>
LitchiCheng 发表于 2024-3-13 11:19
对，改成循环就可以了
<p>改成循环的要怎么改，我改了之后，运行一会就报RK_DMA_HEAP_ALLOC_BUFFER failed:Cannot allocate memory错误，不知道哪里没释放</p>
页:
[1]