【2024 DigiKey 创意大赛】基于AI图像识别的宠物监控报警装置
本帖最后由 土豆12 于 2024-10-20 21:01 编辑<p><strong>一、作品简介</strong><br />
我在养狗的过程中一只有个困扰，就是狗需要运动，不能把它一直关到笼子里。但家里又有很多区域不想让它进入，但往往一眼没看住他就跑了进去。因此我打算设计一个报警装置，我们可以将其部署在一些不希望宠物进入的区域，比如厨房或阳台。当宠物进入该区域并被摄像头捕捉到后，通过放置在其他区域的ESP32-S3模块播放声音，同时使用ESP32-C6模块控制灯光闪烁，以此吸引宠物离开区域。</p>
<p><strong>二、系统框图</strong><br />
项目的技术实现可以分为下面几个部分。</p>
<p>1，物联网中枢平台，我计划是用部署在树莓派zero2W上的MQTT 服务器。</p>
<p>2，其次是ESP32-S3和ESP32-C6，当然还有作为摄像头模组使用的ESP32-CAMERA，代码可以使用arduino和 circuitpython进行编写。</p>
<p>3，AI图像识别，计划使用Yolov8进行实现，代码同样部署在树莓派zero2W上。</p>
<p>4，识别完成的结果发送到MQTT服务器，在通过MQTT服务器广播到所有其他节点。</p>
<p>5，若发现宠物，则通过放置在其他区域的ESP32-S3模块播放声音，同时使用ESP32-C6模块控制灯光闪烁，以此吸引宠物离开区域。</p>
<p> </p>
<p> </p>
<p> </p>
<p>三、各部分功能说明</p>
<p>先在树莓派安装系统，安装参照树莓派官方教程就可以，非常简单，这里就不赘述了。然后安装docker：</p>
<pre>
<code># Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
# Add the repository to Apt sources:
echo \
"deb https://download.docker.com/linux/debian \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
接着使用docker compose来部署服务。Compose文件内容如下：
version: '3'
services:
mosquitto:
image: eclipse-mosquitto
container_name: mosquitto
ports:
- 1883:1883
- 9001:9001
restart: unless-stopped</code></pre>
<p> </p>
<p>这样就完成了服务的部署。下面开始编写各个节点的代码。先从信号输入开始。我使用的是ESP32-CAMERA模块，直接使用官方例程就可以。</p>
<p><img alt="ESP32-CAM Video Streaming and Face Recognition with Arduino IDE | RandomNerd Tutorials" src="https://i0.wp.com/randomnerdtutorials.com/wp-content/uploads/2024/06/Arduino-ide-esp32-cam-camerawebserver-example.png?resize=823%2C713&quality=100&strip=all&ssl=1" /></p>
<p>下面是图像处理代码，使用python编写。如果发现有宠物出现在视野中，则会像MQTT服务器上报消息:</p>
<pre>
<code>from ultralytics import YOLO
from ha_mqtt_discoverable import Settings
from ha_mqtt_discoverable.sensors import BinarySensor, BinarySensorInfo
CAM_URL = "http://192.168.1.102:81/stream"
MQTT_USERNAME = ""
MQTT_PASSWORD = ""
mqtt_settings = Settings.MQTT(
host="localhost", username=MQTT_USERNAME, password=MQTT_PASSWORD
)
sensor_info = BinarySensorInfo(name="dog", off_delay=3)
mysensor = BinarySensor(Settings(mqtt=mqtt_settings, entity=sensor_info))
model = YOLO("yolov8n.pt")
results = model.predict(CAM_URL, stream=True, show=False, conf=0.5)
# loop
for result in results:
for box in result.boxes:
class_id = result.names.item()]
cords = box.xyxy.tolist()
cords =
confi = round(box.conf.item(), 2)
print("Object type:", class_id)
print("Coordinates:", cords)
print("Probability:", confi)
print("---")
if class_id == "dog":
mysensor.on()</code></pre>
<p> </p>
<p>这样一旦摄像头发现有狗子出现，就会立马发送到MQTT服务器。</p>
<p>下面看看ESP32-S3和ESP32-C6节点部分。我把他们都安装在一块面包板上，方便连线。其中ESP32-S3与SPK2模块连线方式是：</p>
<p>IO4 - BCLK</p>
<p>IO6 - WS</p>
<p>IO5 - DATA</p>
<p>ESP32-C6部分我是在IO9上连接了一个LED。</p>
<p> </p>
<p>首先来写ESP32-S3的代码，这部分使用circuitpython完成，实现的是当接收到MQTT服务器发布的发现狗子消息后，播放声音吸引狗子注意力。</p>
<p>在开始编写代码前，要先写好settings.toml文件，这样才可以正确使用网络以及连接到MQTT服务器。</p>
<pre>
<code>CIRCUITPY_WIFI_SSID=" "
CIRCUITPY_WIFI_PASSWORD=" "
MQTT_BROKER = ""
MQTT_PORT = 1883
MQTT_USER = ""
MQTT_PASSWORD = ""</code></pre>
<p>下面是程序代码：</p>
<pre>
<code>import audiocore
import board
import audiobusio
import os
import wifi
import socketpool
import ssl
import adafruit_minimqtt.adafruit_minimqtt as MQTT
import json
mqtt_topic = "hmd/binary_sensor/dog/state"
state = 0
# Define callback methods which are called when events occur
def connect(client, userdata, flags, rc):
# This function will be called when the client is connected
# successfully to the broker.
print("Connected to MQTT Broker!")
print("Flags: {0}\n RC: {1}".format(flags, rc))
def disconnect(client, userdata, rc):
# This method is called when the client disconnects
# from the broker.
print("Disconnected from MQTT Broker!")
def subscribe(client, userdata, topic, granted_qos):
# This method is called when the client subscribes to a new feed.
print("Subscribed to {0} with QOS level {1}".format(topic, granted_qos))
def unsubscribe(client, userdata, topic, pid):
# This method is called when the client unsubscribes from a feed.
print("Unsubscribed from {0} with PID {1}".format(topic, pid))
def publish(client, userdata, topic, pid):
# This method is called when the client publishes data to a feed.
print("Published to {0} with PID {1}".format(topic, pid))
def message(client, topic, message):
global state
# This method is called when a topic the client is subscribed to
# has a new message.
print(f"New message on topic {topic}: {message}")
if message == "on":
state = 1
mqtt_client = MQTT.MQTT(
broker=os.getenv("MQTT_BROKER"),
port=os.getenv("MQTT_PORT"),
username=os.getenv("MQTT_USER"),
password=os.getenv("MQTT_PASSWORD"),
is_ssl=False,
socket_pool=socketpool.SocketPool(wifi.radio),
ssl_context=ssl.create_default_context(),
)
# Connect callback handlers to client
mqtt_client.on_connect = connect
mqtt_client.on_disconnect = disconnect
mqtt_client.on_subscribe = subscribe
mqtt_client.on_unsubscribe = unsubscribe
mqtt_client.on_publish = publish
mqtt_client.on_message = message
print("Attempting to connect to %s" % mqtt_client.broker)
mqtt_client.connect()
print("Subscribing to %s" % mqtt_topic)
mqtt_client.subscribe(mqtt_topic)
wave = audiocore.WaveFile("music3.wav")
audio = audiobusio.I2SOut(board.IO4, board.IO6, board.IO5)
while True:
mqtt_client.loop(1)
if state:
audio.play(wave, loop=False)
state = 0</code></pre>
<p> </p>
<p>最后是ESP32-C6部分代码，这部分代码的功能是当接收到MQTT服务器发布的发现狗子消息后，闪烁灯光吸引狗子注意力。这部分也是使用circuitpython编写的。</p>
<pre>
<code>import os
import wifi
import socketpool
import ssl
import adafruit_minimqtt.adafruit_minimqtt as MQTT
import json
import digitalio
import board
import time
mqtt_topic = "hmd/binary_sensor/dog/state"
led = digitalio.DigitalInOut(board.IO9)
led.direction = digitalio.Direction.OUTPUT
# Define callback methods which are called when events occur
def connect(client, userdata, flags, rc):
# This function will be called when the client is connected
# successfully to the broker.
print("Connected to MQTT Broker!")
print("Flags: {0}\n RC: {1}".format(flags, rc))
def disconnect(client, userdata, rc):
# This method is called when the client disconnects
# from the broker.
print("Disconnected from MQTT Broker!")
def subscribe(client, userdata, topic, granted_qos):
# This method is called when the client subscribes to a new feed.
print("Subscribed to {0} with QOS level {1}".format(topic, granted_qos))
def unsubscribe(client, userdata, topic, pid):
# This method is called when the client unsubscribes from a feed.
print("Unsubscribed from {0} with PID {1}".format(topic, pid))
def publish(client, userdata, topic, pid):
# This method is called when the client publishes data to a feed.
print("Published to {0} with PID {1}".format(topic, pid))
def message(client, topic, message):
# This method is called when a topic the client is subscribed to
# has a new message.
print(f"New message on topic {topic}: {message}")
if message == "on":
led.value = True
mqtt_client = MQTT.MQTT(
broker=os.getenv("MQTT_BROKER"),
port=os.getenv("MQTT_PORT"),
username=os.getenv("MQTT_USER"),
password=os.getenv("MQTT_PASSWORD"),
is_ssl=False,
socket_pool=socketpool.SocketPool(wifi.radio),
ssl_context=ssl.create_default_context(),
)
# Connect callback handlers to client
mqtt_client.on_connect = connect
mqtt_client.on_disconnect = disconnect
mqtt_client.on_subscribe = subscribe
mqtt_client.on_unsubscribe = unsubscribe
mqtt_client.on_publish = publish
mqtt_client.on_message = message
print("Attempting to connect to %s" % mqtt_client.broker)
mqtt_client.connect()
print("Subscribing to %s" % mqtt_topic)
mqtt_client.subscribe(mqtt_topic)
# print("Publishing to %s" % mqtt_topic)d
# mqtt_client.publish(mqtt_topic, "Hello Broker!")
# print("Unsubscribing from %s" % mqtt_topic)
# mqtt_client.unsubscribe(mqtt_topic)
# print("Disconnecting from %s" % mqtt_client.broker)
# mqtt_client.disconnect()
while True:
led.value = False
mqtt_client.loop(1)</code></pre>
<p> </p>
<p> </p>
<p><strong>四、作品源码</strong></p>
<div></div>
<p><strong>五、作品功能演示视频</strong></p>
<p>3c89db2c1fd3a914fbd389fe79c9b76d<br />
</p>
<p><strong>六、项目总结</strong></p>
<p>通过这个项目，让我尝试了多个设备通过MQTT协调工作，体验到了MQTT在物联网中不可撼动的强大能力。word版本报告附在了下面：</p>
<div></div>
<p>AI图像识别是不是能识别出事那个猫？ </p>
秦天qintian0303 发表于 2024-10-22 09:10
AI图像识别是不是能识别出事那个猫？
<p>当然可以</p>
页:
[1]