【Follow me第二季第4期】任务汇总

Netseye · 发表于2024-12-28 13:47

【Follow me第二季第4期】任务汇总 [复制链接]

本帖最后由 Netseye 于 2024-12-28 14:01 编辑

【Follow me第二季第4期】任务汇总

本次一共选购了三件商品.为了方便自己增加额外的一个编码器模块和一个speaker模块

如图:

本次所有任务都尝试了一下:

https://bbs.eeworld.com.cn/thread-1301096-1-1.html

https://bbs.eeworld.com.cn/thread-1301100-1-1.html

https://bbs.eeworld.com.cn/thread-1301103-1-1.htm

https://bbs.eeworld.com.cn/thread-1301110-1-1.html

选做任务二（非必做）：通过IMU数据结合机器学习算法，识别运动状态，并通过串口打印。

做了一些大概的了解和尝试做一下总结:

首先介绍一下LSM6DSOX 传感器自带机器学习核心（MLC，Machine Learning Core）。这是由意法半导体（STMicroelectronics）开发的一项创新功能，使 LSM6DSOX 能够通过机器学习算法对传感器数据进行分类和模式识别。(好像STM家所有以X 结尾的传感器都会带MLC)

实现方法:

大致的实现方案就是可以基于LSM6DSOX (MLC) 或者MCU Tinyml 实现,整体实现思路是完全一致的只是.使用的工具不同

开发流程：

数据采集：
使用 LSM6DSOX 采集运动数据。

模型训练：
借助 ST 提供的 Unico GUI 工具或者Edge Impulse，使用采集的数据进行模型训练。

模型导出：
将训练好的模型转化下载到项目中

传感器部署：
将模型上传到 LSM6DSOX，并通过寄存器配置启用 MLC。或者使用MCU 中.

运行与分类：
传感器在运行时实时分类并输出结果。

工作的大头是在数据采集上.由于我比较懒提供下方案和代码就不做对应的实践了.

LSM6DSOX (MLC)

参考:

https://github.com/STMicroelectronics/STMems_Machine_Learning_Core/tree/master/configuration_examples

数据采集格式

C++
A_X [mg]          A_Y [mg]          A_Z [mg]         G_X [dps]          G_Y [dps]          G_Z [dps]
80        931        -354        0.665        -5.075        -5.01375
65        931        -352        -0.02625        -7.72625        -8.575
4        949        -344        -1.40875        -12.8713        -11.515
50        960        -343        -4.66375        -23.8613        -15.54
174        933        -341        -7.81375        -25.025        -11.7075

实现(按下按钮进行动作录制)

C++
#include <Arduino_LSM6DSOX.h> // IMU 库
#include <OneButton.h>

const int PIN_INPUT = D6;     // D6 引脚连接按钮
const int ledPin = LED_BUILTIN; // 板载LED

OneButton button;

float Ax, Ay, Az;
float Gx, Gy, Gz;
bool isRecording = false;
unsigned long startTime = 0;

void setup() {
  Serial.begin(115200);
  while (!Serial);

    if (!IMU.begin()) {
    Serial.println("Failed to initialize IMU!");
    while (1);
  }
  button.setup(PIN_INPUT, INPUT_PULLUP);

  button.attachPress(singleClick); // 设置单击事件

}

void loop() {
  button.tick(); // 检测按钮事件

  if (isRecording) {
    if (IMU.accelerationAvailable() && IMU.gyroscopeAvailable()) {
      IMU.readAcceleration(Ax, Ay, Az);
      IMU.readGyroscope(Gx, Gy, Gz);

      Serial.print(Ax * 1000.0f);
      Serial.print("\t");
      Serial.print(Ay * 1000.0f);
      Serial.print("\t");
      Serial.print(Az * 1000.0f);
      Serial.print("\t");
      Serial.print(Gx);
      Serial.print("\t");
      Serial.print(Gy);
      Serial.print("\t");
      Serial.println(Gz);
    }
  }
}

void singleClick() {
  isRecording = !isRecording;
  digitalWrite(ledPin, isRecording);
    if (isRecording) {
        Serial.println("开始录制...");
        startTime = millis();
        Serial.println("A_X [mg]\tA_Y [mg]\tA_Z [mg]\tG_X [dps]\tG_Y [dps]\tG_Z [dps]");
      } else {
        Serial.println("停止录制...");
        Serial.print("录制时长：");
        Serial.print(millis()-startTime);
        Serial.println("ms");
      }
}

使用 Edge Impulse实现

参考

https://wiki.seeedstudio.com/cn/XIAO-RP2040-EI/

创建Edge Impulse账号项目和安装Edge Impulse cli我就不再赘述了. 数据格式可以完全使用上面的代码输出格式.可以使用同一份数据

执行采集命令

C++
edge-impulse-data-forwarder --clean                                                                                                                                                                                                 ⏎
Edge Impulse data forwarder v1.30.1
? What is your user name or e-mail address (edgeimpulse.com)? netseye@gmail.com
? What is your password? [hidden]
Endpoints:
    Websocket: wss://remote-mgmt.edgeimpulse.com
    API:       https://studio.edgeimpulse.com
    Ingestion: https://ingestion.edgeimpulse.com

[SER] Connecting to /dev/tty.usbmodem11101
[SER] Serial is connected (50:24:B0:93:82:61:7D:18)
[WS ] Connecting to wss://remote-mgmt.edgeimpulse.com
[WS ] Connected to wss://remote-mgmt.edgeimpulse.com

[SER] Detecting data frequency...
[SER] Detected data frequency: 10Hz
? 6 sensor axes detected (example values: [-0.04,-0.15,0.99,0.12,0.06,-0.37]). What do you want to call them? Separate the names with ',': A_X,A_Y,A_Z,G_X,G_Y,G_Z
? What name do you want to give this device? Nano RP2040 Connect
[WS ] Device "Nano RP2040 Connect" is now connected to project "nano rp2040". To connect to another project, run `edge-impulse-data-forwarder --clean`.

训练:

STMems_Machine_Learning_Core 中看起来有4种可以使用的方式:Unico GUI Weka MATLAB Scikit-learn (Python)

Edge Impulse:直接在web平台上进行训练:

例子体验:

本次直接使用了Head gestures 相关代码进行演示,只需要下载lsm6dsox_head_gestures.h 带自己项目中.

C++
#include "../../bsp/config.h"
#include "../../bsp/display.hpp"
#include "../App.h"
#include "LSM6DSOXSensor.h"
// #include "lsm6dsox_activity_recognition_for_mobile.h"
#include "lsm6dsox_head_gestures.h"

// 活动状态映射表
const char *activityNames[] = {
    "点头", // 0
    "摇动", // 1
    "静止", // 2
    "摆动", // 3
    "步行" // 4
};

void drawAccelBar(LGFX_Sprite *canvas, const char *label, float accel,
                  int yPos) {
  int centerX = 64;
  int barMaxWidth = 60;
  int barHeight = 10;
  int barY = yPos + 10;

  // 将加速度值映射到 [-barMaxWidth, barMaxWidth]
  // 使用浮点数计算，最后再转换为整数
  float barWidthFloat = accel * barMaxWidth;
  int barWidth = constrain(int(barWidthFloat), -barMaxWidth, barMaxWidth);

  canvas->fillRect(centerX - barMaxWidth - 1, barY - 1, barMaxWidth * 2 + 2,
                   barHeight + 2, TFT_WHITE); // 清除区域略微扩大

  // 绘制中心线（在清除之后绘制，确保可见）
  canvas->drawLine(centerX, barY, centerX, barY + barHeight,
                   TFT_BLACK); // 使用白色绘制中心线

  if (barWidth < 0) {
    canvas->fillRect(centerX + barWidth, barY, abs(barWidth), barHeight,
                     TFT_BLACK);
  } else if (barWidth > 0) {
    canvas->fillRect(centerX, barY, barWidth, barHeight, TFT_BLACK);
  }
}

// Interrupts.
volatile int mems_event = 0;

// Components
LSM6DSOXSensor AccGyr(&Wire, LSM6DSOX_I2C_ADD_L);

// MLC
ucf_line_t *ProgramPointer;
int32_t LineCounter;
int32_t TotalNumberOfLine;

void INT1Event_cb();
void printMLCStatus(uint8_t status);

void App::_imu_mlc_init() {
  uint8_t mlc_out[8];
  // Led.
  pinMode(LED_BUILTIN, OUTPUT);

  // Force INT1 of LSM6DSOX low in order to enable I2C
  pinMode(INT_IMU, OUTPUT);

  digitalWrite(INT_IMU, LOW);

  delay(200);

  // Initialize I2C bus.
  Wire.begin();

  AccGyr.begin();
  AccGyr.Enable_X();
  AccGyr.Enable_G();

  /* Feed the program to Machine Learning Core */
  /* Activity Recognition Default program */
  ProgramPointer = (ucf_line_t *)lsm6dsox_head_gestures;
  TotalNumberOfLine = sizeof(lsm6dsox_head_gestures) / sizeof(ucf_line_t);
  Serial.println("Activity Recognition for LSM6DSOX MLC");
  Serial.print("UCF Number Line=");
  Serial.println(TotalNumberOfLine);

  for (LineCounter = 0; LineCounter < TotalNumberOfLine; LineCounter++) {
    if (AccGyr.Write_Reg(ProgramPointer[LineCounter].address,
                         ProgramPointer[LineCounter].data)) {
      Serial.print("Error loading the Program to LSM6DSOX at line: ");
      Serial.println(LineCounter);
      while (1) {
        // Led blinking.
        digitalWrite(LED_BUILTIN, HIGH);
        delay(250);
        digitalWrite(LED_BUILTIN, LOW);
        delay(250);
      }
    }
  }

  Serial.println("Program loaded inside the LSM6DSOX MLC");

  // Interrupts.
  pinMode(INT_IMU, INPUT);
  attachInterrupt(INT_IMU, INT1Event_cb, RISING);

  /* We need to wait for a time window before having the first MLC status */
  delay(3000);

  AccGyr.Get_MLC_Output(mlc_out);

  Serial.println("MLC Status: ");
  if (mlc_out[0] <= 4) {
    const char *activity = activityNames[mlc_out[0]];
    Serial.print("Activity: ");
    Serial.println(activity);
  }
}

void App::_imu_mlc_show() {

  _canvas->setFont(&fonts::efontCN_14);

  while (1) {
    _canvas->fillScreen(TFT_WHITE);

    _canvas->setTextColor(TFT_BLACK);
    _canvas->drawString("-IMU Motion", 8, 0);

    if (mems_event) {
      mems_event = 0;
      LSM6DSOX_MLC_Status_t status;
      AccGyr.Get_MLC_Status(&status);
      Serial.println("MLC Status: ");
      if (status.is_mlc1) {
        uint8_t mlc_out[8];
        AccGyr.Get_MLC_Output(mlc_out);
        Serial.print("MLC1: ");
        // Serial.println(mlc_out[0]);
        const char *activity =
            (mlc_out[0] <= 4) ? activityNames[mlc_out[0]] : "Unknown";
        Serial.println(activity);
        _canvas->drawCenterString(String("Activity: ") + activity,
                                  _canvas->width() / 2, 30);
        _canvas_update();
      }
    }

    if (_check_next())
      break;
  }
}

void INT1Event_cb() { mems_event = 1; }

void App::_imu_x_show() {

  _canvas->setFont(&fonts::efontCN_14);

  while (1) {
    _canvas->fillScreen(TFT_WHITE);

    _canvas->setTextColor(TFT_BLACK);
    _canvas->drawString("-IMU", 8, 0);

    float ax, ay, az; // 加速度
    float gx, gy, gz; // 角速度

    // 读取加速度值
    uint8_t acceleroStatus;
    AccGyr.Get_X_DRDY_Status(&acceleroStatus);
    if (acceleroStatus == 1) { // 新数据可用
      int32_t acceleration[3];
      AccGyr.Get_X_Axes(acceleration);

      // 将 mg 转换为 m/s²（1g ≈ 9.80665 m/s², 1000mg = 1g）
      ax = acceleration[0] * 9.80665f / 1000.0f;
      ay = acceleration[1] * 9.80665f / 1000.0f;
      az = acceleration[2] * 9.80665f / 1000.0f;

      Serial.print("加速度 (m/s^2): X=");
      Serial.print(ax);
      Serial.print(" Y=");
      Serial.print(ay);
      Serial.print(" Z=");
      Serial.println(az);

      // 绘制加速度条形图
      drawAccelBar(_canvas, "Accel X:", ax, 10);
      drawAccelBar(_canvas, "Accel Y:", ay, 25);
      drawAccelBar(_canvas, "Accel Z:", az, 40);
    }

    // 读取角速度值
    uint8_t gyroStatus;
    AccGyr.Get_G_DRDY_Status(&gyroStatus);
    if (gyroStatus == 1) { // 新数据可用
      int32_t rotation[3];
      AccGyr.Get_G_Axes(rotation);

      // 将 mdps (milli degrees per second) 转换为 rad/s
      // 1度 = π / 180 弧度，1000 mdps = 1 dps
      gx = rotation[0] * (PI / 180.0f) / 1000.0f;
      gy = rotation[1] * (PI / 180.0f) / 1000.0f;
      gz = rotation[2] * (PI / 180.0f) / 1000.0f;

      Serial.print("陀螺仪 (rad/s): X=");
      Serial.print(gx);
      Serial.print(" Y=");
      Serial.print(gy);
      Serial.print(" Z=");
      Serial.println(gz);
    }

    _canvas_update();

    if (_check_next())
      break;
  }
}

由于本次吧所有任务集成的一个项目中,并做了相关的菜单切换和展示.所以代码也做了一些调整

之前的任务中的imu直接使用的#include <Arduino_LSM6DSOX.h> 这次直接为了使用mlc试了的STM的库

看起来rp2040在aruduino环境中没有rtos可用所有blink这个任务也做了相关代码的调整.好像有一个TimeAlarms 库可以做一些异步处理. 不过我选择了直接用计时的方式来实现
12月28日

20241228135518108.zip (125.5 KB, 下载次数: 0)

6 天前上传
点击文件名下载附件
code