#AI挑战营第一站# pytorch环境+minst数据集训练

tinnu

#AI挑战营第一站# pytorch环境+minst数据集训练 [复制链接]

<i class="pstatus"> 本帖最后由 tinnu 于 2024-4-13 18:25 编辑 </i>

- 安装显卡加速支持
    1. 英伟达
        - 安装cuda
        - [cuda](https://developer.nvidia.com/cuda-toolkit-archive)
        - [cudnn](https://developer.nvidia.com/cudnn-downloads?target_os=Windows)
        - 装完之后在命令行输入 nvcc --version 没有报错即通过
        ```shell
        > nvcc --version
        nvcc: NVIDIA (R) Cuda compiler driver
        Copyright (c) 2005-2023 NVIDIA Corporation
        Built on Mon_Apr__3_17:36:15_Pacific_Daylight_Time_2023
        Cuda compilation tools, release 12.1, V12.1.105
        Build cuda_12.1.r12.1/compiler.32688072_0
        ```
    2. AMD
        - 安装ROCm
        - [linux](https://github.com/ROCm/ROCm)
        - [windows](https://www.amd.com/zh-cn/developer/resources/rocm-hub/hip-sdk.html)
    3. 国产之光摩尔线程
        - 安装musa
        - https://developer.mthreads.com/sdk/download/musa?equipment=&os=Ubuntu&driverVersion=&version=
        - PS：且不说摩尔线程硬件用的是IMG的IP，到底能不能算国产之类的话题，但这么多所谓自主的显卡，你就说他能不能pytorch吧，起码软件方面MTT是在脚踏实地地干活。
    4. CPU
        - 完全不用管
- 安装python
    - pip3 install python3
    - pip3 install matplotlib
- 根据pytorch官网提示安装pytorch
    - https://pytorch.org/get-started/locally/
    1. CPU安装
        - win下纯cpu是这个命令：
        - pip3 install torch torchvision torchaudio
    2. 英伟达安装
        - windows下安装：
        - pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
    3. AMD显卡安装
        - LINUX+ROCm
        - pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.7
    4. 摩尔线程显卡安装
        - 按照摩尔线程开源仓库步骤安装：https://github.com/MooreThreads/torch_musa
        
- 因为我的显卡是N年前的MX130，装完之后发现跑得比CPU还慢，而且满三倍，于是选择纯CPU安装；安装完后一跑，发现下载贼久，因为minst的数据集在……你懂的，所以我觉得找个外面的云平台可能会比较快。

- 找来找去，能用的云端免费平台也就大名鼎鼎的kaggle，不过虽然能用，但注册的时候还是存在你懂的环节，自行解决不再赘述。所以只要用云平台，上面这些繁琐的步骤其实统统不用……

# 训练
- 首先加载一些程序库
    - torch
    - torchvision
    - matplotlib
    ```py
    import torch
    import torchvision
    from torch.utils.data import DataLoader
    import torch.nn as nn
    import torch.nn.functional as F
    import torch.optim as optim
    import matplotlib.pyplot as plt
    ```
1. 加载数据集
    - python 下可以通过 torchvision 库直接从网络下载数据集到本地，然后自动加载，前面说下载贼旧就是因为这个minst数据集。
    - pytorch 里面加载数据有一套范式，一般是通过 torch.utils.data.DataLoader ，可以控制在训练的时候控制每一轮输出的数据量
    - 以加载训练集为例：
    ```py
    train_loader = torch.utils.data.DataLoader(
        torchvision.datasets.MNIST(
            "./data/",
            train=True,
            download=True,
            transform=torchvision.transforms.Compose(
                [
                    torchvision.transforms.ToTensor(),
                    torchvision.transforms.Normalize((0.1307,), (0.3081,)),
                ]
            ),
        ),
        batch_size=batch_size_train,
        shuffle=True,
    )
    ```
2. 加载数据集
    - minst 可以应用非常典型的网络结构，比如 两个卷积->全连接层->relu->droupout->全连接层
    ```py
    class Net(nn.Module):
        def __init__(self):
            super(Net, self).__init__()
            self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
            self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
            self.conv2_drop = nn.Dropout2d()
            self.fc1 = nn.Linear(320, 50)
            self.fc2 = nn.Linear(50, 10)

def forward(self, x):
            x = F.relu(F.max_pool2d(self.conv1(x), 2))
            x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
            x = x.view(-1, 320)
            x = F.relu(self.fc1(x))
            x = F.dropout(x, training=self.training)
            x = self.fc2(x)
            return F.log_softmax(x, dim=1)
    ```
    - 配置一下优化器，这里可选收敛比较快的SGD，也可以用 Adam 反正minst训练也很快
    
    
<ignore_js_op>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_801783" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=801783&handlekey=savephoto_801783">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

- Adam收敛速度：
    
    
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_801779_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>output_figure.png</strong> <em class="xg1">(145.56 KB, 下载次数: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >下载附件</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_801779" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=801779&handlekey=savephoto_801779">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

- 损失函数，这里没有定义损失函数，是直接在训练里面用 nll_loss (Negative Log Likelihood Loss) 说到底就是向量对数
3. 训练
    ```py
    def train(epoch):
        network.train()
        for batch_idx, (data, target) in enumerate(train_loader):
            optimizer.zero_grad()
            output = network(data)
            loss = F.nll_loss(output, target)
            loss.backward()
            optimizer.step()
    ```
    - 训练也很简单，非常标准的流程：数据对齐->前向传播->计算损失->反向传播->优化器计算
    - 5条语句，5个功能，完事
4. 测试
    ```py
    def test():
        network.eval()
        test_loss = 0
        correct = 0
        with torch.no_grad():
            for data, target in test_loader:
                output = network(data)
                pred = output.data.max(1, keepdim=True)[1]
    ```
5. 显示
    ```py
    examples = enumerate(test_loader)
    batch_idx, (example_data, example_targets) = next(examples)
    with torch.no_grad():
        output = network(example_data)
    fig = plt.figure()
    for i in range(6):
        plt.subplot(2, 3, i + 1)
        plt.tight_layout()
        plt.imshow(example_data<i>[0], cmap="gray", interpolation="none")
        plt.title("Prediction: {}".format(output.data.max(1, keepdim=True)[1]<i>.item()))
        plt.xticks([])
        plt.yticks([])
    plt.show()
    ```
    
    
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_801784_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>output_test.png</strong> <em class="xg1">(12.34 KB, 下载次数: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >下载附件</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_801784" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=801784&handlekey=savephoto_801784">保存到相册</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

6. 模型导出
    ```py
    torch.save(network.state_dict(), "./model.pth")
    torch.save(optimizer.state_dict(), "./optimizer.pth")
    ```

7. onnx转化
    ```py
    def export_to_onnx(model, input_example, output_path="model.onnx"):
        model.eval()
        torch.onnx.export(
            model,     
            input_example, 
            output_path,   
            opset_version=11,
            do_constant_folding=True,
            input_names=["input"], 
            output_names=["output"], 
            dynamic_axes={
                "input": {0: "batch_size"},
                "output": {0: "batch_size"}
            },
        )

print("Model exported to ONNX format successfully.")

input_example = torch.randn(1, 28, 28)
    export_to_onnx(network, input_example=input_example)
    ```

<ignore_js_op>

<a href="javascript:;" class="loginf">model.onnx</a>

<em class="xg1">(88.09 KB, 下载次数: 2)</em>
</span>
<div class="tip tip_4" id="attach_801780_menu" style="position: absolute; display: none" disautofocus="true">
<div class="tip_c xs0">
<div class="y">2024-4-13 18:17 上传</div>
点击文件名下载附件

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

<ignore_js_op>

<a href="javascript:;" class="loginf">model.pth</a>

<em class="xg1">(88.16 KB, 下载次数: 0)</em>
</span>
<div class="tip tip_4" id="attach_801781_menu" style="position: absolute; display: none" disautofocus="true">
<div class="tip_c xs0">
<div class="y">2024-4-13 18:17 上传</div>
点击文件名下载附件

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

<ignore_js_op>

<a href="javascript:;" class="loginf">optimizer.pth</a>

<em class="xg1">(177.19 KB, 下载次数: 0)</em>
</span>
<div class="tip tip_4" id="attach_801782_menu" style="position: absolute; display: none" disautofocus="true">
<div class="tip_c xs0">
<div class="y">2024-4-13 18:17 上传</div>
点击文件名下载附件

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>

硬核王同学

牛

#AI挑战营第一站# pytorch环境+minst数据集训练 [复制链接]

最新回复

浏览过的版块