基于 NCNN, 实现 yolov8

记录下 基于 ncnn 实现 yolov8 的全部过程

1. 修改 ultralytics/nn/modules.py class Detect forward 和 class C2f forward

   class Detect(nn.Module):
   ...
       def forward(self, x):
           shape = x[0].shape  # BCHW
           for i in range(self.nl):
               x[i] = torch.cat((self.cv2[i](x[i]), self.cv3[i](x[i])), 1)
           if self.training:
               return x
           elif self.dynamic or self.shape != shape:
               self.anchors, self.strides = (x.transpose(0, 1) for x in make_anchors(x, self.stride, 0.5))
               self.shape = shape
   
           # x_cat = torch.cat([xi.view(shape[0], self.no, -1) for xi in x], 2)
           # if self.export and self.format in ('saved_model', 'pb', 'tflite', 'edgetpu', 'tfjs'):  # avoid TF FlexSplitV ops
           #     box = x_cat[:, :self.reg_max * 4]
           #     cls = x_cat[:, self.reg_max * 4:]
           # else:
           #     box, cls = x_cat.split((self.reg_max * 4, self.nc), 1)
           # dbox = dist2bbox(self.dfl(box), self.anchors.unsqueeze(0), xywh=True, dim=1) * self.strides
           # y = torch.cat((dbox, cls.sigmoid()), 1)
           # return y if self.export else (y, x)
           
           pred = torch.cat([xi.view(shape[0], self.no, -1) for xi in x], 2).permute(0, 2, 1)
           return pred
   
   
   class C2f(nn.Module):
       export = False  # export mode
       # CSP Bottleneck with 2 convolutions
       def __init__(self, c1, c2, n=1, shortcut=False, g=1, e=0.5):  # ch_in, ch_out, number, shortcut, groups, expansion
           super().__init__()
           self.c = int(c2 * e)  # hidden channels
           self.cv1 = Conv(c1, 2 * self.c, 1, 1)
           self.cv2 = Conv((2 + n) * self.c, c2, 1)  # optional act=FReLU(c2)
           self.m = nn.ModuleList(Bottleneck(self.c, self.c, shortcut, g, k=((3, 3), (3, 3)), e=1.0) for _ in range(n))
   
       def forward(self, x):
           if not self.export:
               y = list(self.cv1(x).chunk(2, 1))
               y.extend(m(y[-1]) for m in self.m)
               return self.cv2(torch.cat(y, 1))
           
           x = self.cv1(x)
           x = [x, x[:, self.c:, ...]]
           x.extend(m(x[-1]) for m in self.m)
           x.pop(1)
           return self.cv2(torch.cat(x, 1))
   
       def forward_split(self, x):
           if not self.export:
               print("------------> c2f forward_split:")
               y = list(self.cv1(x).split((self.c, self.c), 1))
               y.extend(m(y[-1]) for m in self.m)
               return self.cv2(torch.cat(y, 1))
           x = self.cv1(x)
           x = [x, x[:, self.c:, ...]]
           x.extend(m(x[-1]) for m in self.m)
           x.pop(1)
           return self.cv2(torch.cat(x, 1))
   
   

2. 生成 onnx， 创建 export.py

   from ultralytics import YOLO
   # load yolov8 segment model
   model = YOLO("yolov8n.pt")
   # Use the model
   success = model.export(format="onnx", opset=13, simplify=True)
   

3. 导出 ncnn 模型

   ./onnx2ncnn models/yolov8n.onnx models/yolov8n.param models/yolov8n.bin
   ./ncnnoptimize models/yolov8n.param models/yolov8n.bin models/yolov8n-opt.param models/yolov8n-opt.bin 65536
   

4. 测试参考 https://github.com/FeiGeChuanShu/ncnn-android-yolov8

Q&A

1. Shape not supported yet!
   修改 modules.py class detect 和 class c2f

2. ./ncnnoptimize models/yolov8n.param models/yolov8n.bin models/yolov8n-opt.param models/yolov8n-opt.bin 65536 报错
   尝试通过网上在线方式转换 fp16， 成功了。但使用ncnn 加载模型总是报错。在网上一顿找，各位大佬都验证好用，到我这就怎么转都报错，很是苦逼。后来索性换了个yolov8版本，结果好使了，麻蛋的。

   # 其实警告已经表示 该版本不支持 onnx exported.  没注意，以为无伤大雅了。
   Results saved to /home/qh/workspace/train/ultralytics-v8
   Predict:         yolo task=detect mode=predict model=yolov8n.onnx -WARNING ⚠️ not yet supported for YOLOv8 exported models
   Validate:        yolo task=detect mode=val model=yolov8n.onnx -WARNING ⚠️ not yet supported for YOLOv8 exported models
   

参考

1. https://github.com/ultralytics/ultralytics
2. https://github.com/Tencent/ncnn
3. https://github.com/FeiGeChuanShu/ncnn-android-yolov8
4. https://github.com/Digital2Slave/ncnn-android-yolov8-seg/wiki/Convert-yolov8-model-to-ncnn-model