如何在网络工作者中运行handpose tfjs模型

我想使用网络摄像头获取帧并运行张量流模型“handpose”来估计手部可见度。众所周知，手势模型有点慢，所以我尝试将估计转移到网络工作人员。

问题是HTMLVideoElement object could not be cloned.（我需要将视频传递给estimateHand方法）。

是否可以用另一种方式做到这一点？

网络工作者：

import { getModel } from "../utils/tf-model";

let model: any;

const modelLoading = async () => {
    model = await getModel();
}

export const estimate = async (video: HTMLVideoElement) => {
    if (model) {
        const hands = await model.estimateHands(video);
        return hands;
    }
    return null;
}

modelLoading();

export default {} as typeof Worker & (new () => Worker);

主线程：

import MyWorker from 'comlink-loader!./worker';

const worker = new MyWorker();

...

func = async () => {
        const res = await worker.estimate(this.video);
        ...

EDIT我自己找到了解决方案，可以得到ImageData从画布上下文并将其传递给网络工作者。

...
const canvas = document.createElement('canvas');
// ofc we have to set width and height equal to video sizes ...

const ctx = canvas.getContext('2d')
ctx.drawImage(this.video, 0, 0, this.video.width, this.video.height);
const img = ctx.getImageData(0, 0, this.video.width, this.video.height);
const res = await worker.estimate(img);

好的，我找到了解决方案。它太好了，我们无法将 HTML 元素作为参数传递给 Web Worker，但是我们可以使用画布上下文，如下所示：

...
const canvas = document.createElement('canvas');
// ofc we have to set width and height equal to video sizes ...

const ctx = canvas.getContext('2d')
ctx.drawImage(this.video, 0, 0, this.video.width, this.video.height);
const img = ctx.getImageData(0, 0, this.video.width, this.video.height);
const res = await worker.estimate(img);