我对此有一个解决方案:
我编写了一个 gstreamer 过滤器插件(基于插件模板),在将帧传递到 H.264 编码器和网络传输之前,它可以在捕获帧(并在视频缓冲区上做标记)时节省系统时间。
在接收端,我找到标记(它为我提供了 20 索引中的 1)并再次记下系统时间。
我希望关联索引和比较系统时间将是一个相对简单的练习。只要两个系统的时钟合理同步(或有已知的差异),我就应该能够计算出差异(即延迟)。
The filter->source
在发送器和接收器上设置不同,以确定过滤器的定时行为。
/* chain function
* this function does the actual processing
*/
static GstFlowReturn
gst_my_filter_chain (GstPad * pad, GstBuffer * buf)
{
GstMyFilter *filter;
struct timeval nowTimeval;
guint8* data;
int i,j,offset;
filter = GST_MYFILTER (GST_OBJECT_PARENT (pad));
if (filter->startTime == 0){
filter->startTime = GST_BUFFER_TIMESTAMP(buf);
gettimeofday(&filter->startTimeval, NULL);
filter->startTimeUL = (filter->startTimeval.tv_sec*1e6 + filter->startTimeval.tv_usec)/1e3; // in milliseconds?
filter->index = 0;
GstCaps* caps;
gint width, height;
const GstStructure *str;
caps = GST_BUFFER_CAPS(buf);
str = gst_caps_get_structure (caps, 0);
if (!gst_structure_get_int (str, "width", &width) ||
!gst_structure_get_int (str, "height", &height)) {
g_print ("No width/height available\n");
} else {
g_print ("The video size of this set of capabilities is %dx%d\n",
width, height);
filter->width=width;
filter->height=height;
}
}
gettimeofday(&nowTimeval, NULL);
unsigned long timeNow = (nowTimeval.tv_sec*1e6 + nowTimeval.tv_usec)/1e3; // in milliseconds?
if (filter->silent == FALSE){
fprintf(filter->ofp, "%20lu,",
timeNow);
}
data = GST_BUFFER_DATA(buf);
if (filter->source){
offset = filter->index % 20;
for (i = 0; i < 10; i++){
for (j = 0; j < 10; j++){
data[(i+20)*filter->width+(j+offset*10)*1]=23;
}
}
fprintf(filter->ofp, " %u", offset);
} else {
unsigned long avg;
unsigned int min=(unsigned int)(-1UL);
unsigned int minpos=0;
int k=0;
for (k=0; k < 20; k++){
avg=0;
i=5; // in the middle of the box row
for (j = 0; j < 10; j++){
avg += data[(i+20)*filter->width+(j+k*10)*1];
}
if (avg < min){
min = avg;
minpos=k;
}
}
fprintf(filter->ofp, " %u", minpos);
}
fprintf(filter->ofp, "\n");
filter->index++;
/* just push out the incoming buffer without touching it */
return gst_pad_push (filter->srcpad, buf);
}
用法如下:
发送者/服务器:
GST_DEBUG="*:2" gst-launch-0.10 -v --gst-plugin-path=../../src/.libs videotestsrc num-buffers=100 ! myfilter src=1 ! x264enc tune=zerolatency,speed-preset=fast ! rtph264pay ! udpsink port=3000 host=127.0.0.1
接收者/客户端:
GST_DEBUG="*:2" gst-launch-0.10 -v --gst-plugin-path=../../src/.libs udpsrc port=3000 ! "application/x-rtp, media=(string)video, encoding-name=(string)H264, payload=(int)96" ! gstrtpjitterbuffer do-lost=true ! rtph264depay ! ffdec_h264 ! myfilter src=0 ! ffmpegcolorspace ! ximagesink
显然,在测试实现中我不会使用 localhost (127.0.0.1)!
我用--gst-plugin-path
因为我还没有安装定时过滤器。
该项目要求延迟尽可能小——最好是 100 毫秒或更小。现在有了一些数字,我可以开始微调所需的参数以最小化延迟。