precision plot图
类似如下图:
其含义为:跟踪算法估计的目标位置(bounding box)的中心点与人工标注(ground-truth)的目标的中心点,这两者的距离小于给定阈值的视频帧的百分比。不同的阈值,得到的百分比不一样,因此可以获得一条曲线。
该评估方法的缺点:无法反映目标物体大小与尺度的变化。
简单来说:横坐标代表阈值,阈值范围是1到50,纵坐标代表 跟踪算法估计的目标位置(bounding box)的中心点与人工标注(ground-truth)的目标的中心点的距离小于给定阈值的视频帧的百分比。(比如:横坐标阈值为10,对应纵坐标百分比为0.6,就代表当前跟踪视频序列有60%的视频帧是满足bounding box的中心与ground-truth的中心的距离是小于10个像素)
其matlab代码为:
function [precisions, distances, averageLocationError] = precision_plot(positions, ground_truth,colortype,show)
%PRECISION_PLOT
% Calculates precision for a series of distance thresholds (percentage of
% frames where the distance to the ground truth is within the threshold).
% The results are shown in a new figure if SHOW is true.
%
% Accepts positions and ground truth as Nx2 matrices (for N frames), and
% a title string.
% modified: 5/31/2016--------------------------
% positions, and ground_truth is the coordinates of center points
% ---------------------------------------------
%
% Joao F. Henriques, 2014
% http://www.isr.uc.pt/~henriques/
max_threshold = 50; %used for graphs in the paper
precisions = zeros(max_threshold, 1);
if size(positions,1) ~= size(ground_truth,1),
% fprintf('%12s - Number of ground truth frames does not match number of tracked frames.\n', title)
%just ignore any extra frames, in either results or ground truth
n = min(size(positions,1), size(ground_truth,1));
positions(n+1:end,:) = [];
ground_truth(n+1:end,:) = [];
end
%calculate distances to ground truth over all frames
distances = sqrt((positions(:,1) - ground_truth(:,1)).^2 + ...
(positions(:,2) - ground_truth(:,2)).^2);
distances(isnan(distances)) = [];
%compute precisions
for p = 1:max_threshold,
precisions(p) = nnz(distances <= p) / numel(distances);
end
averageLocationError = sum(distances)/numel(distances);
%plot the precisions
if show == 1,
% figure('Number','off', 'Name',['Precisions - ' title])
plot(precisions, colortype, 'LineWidth',1.2)
xlabel('Local error threshold'), ylabel('Precision')
end
end
clear all;
gt = importdata('groundtruth.txt');%ground truth file
A = importdata('boundingbox.txt'); %output file
gt(:,3:4) = gt(:,3:4)+0.001;
[precisions1, distances1, averageLocationError1]=precision_plot(A,gt,'k',1);hold on;
legend('tracker','Location','NorthEast');
注意:以上代码只是算一个视频序列的结果,那要怎么求出跟踪算法对于一整个数据集(如OTB 、VOT)的precision plot呢?一般来说直接跑benchmark就可以直接出来了 但是我就想手算 我就不嫌麻烦 行不行? 行!!!
运行完上面第二段代码,matlab右边工作台会出现一个:
双击点开:
就是50个点
再对视频集中所有的视频序列都跑一遍跟踪算法,比如跑OTB50(OTB50一个50个视频序列)是不是就得到了 50列这样的数据,全部拿到Excel中,每行求平均,再对求出的平均一画折线图,OK 完成。
success plot图
类似如下图:
其含义是:首先定义重合率得分(overlap score,OS),追踪算法得到的bounding box(记为a),与ground-truth给的box(记为b),重合率定义为:OS = |a∩b|/|a∪b|,|·|表示区域的像素数目。当某一帧的OS大于设定的阈值时,则该帧被视为成功的(Success),总的成功的帧占所有帧的百分比即为成功率(Success rate)。OS的取值范围为0~1,因此可以绘制出一条曲线。一般阈值设定为0.5。
简单来说:假设一个视频序列一共有1000帧,当横坐标为0.2时,对应的纵坐标为0.7:代表的意思就是:有70%的帧(即700帧)的OS大于0.2
matlab代码为:
function [ success_rate, OverlapRatio ] = success_plot( positions, ground_truth, colortype,show )
% positions and ground_truth format: [x,y,w,h] of the top-left point of the
% bounding box
Ratio = bboxOverlapRatio(positions, ground_truth);
OverlapRatio = diag(Ratio);
threshold = [0.02:0.02:1];
N = size(threshold,2);
success_rate = zeros(N,1);
for i = 1:N
success_rate(i) = nnz(OverlapRatio >= threshold(i)) / numel(OverlapRatio);
end
if show==1
plot(success_rate,threshold, colortype, 'LineWidth',1.2)
xlabel('Overlap threshold'), ylabel('Success rate')
end
end
clear all;
gt = importdata('bird1.txt');%ground truth file
A = importdata('bird1_bboxes.txt'); %output file
gt(:,3:4) = gt(:,3:4)+0.001;
success_plot(A,gt,'r',1);hold on;
legend('ours','Location','NorthEast');
注意:以上代码只是算一个视频序列的结果,算全部序列方法和precision plot一致