IOB 准确度和精密度之间的差异

我正在使用命名实体识别和分块器对 NLTK 进行一些工作。我使用重新训练了分类器nltk/chunk/named_entity.py为此，我采取了以下措施：

ChunkParse score:
    IOB Accuracy:  96.5%
    Precision:     78.0%
    Recall:        91.9%
    F-Measure:     84.4%

但我不明白在这种情况下 IOB 精度和精度之间的确切区别是什么。实际上，我在文档中找到了（here http://nltk.googlecode.com/svn/trunk/doc/book/ch07.html）以下为具体示例：

IOB 标签准确度表明超过三分之一的单词是 用 O 标记，即不在 NP 块中。然而，由于我们的标记器做了 没有找到任何块，它的精度、召回率和f-measure都是 零。

那么，如果 IOB 准确度只是 O 标签的数量，那么为什么我们没有块，并且 IOB 准确度同时不是 100%（在该示例中）？

先感谢您

维基百科上有关于精度和准确度之间差异的非常详细的解释（参见https://en.wikipedia.org/wiki/Accuracy_and_ precision https://en.wikipedia.org/wiki/Accuracy_and_precision）， 简单来说：

accuracy = (tp + tn) / (tp + tn + fp + fn)
precision = tp / tp + fp

回到NLTK，有一个模块调用块分数 http://nltk.googlecode.com/svn-/trunk/doc/api/nltk.chunk.util-pysrc.html#ChunkScore计算出accuracy, precision and recall您的系统的。这是 NLTK 计算方式的有趣部分tp,fp,tn,fn for accuracy and precision，它以不同的粒度进行。

For accuracy，NLTK计算令牌总数（不是大块！！），用 POS 标签和 IOB 标签猜对，然后除以金句中的 token 总数。

accuracy = num_tokens_correct / total_num_tokens_from_gold

For 精确 and recall，NLTK 计算：

• True Positives通过计算块的数量（不是代币！！！) 猜对了
• False Positives通过计算块的数量（不是代币！！！）是猜测的，但它们是错误的。
• True Negatives通过计算块的数量（不是代币！！！）是系统无法猜测的。

然后计算精度和召回率，如下所示：

precision = tp / fp + tp
recall = tp / fn + tp

为了证明以上几点，请尝试以下脚本：

from nltk.chunk import *
from nltk.chunk.util import *
from nltk.chunk.regexp import *
from nltk import Tree
from nltk.tag import pos_tag

# Let's say we give it a rule that says anything with a [DT NN] is an NP
chunk_rule = ChunkRule("<DT>?<NN.*>", "DT+NN* or NN* chunk")
chunk_parser = RegexpChunkParser([chunk_rule], chunk_node='NP')

# Let's say our test sentence is:
# "The cat sat on the mat the big dog chewed."
gold = tagstr2tree("[ The/DT cat/NN ] sat/VBD on/IN [ the/DT mat/NN ] [ the/DT big/JJ dog/NN ] chewed/VBD ./.")

# We POS tag the sentence and then chunk with our rule-based chunker.
test = pos_tag('The cat sat on the mat the big dog chewed .'.split())
chunked = chunk_parser.parse(test)

# Then we calculate the score.
chunkscore = ChunkScore()
chunkscore.score(gold, chunked)
chunkscore._updateMeasures()

# Our rule-based chunker says these are chunks.
chunkscore.guessed()

# Total number of tokens from test sentence. i.e.
# The/DT , cat/NN , on/IN , sat/VBD, the/DT , mat/NN , 
# the/DT , big/JJ , dog/NN , chewed/VBD , ./.
total = chunkscore._tags_total
# Number of tokens that are guessed correctly, i.e.
# The/DT , cat/NN , on/IN , the/DT , mat/NN , chewed/VBD , ./.
correct = chunkscore._tags_correct
print "Is correct/total == accuracy ?", chunkscore.accuracy() == (correct/total)
print correct, '/', total, '=', chunkscore.accuracy()
print "##############"

print "Correct chunk(s):" # i.e. True Positive.
correct_chunks = set(chunkscore.correct()).intersection(set(chunkscore.guessed()))
##print correct_chunks
print "Number of correct chunks = tp = ", len(correct_chunks)
assert len(correct_chunks) == chunkscore._tp_num
print

print "Missed chunk(s):" # i.e. False Negative.
##print chunkscore.missed()
print "Number of missed chunks = fn = ", len(chunkscore.missed())
assert len(chunkscore.missed()) == chunkscore._fn_num
print 

print "Wrongly guessed chunk(s):" # i.e. False positive.
wrong_chunks = set(chunkscore.guessed()).difference(set(chunkscore.correct()))
##print wrong_chunks
print "Number of wrong chunks = fp =", len(wrong_chunks)
print chunkscore._fp_num
assert len(wrong_chunks) == chunkscore._fp_num
print 

print "Recall = ", "tp/fn+tp =", len(correct_chunks), '/', len(correct_chunks)+len(chunkscore.missed()),'=', chunkscore.recall()

print "Precision =", "tp/fp+tp =", len(correct_chunks), '/', len(correct_chunks)+len(wrong_chunks), '=', chunkscore.precision()