The Entity Linking (EL) task identifies entity mentions in a text corpus and
associates them with an unambiguous identifier in a Knowledge Base. While much
work has been done on the topic, we first present the results of a survey that
reveal a lack of consensus in the community regarding what forms of mentions in
a text and what forms of links the EL task should consider. We argue that no
one definition of the Entity Linking task fits all, and rather propose a
fine-grained categorization of different types of entity mentions and links. We
then re-annotate three EL benchmark datasets - ACE2004, KORE50, and VoxEL -
with respect to these categories. We propose a fuzzy recall metric to address
the lack of consensus and conclude with fine-grained evaluation results
comparing a selection of online EL systems.