(donwload csv results here)
The main objective of this questionnaire is to gain insights into what the community considers to be the goal of Entity Linking. We will present two sentences as examples. These examples consider English Wikipedia as the target of Entity Linking. We then ask you which annotations you think an Entity Linking system should output. Given our objective, we do not provide explicit criteria (there are no "correct answers"). Instead, we would like you to select the annotations that, according to your criteria, you think should ideally be targetted by Entity Linking systems (independent of difficulty).
To save time, note that we generally do not expect you to check Wikipedia pages except for cases where you feel clarification is needed. In order to gauge your opinion, we encourage you to select an available option for each question; however, in case you do not understand an annotation, you may leave a question blank.
Please note that we plan to publish the responses for all multiple-choice answers. However, we will only publish/directly quote optional comments where you indicate that you are okay with that. We will not ask for your name or contact details.
We thank you in advance!
For the previous text, which of the following annotations do you think an Entity Linking system should produce?
In an [[interview]] with Martin Bashir for ....
In an interview with [[Martin Bashir]] for ....
... with Martin Bashir for the [[2003]] documentary ...
... the 2003 [[documentary]] Living with Michael Jackson ...
... the 2003 documentary [[Living with Michael Jackson]], the King of Pop recalled that ...
... the 2003 documentary Living with [[Michael Jackson]], the King of Pop recalled that ...
... the 2003 documentary Living with Michael Jackson, the [[King]] of Pop recalled that ...
... the 2003 documentary Living with Michael Jackson, the King of [[Pop]] recalled that ...
... the 2003 documentary Living with Michael Jackson, the [[King of Pop]] recalled that ...
... the King of Pop [[recalled]] that Joe often sat ...
... the King of Pop recalled that [[Joe]] often sat ...
... that Joe often [[sat]] with a white belt ...
... Joe often sat with a [[white]] belt at hand ...
... Joe often sat with a white [[belt]] at hand as he and ...
... white belt at [[hand]] as he and ...
... as [[he]] and his four siblings rehearsed.
... as he and [[his]] four siblings rehearsed.
... as he and his [[four]] siblings rehearsed.
... as he and his four [[siblings]] rehearsed.
... as [[he and his four siblings]] rehearsed.
... as he and his four siblings [[rehearsed]].
For the previous text, which of the following annotations do you think an Entity Linking system should produce?
[[Russian]] daily Kommersant reports that ...
Russian [[daily]] Kommersant reports that ...
Russian daily [[Kommersant]] reports that ...
... daily Kommersant [[reports]] that Moscow will ...
... Kommersant reports that [[Moscow]] will supply ...
... Moscow will [[supply]] the Greeks with gas ...
... Moscow will supply the [[Greeks]] with gas ...
... will supply the Greeks with [[gas]] at rock bottom prices ...
... with gas at [[rock]] bottom prices as ...
... with gas at [[rock bottom prices]] as ...
... with gas at rock bottom [[prices]] as ...
... rock bottom prices as [[Tsipras]] prepares to meet ...
... rock bottom prices as Tsipras [[prepares]] to meet ...
... as Tsipras prepares to [[meet]] the Russian President.
... as Tsipras prepares to meet the [[Russian]] President.
... as Tsipras prepares to meet the Russian [[President]].
... as Tsipras prepares to meet the [[Russian President]].
Here we'd like to know a little more about your background and expertise with Entity Linking.
Please select your main area(s) of research:
Please select all that apply to you:
Please indicate your overall confidence in your answers: