Entity linking has become a popular task in both natural language processing and semantic web communities. However, we nd that the benchmark datasets for entity linking tasks do not accurately evaluate entity linking systems. In this paper, we aim to chart strengths and weaknesses of current benchmark datasets and sketch a roadmap for the community to devise better benchmark datasets. An extended abstract that followed from this discussion was submitted to LREC 2016.