NLPIR/ICTCLA2018 ACADEMIC SEMINAR 5th ISSUE COMPLETED
Last week, Qinghong Jiang gave a presentation about the paper, Towards Designing Better Session Search Evaluation Metrics and shared some opinion on it. At first it was a little hard to understand what is the purpose, after Nihad offered backgraound information about their previous work, the whole picture was gradually clear.
After the presentation, several questions were asked. The Q&As are listed as following:
Q: What is the meaning of cCG cDCG cERR cMin cMax? And what are the differences?
A: They are different metrics with query-level satisfaction (All correlations are significant at p < 0.001) which are based on users’ useful- ness judgments of click sequence.
Q: What’s the result of the score in table1?
A: The Pearson’s correlation coefficient between these metrics and query satisfaction for all queries and queries with at least one click are shown in Table 1. The results show that all metrics have signifi- cant correlations with query satisfaction.
Q: what is the meaning of 2ur and dr?
A: dr is a clicked document, ur is the usefulness judgment of dr and we use (2ur −1 ) to represent the gain of it.
Finally Professor Zhang commented that although the study of search engine is less popular nowadays, people use recommendation based on searching. And fake news is common on search engine. Another point is to personalize the seach result, what he wants is that when different people search the same things, they can get different result.
The powerpoint of the presentation can be download.
