Q-Measure

Definition

Q-measure is a graded-relevance version of the well-known Average Precision. Let R denote the number of known relevant documents for a topic; let gain _l denote the gain value for a document of relevance level l: for example, let it be 3 for a highly relevant document, 2 for a relevant document, and 1 for a partially relevant document. For a given ranked list of documents, let I(r) = 0 if the document at rank r is nonrelevant, and I(r) = 1 otherwise; then C(r) = ∑ _{k = 1} ^r I(k): the number of relevant documents within top r. Note that Precision at r is given by C(r)∕r. Let g(r) = gain _l if the document at rank r is l-relevant and let g(r) = 0 otherwise; let the cumulative gain be cg(r) = ∑ _{k = 1} ^r g(k). Define an ideal ranked list for the topic by sorting the R relevant documents by relevance level; let cg ^∗(r) denote the cumulative gain at r for the ideal list. Then Q-measure is defined as:

$$\displaystyle\begin{array}{rcl} Q\mbox{ -}measure& =&...