Introduction 3
Results for CLEF 2009 GRID@CLEF Track 3. Individual Experiment Results and Graphs This section provides the individual results for each official experiment. For each experiment the following tables and graphs are shown: - Overall statistics and information - Interpolated recall vs precision averages plot - Average precision statistics and box plot - Average precision comparison to median plot - Document cutoff levels vs precision at DCL plot - R-Precision statistics and box plot - R-Precision comparison to median plot Topics are identified with DOIs, as well as the experiments. The prefix for the DOI of a topic is 10.2452. The following example shows how to build the DOI for a topic given its number: for topic 200-AH, the corresponding DOI is 10.2452/200-AH List of Submitted Experiments 7 Participant 10.2415/GRIDCLEF-MONO-ENCLEF2009.CHEMNITZ.CUT_GRID_MONO_EN_LUCENE _KROVETZ 10.2415/GRIDCLEF-MONO-ENCLEF2009.CHEMNITZ.CUT_GRID_MONO_EN_LUCENE _PORTER 10.2415/GRIDCLEF-MONO-ENCLEF2009.CHEMNITZ.CUT_GRID_MONO_EN_MERGED _LUCENE_TERRIER 10.2415/GRIDCLEF-MONO-ENCLEF2009.CHEMNITZ.CUT_GRID_MONO_EN_TERRIE
10.2415/GRIDCLEF-MONO-ENCLEF2009.CHEMNITZ.CUT_GRID_MONO_EN_TERRIE
en en en en en en de de de de de de fr fr fr fr fr fr TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD TD
Track Overview Results and Graphs 10 cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; MAP 53.13%; Not Pooled] 90% 0.5 −1 0.5 −1 0.5 −1 0.5 −1 100−A1H02−A1H03−A1H04−A1H05−A1H06−A1H07−A1H08−A1H09−A1H11−A1H12−A1H13−A1H14−A1H15−A1H16−A1H19−A1H21−A1H22−A1H23−A1H24−A1H25−A1H26−A1H28−A1H29−A1H30−AH
Grid@CLEF 2009 Monolingual English Top 5 Participants − Comparison to Median Average Precision by Topic (Topics 131−AH to 62−AH 1 89−AH 90−AH 91−AH 92−AH 95−AH 97−AH 98−AH
99−AH 94−AH
Grid@CLEF 2009 Monolingual English Top 5 Participants − Comparison to Median Average Precision by Topic (Topics 63−AH 1 63−AH65−AH66−AH67−AH68−AH69−AH70−AH71−AH72−AH73−AH74−AH75−AH76−AH77−AH78−AH79−AH80−AH81−AH82−AH83−AH84−AH85−AH86−AH87−AH88−AH
Grid@CLEF 2009 Monolingual English Top 5 Participants − Comparison to Median Average Precision by Topic (Topics 89−AH 1 chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; MAP 54.46%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; MAP 53.13%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; MAP 54.46%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; MAP 53.13%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; MAP 54.46%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; MAP 53.13%; Not Pooled] 50% Recall 10% 20% 30% 40% 60% 70% 80% 90% 100% Grid@CLEF 2009 Monolingual English Top 5 Participants − Comparison to Median Average Precision by Topic (Topics 100−AH to 130−AH) 1 chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; MAP 54.46%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; MAP 53.13%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; MAP 54.46%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; MAP 53.13%; Not Pooled] s t n e m i r e p x E chemnitz [Experiment CUT_GRID_MONO_EN_LUCENE_PORTER; MAP 50.67%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_LUCENE_KROVETZ; MAP 49.37%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_TERRIER_PORTER; MAP 49.26%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_TERRIER_KROVETZ; MAP 48.59%; Not Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
10.2455/TUKEY_T_TEST.643A64E0568043C7BE4717927BF3C689
Grid@CLEF 2009 Monolingual English − Tukey T test with "top group" highlighted CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER s t n e m i r e p x E
CHESHIRE_GRID_ENG_T2FB CUT_GRID_MONO_EN_LUCENE_PORTER CUT_GRID_MONO_EN_LUCENE_KROVETZ CUT_GRID_MONO_EN_TERRIER_PORTER CUT_GRID_MONO_EN_TERRIER_KROVETZ cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; R−Prec 49.83%; Not Pooled] 90% 100% 80% 70% 60% 50% 40% 30% 20% 10% 0%
5 0.5 −1 0.5 −1 0.5 −1 0.5 −1 100−A1H02−A1H03−A1H04−A1H05−A1H06−A1H07−A1H08−A1H09−A1H11−A1H12−A1H13−A1H14−A1H15−A1H16−A1H19−A1H21−A1H22−A1H23−A1H24−A1H25−A1H26−A1H28−A1H29−A1H30−AH
Grid@CLEF 2009 Monolingual English Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH 1 89−AH 90−AH 91−AH 92−AH 95−AH 97−AH 98−AH
99−AH 94−AH
Grid@CLEF 2009 Monolingual English Top 5 Participants − Comparison to Median R−Precision by Topic (Topics 63−AH 1 63−AH65−AH66−AH67−AH68−AH69−AH70−AH71−AH72−AH73−AH74−AH75−AH76−AH77−AH78−AH79−AH80−AH81−AH82−AH83−AH84−AH85−AH86−AH87−AH88−AH
chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; R−Prec 49.19%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; R−Prec 49.83%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; R−Prec 49.19%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; R−Prec 49.83%; Not Pooled] cheshire [Experiment CHESHIRE_GRID_ENG_T2FB; R−Prec 49.83%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER; R−Prec 49.19%; Not Pooled] s t n e m i r e p x E chemnitz [Experiment CUT_GRID_MONO_EN_LUCENE_PORTER; R−Prec 45.67%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_LUCENE_KROVETZ; R−Prec 45.55%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_TERRIER_KROVETZ; R−Prec 43.85%; Not Pooled] chemnitz [Experiment CUT_GRID_MONO_EN_TERRIER_PORTER; R−Prec 43.76%; Not Pooled] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%
Grid@CLEF 2009 Monolingual English − Tukey T test with "top group" highlighted CHESHIRE_GRID_ENG_T2FB CUT_GRID_MONO_EN_MERGED_LUCENE_TERRIER s t n e m i r e p x E
CUT_GRID_MONO_EN_LUCENE_KROVETZ
CUT_GRID_MONO_EN_LUCENE_PORTER CUT_GRID_MONO_EN_TERRIER_KROVETZ CUT_GRID_MONO_EN_TERRIER_PORTER 0.65
0.7 0.75 arcsin(sqrt(R−Precision)) Topic 100-AH 102-AH 103-AH 104-AH 105-AH 106-AH 107-AH 108-AH 109-AH 111-AH 112-AH 113-AH 114-AH 115-AH 116-AH 119-AH 121-AH 122-AH 123-AH 124-AH 125-AH 126-AH 128-AH 129-AH 130-AH 131-AH 133-AH 134-AH 135-AH 136-AH 137-AH 138-AH 139-AH 140-AH 41-AH 42-AH 45-AH 46-AH 47-AH 48-AH 49-AH 50-AH 51-AH 53-AH 55-AH 56-AH 58-AH 59-AH 61-AH 62-AH SNO 0.00 0.00 0.00 0.00 0.52 0.00 0.00 0.00 0.98 0.00 0.00 0.00 0.00 1.05 0.00 0.00 0.00 0.00 0.00 1.33 0.00 0.00 1.33 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.87 2.40 0.00 0.00 0.00
Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 1 0.5 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
CUT_GRID_MONO_DE_LUCENE_PORTER CUT_GRID_MONO_DE_LUCENE_PORTER CUT_GRID_MONO_DE_LUCENE_PORTER CUT_GRID_MONO_DE_LUCENE_PORTER Docs Cutoff Levels Precision at DCL (%) 5 docs 54.05 10 docs 49.88 15 docs 47.46 20 docs 46.01 30 docs 40.20 100 docs 23.58 200 docs 15.44 500 docs 7.51 1000 docs 4.00 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 41.51 100% 90% 80% 70% 60% Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
merged porter pgdec lucene terrier prf Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment ( % ) f o r
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E e5 h tf so4 cp i o3 T f o r 2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision 89−AH 90−AH 91−AH 92−AH 95−AH 97−AH 98−AH
99−AH 94−AH
Topic Identifier Docs Cutoff Levels Precision at DCL (%) 5 docs 60.71 10 docs 55.48 15 docs 52.22 20 docs 49.05 30 docs 43.61 100 docs 27.49 200 docs 17.52 500 docs 8.10 1000 docs 4.20 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 47.01 100% 90% 80% 70% 60%
Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
( % ) f o r 0.5 −1100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual German − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH −1131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH41−AH42−AH 45−AH 46−AH47−AH 48−AH49−AH 50−AH51−AH 53−AH55−AH 56−AH58−AH 59−AH61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual German − Comparison to Median R−Precision by Topic (Topics 63−AH Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment CUT_GRID_MONO_DE_TERRIER_PGDEC CUT_GRID_MONO_DE_TERRIER_PGDEC CUT_GRID_MONO_DE_TERRIER_PGDEC CUT_GRID_MONO_DE_TERRIER_PGDEC
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual German − Comparison to Median Average Precision by Topic (Topics 100−AH to 130−AH) Docs Cutoff Levels Precision at DCL (%) 5 docs 61.19 10 docs 55.48 15 docs 51.11 20 docs 48.99 30 docs 43.33 100 docs 25.38 200 docs 16.50 500 docs 7.81 1000 docs 4.07 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 45.02 100% 90% 80% 70% 60% 0%5 10 15 20 Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment Precision averages (%) for individual queries 100-AH 85.19 131-AH 0.00 63-AH 102-AH 46.67 133-AH 58.18 65-AH 103-AH 52.94 134-AH 57.14 66-AH 104-AH 50.00 135-AH 47.13 67-AH 105-AH 63.64 136-AH 71.43 68-AH 106-AH 28.07 137-AH 0.00 69-AH 107-AH 28.57 138-AH 58.06 70-AH 108-AH 42.55 139-AH 28.57 71-AH 109-AH 42.86 140-AH 65.43 72-AH 111-AH 12.50 41-AH 78.57 73-AH 112-AH 71.43 42-AH 70.28 74-AH 113-AH 40.00 45-AH 52.83 75-AH 114-AH 57.14 46-AH 30.00 76-AH 115-AH 4.17 47-AH 18.75 77-AH 116-AH 74.19 48-AH 12.73 78-AH 119-AH 84.38 49-AH 27.78 79-AH 121-AH 33.33 50-AH 47.32 80-AH 122-AH 21.05 51-AH 0.00 81-AH 123-AH 85.71 53-AH 0.00 82-AH 124-AH 50.00 55-AH 80.38 83-AH 125-AH 61.43 56-AH 33.33 84-AH 126-AH 4.35 58-AH 70.73 85-AH 128-AH 0.00 59-AH 0.00 86-AH 129-AH 0.00 61-AH 81.40 87-AH 130-AH 66.67 62-AH 55.56 88-AH 71.05 89-AH 26.15 90-AH 66.67 91-AH 68.00 92-AH 11.36 94-AH 0.00 95-AH 25.00 97-AH 30.77 98-AH 30.00 99-AH 33.33 40.00 93.75 38.18 15.38 50.00 58.33 75.83 82.00 81.48 33.33 20.00 40.54 40.54 41.18 72.41
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual German − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH 0% 0% Precision averages (%) for individual queries 100-AH 89.18 131-AH 0.13 63-AH 102-AH 62.84 133-AH 66.15 65-AH 103-AH 47.70 134-AH 90.34 66-AH 104-AH 50.92 135-AH 24.77 67-AH 105-AH 8.91 136-AH 62.03 68-AH 106-AH 15.18 137-AH 1.14 69-AH 107-AH 10.00 138-AH 73.23 70-AH 108-AH 42.41 139-AH 7.80 71-AH 109-AH 0.00 140-AH 0.13 72-AH 111-AH 3.22 41-AH 89.02 73-AH 112-AH 94.88 42-AH 74.47 74-AH 113-AH 19.24 45-AH 64.80 75-AH 114-AH 38.86 46-AH 6.08 76-AH 115-AH 9.79 47-AH 23.04 77-AH 116-AH 61.35 48-AH 11.67 78-AH 119-AH 83.78 49-AH 28.41 79-AH 121-AH 46.25 50-AH 71.09 80-AH 122-AH 6.61 51-AH 1.83 81-AH 123-AH 100.00 53-AH 23.86 82-AH 124-AH 57.73 55-AH 71.41 83-AH 125-AH 70.25 56-AH 40.56 84-AH 126-AH 0.29 58-AH 87.52 85-AH 128-AH 29.64 59-AH 0.39 86-AH 129-AH 68.03 61-AH 1.94 87-AH 130-AH 59.62 62-AH 48.63 88-AH 89.12 89-AH 21.45 90-AH 76.84 91-AH 65.33 92-AH 62.80 94-AH 5.11 95-AH 32.10 97-AH 2.67 98-AH 4.25 99-AH 17.46 62.50 99.24 42.48 12.88 69.73 37.36 68.08 85.62 72.90 27.08 9.18 22.00 60.57 48.76 74.66 1.0000 0.0000 Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment CUT_GRID_MONO_DE_TERRIER_PORTER CUT_GRID_MONO_DE_TERRIER_PORTER CUT_GRID_MONO_DE_TERRIER_PORTER CUT_GRID_MONO_DE_TERRIER_PORTER
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual German − Comparison to Median Average Precision by Topic (Topics 100−AH to 130−AH) Docs Cutoff Levels Precision at DCL (%) 5 docs 54.29 10 docs 51.67 15 docs 48.81 20 docs 46.07 30 docs 41.15 100 docs 24.15 200 docs 15.84 500 docs 7.59 1000 docs 4.02 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 42.41 100% 90% 80% 70% 60% 0%5 10 15 20 Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment Precision averages (%) for individual queries 100-AH 74.07 131-AH 1.52 63-AH 102-AH 53.33 133-AH 65.45 65-AH 103-AH 42.86 134-AH 85.71 66-AH 104-AH 35.71 135-AH 27.59 67-AH 105-AH 16.36 136-AH 71.43 68-AH 106-AH 21.05 137-AH 0.00 69-AH 107-AH 14.29 138-AH 67.74 70-AH 108-AH 41.49 139-AH 10.71 71-AH 109-AH 0.00 140-AH 0.00 72-AH 111-AH 0.00 41-AH 78.57 73-AH 112-AH 85.71 42-AH 69.81 74-AH 113-AH 28.00 45-AH 61.32 75-AH 114-AH 46.43 46-AH 0.00 76-AH 115-AH 20.83 47-AH 25.00 77-AH 116-AH 67.74 48-AH 21.82 78-AH 119-AH 76.04 49-AH 27.78 79-AH 121-AH 33.33 50-AH 65.18 80-AH 122-AH 10.53 51-AH 0.00 81-AH 123-AH 100.00 53-AH 28.12 82-AH 124-AH 56.10 55-AH 68.90 83-AH 125-AH 67.14 56-AH 41.67 84-AH 126-AH 4.35 58-AH 75.61 85-AH 128-AH 23.08 59-AH 0.00 86-AH 129-AH 60.00 61-AH 0.00 87-AH 130-AH 66.67 62-AH 44.44 88-AH 73.68 89-AH 29.23 90-AH 70.59 91-AH 58.00 92-AH 61.36 94-AH 0.00 95-AH 30.00 97-AH 0.00 98-AH 10.00 99-AH 25.00 60.00 93.75 43.64 7.69 75.00 41.67 65.00 78.00 70.37 33.33 0.00 22.97 57.66 52.94 68.97 1.0000 0.0000
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual German − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH 20 30 40 50 60 70 80 90 100 Monolingual German using TREC2 with Blind Feedback on title and description
Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 15 ( % ) f o r CHESHIRE_GRID_GER_T2FB CHESHIRE_GRID_GER_T2FB CHESHIRE_GRID_GER_T2FB Docs Cutoff Levels Precision at DCL (%) 5 docs 53.10 10 docs 50.12 15 docs 47.78 20 docs 44.17 30 docs 38.77 100 docs 22.50 200 docs 15.12 500 docs 7.44 1000 docs 3.99 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 40.36 100% 90% 80% 70% 60%
Grid@CLEF 2009 Monolingual German − Box plot of the Topics of the Experiment
Grid@CLEF 2009 Monolingual German − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
( % ) f o r 0.5 −1 15 94−AH
Topic Identifier −1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual German − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH −1 131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH 41−AH 42−AH 45−AH 46−AH 47−AH 48−AH 49−AH 50−AH 51−AH 53−AH 55−AH 56−AH 58−AH 59−AH 61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual German − Comparison to Median R−Precision by Topic (Topics 63−AH
Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
1 0.5 Docs Cutoff Levels Precision at DCL (%) 5 docs 0.24 10 docs 0.24 15 docs 0.24 20 docs 0.18 30 docs 0.16 100 docs 0.18 200 docs 0.09 500 docs 0.04 1000 docs 0.02 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 0.24 100% 90% 80% 70% 60% 0%5 10 15 20 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
0.5 94−AH
Topic Identifier −1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH Overall statistics for 84 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved 0% 0% Precision averages (%) for individual queries 100-AH 20.81 131-AH 56.12 63-AH 79.02 89-AH 102-AH 70.49 133-AH 0.57 65-AH 14.31 90-AH 103-AH 46.87 134-AH 91.67 66-AH 88.94 91-AH 104-AH 28.29 135-AH 58.12 67-AH 37.32 92-AH 105-AH 25.12 136-AH 100.00 68-AH 45.06 94-AH 106-AH 39.67 137-AH 1.38 69-AH 42.31 95-AH 107-AH 23.50 138-AH 39.91 70-AH 78.80 97-AH 108-AH 30.35 139-AH 62.54 71-AH 8.04 98-AH 109-AH 7.41 140-AH 68.36 72-AH 17.01 99-AH 111-AH 1.04 41-AH 100.00 73-AH 20.36 112-AH 68.43 42-AH 38.76 74-AH 25.52 113-AH 4.58 45-AH 61.81 75-AH 50.00 114-AH 37.27 46-AH 3.28 76-AH 37.91 115-AH 75.13 47-AH 32.17 77-AH 9.16 116-AH 61.23 48-AH 5.81 78-AH 14.16 119-AH 82.61 49-AH 24.15 79-AH 55.50 121-AH 5.56 50-AH 35.65 80-AH 19.67 122-AH 80.61 51-AH 4.22 81-AH 47.87 123-AH 58.33 53-AH 14.94 82-AH 89.64 124-AH 59.96 55-AH 39.94 83-AH 100.00 125-AH 56.30 56-AH 37.71 84-AH 100.00 126-AH 74.17 58-AH 90.27 85-AH 17.71 128-AH 40.89 59-AH 70.00 86-AH 36.03 129-AH 34.47 61-AH 0.24 87-AH 51.00 130-AH 72.70 62-AH 74.48 88-AH 98.90 1.0000 0.0024 0.1956 0.3993 0.6840
Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
1 0.5 CUT_GRID_MONO_FR_LUCENE_SAVOY CUT_GRID_MONO_FR_LUCENE_SAVOY CUT_GRID_MONO_FR_LUCENE_SAVOY Docs Cutoff Levels Precision at DCL (%) 5 docs 43.57 10 docs 39.76 15 docs 37.06 20 docs 35.36 30 docs 31.79 100 docs 18.08 200 docs 11.07 500 docs 4.96 1000 docs 2.60 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 42.05 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 10 15 20 R_PRECISION Precision averages (%) for individual queries 100-AH 20.00 131-AH 65.85 63-AH 76.47 89-AH 102-AH 66.67 133-AH 3.12 65-AH 15.38 90-AH 103-AH 37.93 134-AH 66.67 66-AH 78.43 91-AH 104-AH 25.00 135-AH 57.69 67-AH 42.11 92-AH 105-AH 50.00 136-AH 100.00 68-AH 60.00 94-AH 106-AH 34.62 137-AH 0.00 69-AH 31.25 95-AH 107-AH 31.82 138-AH 45.10 70-AH 74.12 97-AH 108-AH 33.82 139-AH 62.22 71-AH 0.00 98-AH 109-AH 0.00 140-AH 67.97 72-AH 18.18 99-AH 111-AH 0.00 41-AH 100.00 73-AH 5.56 112-AH 53.33 42-AH 40.79 74-AH 23.53 113-AH 0.00 45-AH 55.32 75-AH 0.00 114-AH 43.33 46-AH 0.00 76-AH 58.06 115-AH 60.00 47-AH 30.77 77-AH 14.29 116-AH 60.00 48-AH 0.00 78-AH 20.00 119-AH 78.75 49-AH 25.00 79-AH 40.00 121-AH 0.00 50-AH 32.65 80-AH 22.58 122-AH 71.43 51-AH 0.00 81-AH 47.06 123-AH 50.00 53-AH 23.08 82-AH 87.50 124-AH 55.22 55-AH 47.06 83-AH 100.00 125-AH 48.84 56-AH 45.45 84-AH 100.00 126-AH 66.67 58-AH 80.77 85-AH 20.00 128-AH 25.00 59-AH 33.33 86-AH 41.67 129-AH 46.15 61-AH 0.00 87-AH 47.37 130-AH 69.23 62-AH 75.00 88-AH 96.77 1.0000 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH 94−AH
Topic Identifier 76.27 68.98 −1 Overall statistics for 84 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved merged porter savoy lucene terrier prf ( % ) f o r i n d i v i d u a l
Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 100−AH to 130−AH) CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER −1100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 131−AH to 62−AH −1131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH41−AH42−AH 45−AH 46−AH47−AH 48−AH49−AH 50−AH 51−AH53−AH 55−AH56−AH 58−AH 59−AH61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 63−AH −1 63−AH 65−AH 66−AH67−AH 68−AH69−AH 70−AH 71−AH72−AH 73−AH74−AH 75−AH 76−AH77−AH 78−AH79−AH 80−AH 81−AH82−AH 83−AH84−AH 85−AH 86−AH87−AH 88−AH
Topic Identifier Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 89−AH 0 10 20 30 40 50 60 70 80 90 100 60% 40% 30% 20% 10% 0% 0% 10% 20% 30% 40% 60% 70% 80% 90% 100%
Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E e5 h ft so4 cp i o3 T f o r 2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Average Precision 89−AH 90−AH 91−AH 92−AH 95−AH 97−AH 98−AH
99−AH 94−AH
Topic Identifier Docs Cutoff Levels Precision at DCL (%) 5 docs 50.71 10 docs 44.40 15 docs 42.38 20 docs 39.17 30 docs 34.96 100 docs 19.37 200 docs 11.56 500 docs 5.10 1000 docs 2.65 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 48.19 −1100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH −1131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH41−AH42−AH 45−AH 46−AH47−AH 48−AH49−AH 50−AH 51−AH53−AH 55−AH56−AH 58−AH 59−AH61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 63−AH −1 89−AH 90−AH 91−AH 92−AH 95−AH 97−AH 98−AH
99−AH 94−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Retrieved documents vs Mean Precision
( % ) f o r i n d i v i d u a l Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 100−AH to 130−AH) 100% 90% 80% 70% 60% 40% 30% 20% 10% 0% 10
30 100 200
Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment R_PRECISION 69.57 35.29 0.00 45.83 56.25 33.90 42.86 77.78 23.53 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 8 tn7 e m ire6 xp E e5 h ft so4 cp i o3 T f o r 2 e b m u1 N 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER CUT_GRID_MONO_FR_MERGED_LUCENE_TERRIER 0% 0% Precision averages (%) for individual queries 100-AH 36.99 131-AH 62.29 63-AH 79.00 89-AH 102-AH 62.03 133-AH 6.32 65-AH 12.38 90-AH 103-AH 55.64 134-AH 57.03 66-AH 90.90 91-AH 104-AH 12.60 135-AH 12.54 67-AH 62.49 92-AH 105-AH 5.67 136-AH 100.00 68-AH 61.50 94-AH 106-AH 37.65 137-AH 15.42 69-AH 50.60 95-AH 107-AH 37.27 138-AH 10.74 70-AH 79.29 97-AH 108-AH 38.38 139-AH 54.30 71-AH 3.88 98-AH 109-AH 2.83 140-AH 74.21 72-AH 44.67 99-AH 111-AH 56.59 41-AH 100.00 73-AH 16.66 112-AH 59.81 42-AH 31.76 74-AH 32.72 113-AH 1.08 45-AH 37.33 75-AH 12.50 114-AH 55.72 46-AH 1.09 76-AH 34.10 115-AH 83.33 47-AH 44.20 77-AH 42.28 116-AH 70.11 48-AH 4.67 78-AH 19.16 119-AH 87.21 49-AH 30.01 79-AH 62.09 121-AH 100.00 50-AH 52.72 80-AH 30.15 122-AH 71.97 51-AH 14.35 81-AH 73.82 123-AH 83.33 53-AH 38.59 82-AH 11.26 124-AH 55.84 55-AH 55.51 83-AH 100.00 125-AH 41.36 56-AH 58.43 84-AH 16.67 126-AH 79.40 58-AH 89.27 85-AH 27.77 128-AH 6.52 59-AH 47.78 86-AH 18.09 129-AH 43.72 61-AH 3.69 87-AH 44.23 130-AH 66.48 62-AH 79.54 88-AH 93.20 1 0.5 ce reen 0 if D−0.5 0.5 ce reen 0 if D−0.5 1 1 0.5 ce reen 0 if D−0.5 1 0.5 ce reen 0 if D−0.5 Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment 1.0000 0.0108 0.1666 0.4396 0.6539 79.36 15.60 1.93 43.43 72.07 38.03 38.45 64.29 6.27 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 8 Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 100−AH to 130−AH) CUT_GRID_MONO_FR_TERRIER_PORTER CUT_GRID_MONO_FR_TERRIER_PORTER CUT_GRID_MONO_FR_TERRIER_PORTER CUT_GRID_MONO_FR_TERRIER_PORTER 94−AH
Topic Identifier Docs Cutoff Levels Precision at DCL (%) 5 docs 49.29 10 docs 45.12 15 docs 39.92 20 docs 37.20 30 docs 32.54 100 docs 18.12 200 docs 11.09 500 docs 4.99 1000 docs 2.61 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 43.41 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5
30 100 200
Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Retrieved documents vs Mean Precision
CUT_GRID_MONO_FR_TERRIER_PORTER CUT_GRID_MONO_FR_TERRIER_PORTER CUT_GRID_MONO_FR_TERRIER_PORTER CUT_GRID_MONO_FR_TERRIER_PORTER 73.91 29.41 0.00 47.92 62.50 42.94 42.86 66.67 11.76 Precision averages (%) for individual queries 0.5 ce reen 0 if D−0.5 1 1 0.5 ce reen 0 if D−0.5 1 0.5 ce reen 0 if D−0.5 15 94−AH
Topic Identifier −1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH −1 131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH41−AH 42−AH 45−AH 46−AH 47−AH 48−AH 49−AH 50−AH 51−AH 53−AH 55−AH 56−AH 58−AH 59−AH 61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 63−AH 0% 0% Precision averages (%) for individual queries 100-AH 28.41 131-AH 60.81 63-AH 78.15 89-AH 102-AH 74.65 133-AH 4.84 65-AH 8.94 90-AH 103-AH 67.73 134-AH 75.76 66-AH 91.16 91-AH 104-AH 14.74 135-AH 73.77 67-AH 5.70 92-AH 105-AH 12.61 136-AH 100.00 68-AH 17.60 94-AH 106-AH 47.19 137-AH 56.25 69-AH 47.81 95-AH 107-AH 23.21 138-AH 73.73 70-AH 80.53 97-AH 108-AH 36.97 139-AH 66.73 71-AH 5.33 98-AH 109-AH 3.79 140-AH 75.93 72-AH 41.79 99-AH 111-AH 28.39 41-AH 100.00 73-AH 20.51 112-AH 57.82 42-AH 31.41 74-AH 36.17 113-AH 2.20 45-AH 48.93 75-AH 33.33 114-AH 49.87 46-AH 0.68 76-AH 46.08 115-AH 86.25 47-AH 44.47 77-AH 14.45 116-AH 71.64 48-AH 3.69 78-AH 14.08 119-AH 85.75 49-AH 32.12 79-AH 35.76 121-AH 100.00 50-AH 51.18 80-AH 26.43 122-AH 74.88 51-AH 21.28 81-AH 66.67 123-AH 83.33 53-AH 22.76 82-AH 88.97 124-AH 61.70 55-AH 54.62 83-AH 100.00 125-AH 51.77 56-AH 33.00 84-AH 14.29 126-AH 79.77 58-AH 90.03 85-AH 27.15 128-AH 69.06 59-AH 47.78 86-AH 27.08 129-AH 37.97 61-AH 2.97 87-AH 61.49 130-AH 69.16 62-AH 76.80 88-AH 97.95 1 0.5 ce reen 0 if D−0.5 0.5 ce reen 0 if D−0.5 1 1 0.5 ce reen 0 if D−0.5 1 0.5 ce reen 0 if D−0.5 savoy terrier bm25 prf 2 AUTOMATIC French title, description false Grid@CLEF 2009 Monolingual French − Standard Recall Levels vs Mean Interpolated Precision Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment 1.0000 0.0068 0.2299 0.4749 0.7421 86.13 14.99 1.68 45.90 75.36 26.75 36.99 71.90 12.53 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 8 Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 100−AH to 130−AH) CUT_GRID_MONO_FR_TERRIER_SAVOY CUT_GRID_MONO_FR_TERRIER_SAVOY CUT_GRID_MONO_FR_TERRIER_SAVOY CUT_GRID_MONO_FR_TERRIER_SAVOY 94−AH
Topic Identifier Docs Cutoff Levels Precision at DCL (%) 5 docs 53.10 10 docs 45.60 15 docs 42.54 20 docs 38.39 30 docs 33.69 100 docs 18.79 200 docs 11.25 500 docs 5.00 1000 docs 2.61 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 45.06 100% 90% 80% 70% 60% 40% 30% 20% 10% 0%5 10 15 20 Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment R_PRECISION Precision averages (%) for individual queries 100-AH 26.67 131-AH 62.20 63-AH 82.35 89-AH 102-AH 66.67 133-AH 12.50 65-AH 15.38 90-AH 103-AH 56.90 134-AH 66.67 66-AH 88.24 91-AH 104-AH 0.00 135-AH 67.31 67-AH 5.26 92-AH 105-AH 0.00 136-AH 100.00 68-AH 28.00 94-AH 106-AH 42.31 137-AH 33.33 69-AH 50.00 95-AH 107-AH 22.73 138-AH 64.71 70-AH 70.59 97-AH 108-AH 38.24 139-AH 64.44 71-AH 0.00 98-AH 109-AH 0.00 140-AH 71.09 72-AH 36.36 99-AH 111-AH 50.00 41-AH 100.00 73-AH 16.67 112-AH 60.00 42-AH 32.89 74-AH 35.29 113-AH 0.00 45-AH 55.32 75-AH 0.00 114-AH 43.33 46-AH 0.00 76-AH 45.16 115-AH 80.00 47-AH 40.38 77-AH 21.43 116-AH 70.00 48-AH 0.00 78-AH 20.00 119-AH 78.75 49-AH 25.00 79-AH 20.00 121-AH 100.00 50-AH 46.94 80-AH 32.26 122-AH 75.00 51-AH 12.50 81-AH 58.82 123-AH 50.00 53-AH 30.77 82-AH 87.50 124-AH 59.70 55-AH 55.29 83-AH 100.00 125-AH 48.84 56-AH 34.09 84-AH 0.00 126-AH 66.67 58-AH 88.46 85-AH 30.00 128-AH 75.00 59-AH 33.33 86-AH 36.11 129-AH 30.77 61-AH 0.00 87-AH 73.68 130-AH 69.23 62-AH 75.00 88-AH 90.32 1.0000 73.91 29.41 0.00 47.92 68.75 33.90 42.86 55.56 5.88 1 0.5 ce reen 0 if D−0.5 0.5 ce reen 0 if D−0.5 1 1 0.5 ce reen 0 if D−0.5 1 0.5 ce reen 0 if D−0.5 15 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
−1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 131−AH to 62−AH −1 131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH41−AH 42−AH 45−AH 46−AH 47−AH 48−AH 49−AH 50−AH 51−AH 53−AH 55−AH 56−AH 58−AH 59−AH 61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 63−AH 94−AH
Topic Identifier 20 50 80 A v e r a g e r e l e v a n t Overall statistics for 84 queries : Total number of documents over all queries Retrieved Relevant Relevant retrieved 84,000 2,279 2,261 Monolingual French using TREC2 with Blind Feedback on title and description 60% n o i ics 50% e r P 40% 30% 20% 10% 0% 0% 50%
Recall 10% 20% 30% 40% 60% 70% 80% 90% 100%
Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment 00% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
85.81 15.69 45.36 50.55 66.42 45.92 51.72 80.79 75.90 ce reen 0 if D−0.5 0.5 ce reen 0 if D−0.5 1 1 0.5 ce reen 0 if D−0.5 1 0.5 ce reen 0 if D−0.5 −1 10 ( % ) f o r i n d i v i d u a l
Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 100−AH to 130−AH) −1 100−AH102−AH103−AH104−AH105−AH106−AH107−AH108−AH109−AH111−AH112−AH113−AH114−AH115−AH116−AH119−AH121−AH122−AH123−AH124−AH125−AH126−AH128−AH129−AH130−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 131−AH to 62−AH −1 131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH 41−AH 42−AH 45−AH 46−AH 47−AH 48−AH 49−AH 50−AH 51−AH 53−AH 55−AH 56−AH 58−AH 59−AH 61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median Average Precision by Topic (Topics 63−AH 89−AH 90−AH 91−AH 92−AH 95−AH 97−AH 98−AH
99−AH 94−AH
Topic Identifier CHESHIRE_GRID_FRE_T2FB CHESHIRE_GRID_FRE_T2FB CHESHIRE_GRID_FRE_T2FB Docs Cutoff Levels Precision at DCL (%) 5 docs 57.14 10 docs 51.19 15 docs 46.51 20 docs 42.74 30 docs 35.75 100 docs 19.57 200 docs 11.82 500 docs 5.24 1000 docs 2.69 R-Precision (precision after R document retrieved, where R = Relevant retrieved) 100% 90% 80% 70% 60% Grid@CLEF 2009 Monolingual French − Box plot of the Topics of the Experiment Interquartile range Mean Mean With No Outliers Std With No Outliers ( % ) f o r 100-AH 20.00 131-AH 59.76 63-AH 70.59 89-AH 102-AH 77.78 133-AH 50.00 65-AH 15.38 90-AH 103-AH 37.93 134-AH 66.67 66-AH 84.31 91-AH 104-AH 25.00 135-AH 80.77 67-AH 57.89 92-AH 105-AH 0.00 136-AH 100.00 68-AH 40.00 94-AH 106-AH 42.31 137-AH 0.00 69-AH 37.50 95-AH 107-AH 54.55 138-AH 50.98 70-AH 67.06 97-AH 108-AH 44.12 139-AH 64.44 71-AH 0.00 98-AH 109-AH 0.00 140-AH 71.88 72-AH 18.18 99-AH 111-AH 0.00 41-AH 100.00 73-AH 44.44 112-AH 53.33 42-AH 40.79 74-AH 35.29 113-AH 0.00 45-AH 61.70 75-AH 100.00 114-AH 43.33 46-AH 7.14 76-AH 41.94 115-AH 60.00 47-AH 36.54 77-AH 35.71 116-AH 80.00 48-AH 0.00 78-AH 40.00 119-AH 86.25 49-AH 33.33 79-AH 40.00 121-AH 0.00 50-AH 48.98 80-AH 38.71 122-AH 64.29 51-AH 12.50 81-AH 70.59 123-AH 100.00 53-AH 46.15 82-AH 81.25 124-AH 43.28 55-AH 50.59 83-AH 100.00 125-AH 44.19 56-AH 56.82 84-AH 100.00 126-AH 58.33 58-AH 76.92 85-AH 20.00 128-AH 50.00 59-AH 66.67 86-AH 36.11 129-AH 46.15 61-AH 0.00 87-AH 63.16 130-AH 76.92 62-AH 75.00 88-AH 87.10 15 CHESHIRE_GRID_FRE_T2FB CHESHIRE_GRID_FRE_T2FB CHESHIRE_GRID_FRE_T2FB
Grid@CLEF 2009 Monolingual French − Distribution of the Topics of the Experiment −1 131−AH133−AH134−AH135−AH136−AH137−AH138−AH139−AH140−AH 41−AH 42−AH 45−AH 46−AH 47−AH 48−AH 49−AH 50−AH 51−AH 53−AH 55−AH 56−AH 58−AH 59−AH 61−AH 62−AH
Topic Identifier
Grid@CLEF 2009 Monolingual French − Comparison to Median R−Precision by Topic (Topics 63−AH 89−AH 90−AH 91−AH 92−AH 95−AH 97−AH 98−AH
99−AH