Skip to main content

Does figure-text improve biomedical image retrieval?

Friday, November 08, 2013 — Poster Session IV

2:00 p.m. – 4:00 p.m.

FAES Academic Center (Upper-Level Terrace)




  • D. You
  • S.K. Antani
  • D. Demner-Fushman
  • G.R. Thoma


Figures with graphical illustrations (e.g., graphs, charts, diagrams, or clinical images) are often used in biomedical articles to convey statistical results, biomedical procedures, schematics, etc. These figures are frequently annotated with superimposed text, also called “figure-text”. It is generally assumed that this figure-text complement associated textual metadata, such as the figure captions, or bibliographic citations; and that using it for indexing figures enhances the quality of information retrieval. However, we find nothing in the literature that adequately supports this assumption of information gain. In our research, we attempt to test the validity of this assumption by conducting image retrieval experiments and comparing retrieval results based on captions alone to that using figure-text in addition to captions. We conducted a blind study in which two judges evaluated a set of figures retrieved from biomedical research articles on the topic of lung cancer. Although we find that for a number of specific queries figure-text improves retrieval performance, additional experiments are needed to determine if these improvements can be generalized across all figure types.

back to top