The purpose of reader experiments is to assess the quality of medical images for diagnostic decisions by medical professionals. Their design involves the presentation of images along with various ways of collecting reader responses, ranging from grading scales to image selection and image annotation to label specific findings. A particular challenge of reader experiments is that reader time and availability is scarce, which can easily lead to increased costs and impair the quality of the assessment.
Human Observer Net (HON) is a freely available software that was developed to give flexibility in implementing reader experiments and enable time- and cost-efficient assessment1. The software is web-based, runs in common browsers and enables experimental designs ranging from visual grading analysis (VGA) to multiple alternative forced choice (m-AFC), receiver operating characteristic (ROC), location ROC (LROC), free-response ROC (FROC) and customized designs.
The software was evaluated across more than 17,000 readings in different experimental settings, in which the mean reader response time per image was between 6 and 9 seconds, which means that readers scored an average of 400 to 600 images per hour. The software was developed in close collaboration with radiologists and good usability was confirmed experimentally.
The image shown above shows a screenshot of the software in a reader experiment performed with a customized phantom. Online links to a software demonstration, publication of the software, instructions and software download are provided below.
- Genske U, Jahnke P. Human Observer Net: A Platform Tool for Human Observer Studies of Image Data. Radiology (2022). Link