Interobserver Agreement in the Interpretation of Single-Field Digital Fundus Images for Diabetic Retinopathy Screening
Received 4 February 2005; accepted 28 November 2005.
Purpose
To assess agreement among a group of ophthalmic care providers, including ophthalmologists and trained nonphysician personnel, in the interpretation of single-field digital fundus images for diabetic retinopathy screening.
Design
Interobserver reliability study.
Participants
Twelve ophthalmic care personnel, including 3 retina specialists, 3 general ophthalmologists, 3 ophthalmic nurses, and 3 ophthalmic photographers.
Methods
All participants were to read 400 good single-field digital fundus images of diabetic patients from a community hospital. The nonphysician personnel group read the images 1 month after attending a 2-day intensive instruction course regarding diabetic retinopathy screening. The ophthalmologists read the images without additional training. The 3 retina specialists read the images again together 2 months later to form a consensus regarding retinopathy severity and macular edema for each case. All readers used the Early Treatment Diabetic Retinopathy Study standard photographs as guidelines.
Main Outcome Measures
The κ statistic was used for the reliability assessment of the diabetic retinopathy severity and macular edema, and for the identification of cases that needed referral to ophthalmologists.
Results
There is only fair agreement among all readers. The multirater κ coefficient for retinopathy severity is 0.34; for macular edema, 0.27; and for referral cases, 0.28. Retina specialists have the best agreement among all groups (κ = 0.58 for retinopathy severity or macular edema, κ = 0.63 for referrals). There is also fair agreement when all readers are compared with the consensus of retina specialists (κs = 0.35, 0.28, and 0.29 for retinopathy severity, macular edema, and referrals, respectively), and the retina specialist group also has the best agreement (κs = 0.63, 0.65, and 0.67 for retinopathy severity, macular edema, and referrals).
Conclusions
Without additional training, retina specialists may be the most reliable personnel to interpret single-field digital fundus images for diabetic retinopathy screening. For other ophthalmic care personnel to achieve comparable reliability, a comprehensive instruction course with specific continuing education is essential. Authorized nonphysician interpreters should be experts, and new standard photographs for single-field digital fundus image interpretation may also be required to improve interobserver reliability.
1Department of Ophthalmology, Rajavithi Hospital, Bangkok, Thailand
2Department of Mathematics, Faculty of Science, Mahidol University, Bangkok, Thailand
Correspondence and reprint requests to Paisan Ruamviboonsuk, MD, Department of Ophthalmology, Rajavithi Hospital, Bangkok, Thailand 10400
Manuscript no. 2005-113.
The authors have no proprietary interest in any aspect of the study.
Supported by research funds from Rajavithi Hospital, Bangkok, Thailand (grant no.: 26/2547).
ABSTRACT