Discussion
To our knowledge, this is the first in-depth study objectively assessing the quality and reliability of video information pertaining to cholesteatoma on YouTube. There were on average 55,292 views per video included in this study with videos having an average of 271 likes and only 22 dislikes.
In otolaryngology, YouTube content has been investigated for the educational value of videos about pediatric tonsillectomy, middle ear ventilation tubes, rhinoplasty and thyroid cancer (8, 14-16). Sorenson (16) found there was a low educational quality for YouTube videos describing pediatric adenotonsillectomy with even lower quality scores in videos from patient uploaded testimonial-type content. These findings were mirrored by Strychowsky et al (15), with study findings revealing physician-uploaded content being of a better educational quality than patient experience, surgical technique and news media videos. The findings of the current study confirm that videos uploaded by patients are of a lower educational value when compared to videos from academic sources.
The DISCERN tool was developed in 1999 in order to critically appraise the reliability and validity of published health information (7) and the DISCERN instrument has since been used generically in assessing YouTube sources in several studies (8, 9, 14).
In addition to the DISCERN tool, the senior authors developed a disease specific Essential and Ideal Video Completeness Criteria (EIVC criteria) to represent the necessary video contents required in order to inform patients of all the elements of cholesteatoma and its management. Novel scoring systems have previously been developed to address the intricacies of disease factors being assessed such as Strychowsky et al who developed a usefulness checklist for validity of YouTube health information videos regarding pediatric tonsillectomy(15).
The overall educational quality of the videos examined was poor using both the DISCERN and EIVC Criteria. The DISCERN scores in this study showed a good reliability between raters and there was moderate correlation between raters in the EIVC criteria. The results show that a video’s view count, length and like tally did not correlate with the DISCERN or EIVC criteria however, the more dislikes a video had, the lower the DISCERN or EIVC score.
In a systematic review addressing the health care information on YouTube, Madathil (4) et al found that although YouTube had the potential to be a great source of patient education, it was potentially also at significant risk of bias and misinformation. It was been recommended that professional organizations and government agencies (17) take more of an active role in uploading approved videos which are more likely to meet the criteria and guidelines required. Given the poor educational content found in this study, as well as other past otolaryngology studies (8, 15, 16), the authors advocate for otolaryngology and otology societies working with patients in order to upload high quality patient education videos onto YouTube which surgeons can direct patients to view for more accurate information.
With regards to the limitations of this study, firstly it was limited to the English Language and only the top 6 search results were examined. The authors felt that this would generally match what the average patient would search for, but of course, many may search for more information. The geographical location of the YouTube user also affects the search findings and as such, our study may not be representative of the quality of videos found in other regions. The DISCERN instrument was originally created to assess written material and as such, has not been properly validated for use in the video educational material. Furthermore, the essential and ideal video completeness criteria is an unvalidated instrument based on replicating similar models of disease assessment used to analyze video educational content.