Guideline classification, scoring method and statistical analysis
The guidelines were classified based on their type. The categories were as follows:
The EMA has a template for collecting comments, which involves a first section on overall comments to the draft (Section 1) and a second section for the collection of detailed comments and proposed changes (Section 2). The comments in Section 1 were excluded from the analysis because they are more general in nature and do not refer to a line item in the guideline and lacks a clear indication provided whether they were accepted or rejected. Comments provided in Section 2 that relate to contents of the draft guideline were considered. To allow quantitative analysis of the accepted and rejected comments from different parties, a scoring system was developed. Each fully accepted comment was scored as 2 points, partially accepted as one point and rejected comments as no points. Acknowledgements, duplicates (same comment included in multiple submissions) or comments that were regarded as not applicable did not score points but were counted towards the total comments. In the case of duplicates, only the comment indicated by the EMA as the first comment with the same content was counted towards scoring. This decision was taken because they would not have direct impact on the final content of the guideline. There were only a few guidelines with many duplicate comments and this approach is unlikely to shift the overall trend. Table 2 indicates the scoring system based on the wordings used by the EMA in the document outlining all comments.
Table 2. Scoring system aligned with EMA wording