Guideline classification, scoring method and statistical
analysis
The guidelines were classified based on their type. The categories were
as follows:
- Overarching guidelines with procedural elements (eg. stability
testing), (quasi-)universal scope (eg. first-in-human trials,
pharmacogenomics in pharmacovigilance) or intended for a specific
category of medicines (eg. Biosimilar medicines; common routes of
administration)
- Guidelines intended for a therapeutic area or an indication (eg.
products for urinary incontinence)
- Guidelines intended for a molecule (API or excipients), (eg. products
containing interferon beta), or administration (eg. IV liposomal
products)
The EMA has a template for collecting comments, which involves a first
section on overall comments to the draft (Section 1) and a second
section for the collection of detailed comments and proposed changes
(Section 2). The comments in Section 1 were excluded from the analysis
because they are more general in nature and do not refer to a line item
in the guideline and lacks a clear indication provided whether they were
accepted or rejected. Comments provided in Section 2 that relate to
contents of the draft guideline were considered. To allow quantitative
analysis of the accepted and rejected comments from different parties, a
scoring system was developed. Each fully accepted comment was scored as
2 points, partially accepted as one point and rejected comments as no
points. Acknowledgements, duplicates (same comment included in multiple
submissions) or comments that were regarded as not applicable did not
score points but were counted towards the total comments. In the case of
duplicates, only the comment indicated by the EMA as the first comment
with the same content was counted towards scoring. This decision was
taken because they would not have direct impact on the final content of
the guideline. There were only a few guidelines with many duplicate
comments and this approach is unlikely to shift the overall trend. Table
2 indicates the scoring system based on the wordings used by the EMA in
the document outlining all comments.
Table 2. Scoring system aligned
with EMA wording