Limitations
There might be challenges with coding of the comments. In some
guidelines, it was not clear whether a comment was accepted or not. Some
guidelines had many reviewers with different terminology and approach to
assessing the comments. The guidelines were coded by the research team
and there might be subjective interpretation of some comments that were
not clearly marked as accepted or rejected by the EMA.
The trade associations generally submitted a higher volume of comments
than individual companies alone. One aim for consolidating comments
through trade association is to avoid duplicative comments, but there
were also occasions of similar comments coming from different trade
associations. Furthermore, while qualitative analysis of the nature of
the comments was not included in the study, the scores may be skewed
given that some of the comments were of administrative nature, such as
typos, which were fully accepted.
Exclusion of the concept papers preceding the actual guidelines from our
study was therefore a limitation because some of the ideas are captured
in consultations on them, but the summary of consultation responses for
the concept papers are not published.
Acknowledgements The authors would like to thank Alina Mihaela
Macau, Mpharm for her contributions in collecting the data during her
employment at MSD, Belgium.
Conflict of Interests The authors are employees and
stockholders of MSD.
The data that support the findings of this study are available on the
website of European Medicines Agency at
www.ema.europa.eu. The documents can be
found through the search function using term “overview of received
comments”, using the timeframe of 2013-2017.