Limitations
There might be challenges with coding of the comments. In some guidelines, it was not clear whether a comment was accepted or not. Some guidelines had many reviewers with different terminology and approach to assessing the comments. The guidelines were coded by the research team and there might be subjective interpretation of some comments that were not clearly marked as accepted or rejected by the EMA.
The trade associations generally submitted a higher volume of comments than individual companies alone. One aim for consolidating comments through trade association is to avoid duplicative comments, but there were also occasions of similar comments coming from different trade associations. Furthermore, while qualitative analysis of the nature of the comments was not included in the study, the scores may be skewed given that some of the comments were of administrative nature, such as typos, which were fully accepted.
Exclusion of the concept papers preceding the actual guidelines from our study was therefore a limitation because some of the ideas are captured in consultations on them, but the summary of consultation responses for the concept papers are not published.
Acknowledgements The authors would like to thank Alina Mihaela Macau, Mpharm for her contributions in collecting the data during her employment at MSD, Belgium.
Conflict of Interests The authors are employees and stockholders of MSD.
The data that support the findings of this study are available on the website of European Medicines Agency at www.ema.europa.eu. The documents can be found through the search function using term “overview of received comments”, using the timeframe of 2013-2017.