Discussion
We show that ChatGPT is not only capable of accurately obtaining typical PK parameters from literature, but also has the ability to generate functional R code for predicting drug concentration using a population PK model as well to develop an interactive Shiny application to visualize model predictions.
ChatGPT generated an one-compartment population PK model in R and updated the code based on user specifications. By using ChatGPT to develop a Shiny application in R, users inexperienced with R shiny can easily produce web applications for interpreting their models. Both applications show how ChatGPT can be used without extensive coding or programming knowledge. This can significantly reduce development time and effort, while potentially improveing user experience of such applications. Another advantage of using ChatGPT for programming is its ability to assist developers in identifying and fixing errors in their code. ChatGPT can suggest possible solutions for errors and other coding mistakes, which helps inexperienced users to debug their code19,20. This feature can help streamline the development process and improve the overall quality of R code.
There are also some limitations and challenges to the use of ChatGPT for applications related to pharmacometrics. For example, the accuracy and reliability of AI-generated data may be affected by biases and knowledge gaps in the training data or the complexity of the query, for example when asking to produce code for more complex biological systems21,22. It can be especially difficult for inexperienced users to detect errors in the responses provided by ChatGPT, potentially with errors getting into proposed model code, affecting downstream results. This may lead to inaccurate or misleading results. Additionally, the lack of transparency and interpretability of AI algorithms may raise ethical concerns and limit their widespread adoption23,24. A more practical limitation of ChatGPT v3.5 that was used In this study is that response size is limited, potentially cutting off longer blocks of code. It can still present a challenge for users who require complete and accurate code snippets. Although ChatGPT might be especially interesting for inexperienced programmers, it might still be necessary to carefully review and edit code generated by ChatGPT to ensure its correctness and completeness. It is possible that future versions of ChatGPT may address some of the limitations observed in version 3.5, including the truncation of longer blocks of code. Another limitation to consider is that ChatGPT did not seem to be able to generate NONMEM control streams very well, which is unfortunate as NONMEM is the gold standard in pharmacometrics research, and the identification of errors in these streams can greatly aid students learning to use it25. This may be due to the limited availability of publicly available control streams, making it difficult for ChatGPT to learn from and generate accurate and reliable code for NONMEM models.
In conclusion, the integration of ChatGPT in pharmacometrics has the potential to streamline the development process and improve the user experience for pharmacometrics researchers. We deem it unlikely that ChatGPT will replace pharmacometricians in its current state. ChatGPT does have great value with respect to aiding researchers in finding and explaining information, generating and helping to debug code, and the education of new generations of pharmacometricians. As ChatGPT continues to evolve and improve, it has the potential to become an even more valuable tool in the field of pharmacometrics. As ChatGPT continues to evolve, it is likely that other pharmacometricians will find new and innovative ways to integrate it into their workflows and further enhance its capabilities in the field of pharmacometrics.
Acknowledgements
The SYMPHONY consortium aims to orchestrate personalized treatment in patients with bleeding disorders, and is a unique collaboration between patients, healthcare professionals and translational & fundamental researchers specialized in inherited bleeding disorders, as well as experts from multiple disciplines. It aims to identify best treatment choice for each individual based on bleeding phenotype. In order to achieve this goal, work packages have been organized according to three themes e.g. Diagnostics (workpackage 3&4); Treatment (workpackages 5–9) and Fundamental Research (workpackages 10–12). This research received funding from the Netherlands Organization for Scientific Research (NWO) in the framework of the NWA-ORC Call grant agreement NWA.1160.18.038. Principal investigator: Dr M.H. Cnossen; project coordinator: Dr S.H. Reitsma.
Beneficiaries of the SYMPHONY consortium: Erasmus University Medical Center-Sophia Children’s Hospital, project leadership and coordination; Sanquin Diagnostics; Sanquin Research; Amsterdam University Medical Centers; University Medical Center Groningen; University Medical Center Utrecht; Leiden University Medical Center; Radboud University Medical Center; Netherlands Society of Hemophilia Patients (NVHP); Netherlands Society for Thrombosis and Hemostasis (NVTH); Bayer B.V., CSL Behring B.V., Swedish Orphan Biovitrum (Belgium) BVBA/SPRL.
Autorship contributions: M.E.C. performed the analysis. M.E.C., S.F.K. and A.J. wrote the manuscript. All authors contributed substantially to the critical revision of the manuscript and approved the final draft.
Conflict of interest statement
M.H.C.’s institution has received investigator-initiated research and travel grants as well as speaker fees over the years from the Netherlands Organisation for Scientific Research (NWO) and Netherlands National research Agenda (NWA), the Netherlands Organization for Health Research and Development (ZonMw), the Dutch Innovatiefonds Zorgverzekeraars, Baxter/Baxalta/Shire/ Takeda, Pfizer, Bayer Schering Pharma, CSL Behring, Sobi Biogen, Novo Nordisk, Novartis and Nordic Pharma, and for serving as a steering board member for Roche, Bayer and Novartis for which fees go to the Erasmus MC as an institution. R.A.A.M. has received grants from governmental and societal research institutes such as NWO, ZonMW, Dutch Kidney Foundation and Innovation Fund and unrestricted investigator research grants from Baxter/ Baxalta/ Shire/Takeda, Bayer, CSL Behring, Sobi and CelltrionHC. He has served as advisor for Bayer, CSL Behring, Merck Sharp & Dohme, Baxter/ Baxalta/ Shire/Takeda. All grants and fees paid to the institution. Other authors have no conflict of interest to declare for this paper
Funding information
M.E.C., A.J. and S.F.K. are funded by the SYMPHONY consortium.
Data availability statement
Not applicable
References
1. Bohr A, Memarzadeh K. The Rise of Artificial Intelligence in Healthcare Applications .; 2020. doi:10.1016/B978-0-12-818438-7.00002-2
2. Gordijn B, Have H ten. ChatGPT: evolution or revolution? Med Heal Care Philos . 2023;26(1):1-2. doi:10.1007/s11019-023-10136-0
3. Radford A, Narasimhan K, Salimans T, Sutskever I, others. Improving language understanding by generative pre-training. Published online 2018.
4. Floridi L, Chiriatti M. GPT-3: Its Nature, Scope, Limits, and Consequences. Minds Mach . 2020;30(4):681-694. doi:10.1007/s11023-020-09548-1
5. Taecharungroj V. “What Can ChatGPT Do?” Analyzing Early Reactions to the Innovative AI Chatbot on Twitter. Big Data Cogn Comput . 2023;7(1):35. doi:10.3390/bdcc7010035
6. Liévin V, Hother CE, Winther O. Can large language models reason about medical questions? Published online 2022:1-33. doi:https://doi.org/10.48550/arXiv.2207.08143
7. Kashefi A, Mukerji T. ChatGPT for Programming Numerical Methods. Published online 2023:1-51. http://arxiv.org/abs/2303.12093
8. Temsah O, Khan SA, Chaiah Y, et al. Overview of Early ChatGPT’s Presence in Medical Literature: Insights From a Hybrid Literature Review by ChatGPT and Human Experts. Cureus . 2023;15(4). doi:10.7759/cureus.37281
9. Dahmen J, Kayaalp ME, Ollivier M, et al. Artificial intelligence bot ChatGPT in medical research: the potential game changer as a double-edged sword. Knee Surgery, Sport Traumatol Arthrosc . 2023;31(4):1187-1189. doi:10.1007/s00167-023-07355-6
10. Zhu JJ, Jiang J, Yang M, Ren ZJ. ChatGPT and Environmental Research.Environ Sci Technol . Published online 2023:1-4. doi:10.1021/acs.est.3c01818
11. Sallam M. The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations. medRxiv . Published online 2023:2023.02.19.23286155. http://medrxiv.org/content/early/2023/02/21/2023.02.19.23286155.abstract
12. Franchini M, Mannucci P. Past, present and future of hemophilia: a narrative review. Orphanet J Rare Dis . 2012;7(1):24. doi:10.1186/1750-1172-7-24
13. Knibbe CAJ, Zuideveld KP, Aarts LPHJ, Kuks PFM, Danhof M. Allometric relationships between the pharmacokinetics of propofol in rats, children and adults. Br J Clin Pharmacol . 2005;59(6):705-711. doi:10.1111/j.1365-2125.2005.02239.x
14. Wojciechowski J, Hopkins AM, Upton RN. Interactive pharmacometric applications using R and the Shiny package. CPT Pharmacometrics Syst Pharmacol . 2015;4(3):146-159. doi:10.1002/psp4.21
15. Berntorp E, Hermans C, Solms A, Poulsen L, Mancuso ME. Optimising prophylaxis in haemophilia A: The ups and downs of treatment.Blood Rev . 2021;50. doi:10.1016/j.blre.2021.100852
16. Björkman S, Oh M, Spotts G, et al. Population pharmacokinetics of recombinant factor VIII: the relationships of pharmacokinetics to age and body weight. Blood . 2012;119(2):612-618. doi:10.1182/blood-2011-07-360594
17. McEneny-King A, Chelle P, Foster G, Keepanasseril A, Iorio A, Edginton AN. Development and evaluation of a generic population pharmacokinetic model for standard half-life factor VIII for use in dose individualization. J Pharmacokinet Pharmacodyn . 2019;46(5):411-426. doi:10.1007/s10928-019-09634-7
18. Bukkems LH, Preijers T, Van Spengler MWF, Leebeek FWG, Cnossen MH, Mathôt RAA. Comparison of the Pharmacokinetic Properties of Extended Half-Life and Recombinant Factor VIII Concentrates by in Silico Simulations. Thromb Haemost . 2021;121(6):731-740. doi:10.1055/s-0040-1721484
19. Ahmad A, Waseem M, Liang P, Fehmideh M, Aktar MS, Mikkonen T. Towards Human-Bot Collaborative Software Architecting with ChatGPT. Published online 2023:1-7. http://arxiv.org/abs/2302.14600
20. Biswas S. Role of ChatGPT in Computer Programming.Mesopotamian J Comput Sci . Published online 2023:8-16. doi:10.58496/mjcsc/2023/002
21. Borji A. A Categorical Archive of ChatGPT Failures. Published online 2023:1-41. http://arxiv.org/abs/2302.03494
22. Mann DL. Artificial Intelligence Discusses the Role of Artificial Intelligence in Translational Medicine: A JACC: Basic to Translational Science Interview With ChatGPT. JACC Basic to Transl Sci . 2023;8(2):221-223. doi:10.1016/j.jacbts.2023.01.001
23. Editorials N. Tools such as ChatGPT threaten transparent science; here are our ground rules for their use. Nature . 2023;613:612.
24. Biswas S. ChatGPT and the Future of Medical Writing.Radiology . 2023;307(2):3-5. doi:10.1148/radiol.223312
25. Keizer RJ, Karlsson MO, Hooker A. Modeling and simulation workbench for NONMEM: Tutorial on Pirana, PsN, and Xpose. CPT Pharmacometrics Syst Pharmacol . 2013;2(6):1-9. doi:10.1038/psp.2013.24
Tables
Not applicable
Figure legends
Figure 1. R code for simulated Factor VIII levels over time in a one-compartment pharmacokinetic model . The model and R code for the one-compartment population pharmacokinetic model were generated by ChatGPT.