Materials and Methods
Microorganisms and growth conditions: The list of microbes contaminating the pharmaceutical industry was identified from the FDA’s manual of pharmaceutical microbiology and we also included few environmental microbial source found in pharmaceutical industry [6–10] The list of microbes/cells used in the study and media used for culturing these strains are listed in Table 1.
Raman substrate fabrication and sample preparation: The substrates (21 mm x 21 mm) were made from polished stainless steel with alumina and were coated with a thin layer (50 nm) of Teflon using spin coater as described previously [11] (Figure 1). The surface characteristics of the substrates were performed with the Hitachi S-4800 field emission scanning electron microscope (SEM). The microbes were cultured overnight to obtain 108 cells/mL (as measured by optical density at 600 nm of 0.1 for bacteria, and 0.6 for fungi. The overnight grown cultures were fixed with 2.5 % of glutaraldehyde and washed with water to remove the debris and diluted to a concentration of 105 cells/mL for Raman dataset development. The CHO cells were cultured up to 80% confluent inT75 cell culture flask and the cells were trypsinized and processed for Raman spectroscopy as mentioned by Rangan et.al [12]. 105cells/mL of CHO cells were measured using the Invitrogen Countess™ Automated Cell Counter. The prepared cells/microbes were placed in the substrate using a micropipette (5 µl) on the substrate and dried for 5 mins. Once dry, the sample forms a circular spot on the substrate with a diameter of about 2 mm. The dried cells on the substrate are used to collect the Raman spectra for individual species of microbes/cells. Raman measurements were performed with a customized, micro-Raman system with an argon-ion laser (532nm, 20 mW power at the sample) with thermoelectrically cooled charge-coupled device detector (1,340 pixels x 4000 pixels) mounted on a 300-mm focal length imaging with a working distance of 20 mm as described previously [13]. The spectra were collected on three different days (biological replicates) and 10 different points (technical replicates) on the 2 mm spot. At each point, 200 scans were obtained; a total of 2000 scans were obtained for each microbe/cell every day (10 points x 200 scans/point = 2000 scans). These 6000 spectra were used for the deep learning-based analyses. The Raman spectra signal to noise ratio was 1,000:1 and there is almost no interference of the background (Figure S1).
Deep learning-based classification between the potential microbial contaminants: The architecture for deep learning is composed of the following three layers: 1) initial convolution layer, 2) eight residual blocks, and 3) fully connected layer [14]. The convolution layer is composed of a kernel size of 7 and stride of 2. All the residual blocks consist of kernels with a size of 3 and strides of 1 and 2 [14]. The convolution layer proceeds with the batch normalization layer [15], and ReLU (Rectified Linear Unit) is used as a non-linear function. The residual blocks contain a shortcut connection between input and output, which enhances the training stability and addresses the problem of degradation in the deep neural network [14].
The output of the model is a 1-d (\(R^{d}\),\(R\in[0,1]\)) vector containing the probability distribution over all the classes of bacteria. To train the model, we used Adam optimizer with betas = (0.9, 0.999), and the learning rate is set to 0.001. The factor of 0.1 decays the learning rate if the accuracy on the validation set reaches a plateau during training [16]. In order to train the model, we use 5-fold Leave-One-Out Cross-Validation (LOOCV) method to split the collected data set into training and validation sets. In this method, the reference data set is randomly split into five groups, and in each round of training, one group is held out to be used as the validation set and the remaining data is used as the training set. This process is repeated five times to ensure that all the samples fall into the validation set once. The performance of the model was evaluated on the individual class scale to form a confusion matrix. Furthermore, using Grad-Cam++, we developed a saliency map for each sample that shows the attention map of each microbe/cell with the Raman spectra [17]. With this feature, we can explain how the deep learning model chooses a class for an arbitrary input by providing the corresponding attention map.
Acknowledgment: This work was partially funded by the grant “Continuous Manufacturing of Biologics,” funded by Purdue College of Engineering’s Faculty Conversations (EFC) and by the National Science Foundation (CBET 1700961).
References:
1. Shintani, H (2016). Validation Study of Rapid Assays of Bioburden, Endotoxins and Other Contamination. Biocontrol Sciences , 21, 63–72 . https://doi.org/10.4265/bio.21.63
2. Jiang, M., Severson, KA., Love, JC., Madden, H., Swann, P., Zang, L., Braatz, RD. (2017). Opportunities and challenges of real-time release testing in biopharmaceutical manufacturing. Biotechnology Bioengineering , 114, 2445–2456 . https://doi.org/10.1002/bit.26383
3. England, MR., Stock, F., Gebo, JET., Frank, KM., Lau, AF. (2019). Comprehensive Evaluation of Compendial USP<71>, BacT/Alert Dual-T, and Bactec FX for Detection of Product Sterility Testing Contaminants. Journal of Clinical Microbiology , 57, 1548-18. https://doi.org/10.1128/JCM.01548-18
4. Surrette, C., Scherer, B., Corwin, A., Grossmann, G., Kaushik, AM., Hsieh, K., Zhang, P., Liao, JC., Wong, PK., Wang, TH., Puleo, CM. (2018). Rapid Microbiology Screening in Pharmaceutical Workflows.SLAS Technol , 23. 387–394 . https://doi.org/10.1177/2472630318779758
5. Ho, C-S., Jean, N., Hogan, CA., Blackmon, L., Jeffrey, SS., Holodniy, M., Banaei, N., Saleh, AAE., Ermon, S., Dionne, J. (2019). Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning. Nature Communications , 10. 4927. https://doi.org/10.1038/s41467-019-12898-9
6. Pharmaceutical Microbiology Manual | FDA. https://www.fda.gov/media/88801. Accessed 2 Jan 2020
7. Deal, A., Klein, D., Lopolito, P., Schwarz, JS. (2016). Cleaning and Disinfection of Bacillus cereus Biofilm. PDA Journal of Pharmaceutical Science and Technology , 70. 208–217. https://doi.org/10.5731/pdajpst.2014.005165
8. Pacheco, FLC., Pinto, TDJA. (2010). The bacterial diversity of pharmaceutical clean rooms analyzed by the Fatty Acid methyl ester technique. PDA Journal of Pharmaceutical Science and Technology , 64.156–166 https://pubmed.ncbi.nlm.nih.gov/21502015/
9. Salaman-Byron, AL. (2019). Probable Scenarios of Process Contamination with Cutibacterium (Propionibacterium) acnes in Mammalian Cell Bioreactor. PDA Journal of Pharmaceutical Science and Technology . pdajpst.2019.010710. https://doi.org/10.5731/pdajpst.2019.010710
10. Cobo, F., Concha, Á. (2007). Environmental microbial contamination in a stem cell bank. Letters in Applied Microbiology , 44, 379–386 . https://doi.org/10.1111/j.1472-765X.2006.02095.x
11. Zhang, D., Xie, Y., Mrozek, MF., Ortiz, C., Davisson, VJ., Ben-Amotz, D. (2003) Raman Detection of Proteomic Analytes.Analytical Chemistry , 75, 5703–5709 . https://doi.org/10.1021/ac0345087
12. Rangan, S., Kamal, S., Konorov, SO., Schulze, HG., Blades, MW., Turner, RFB., Piret, JM. (2018). Types of cell death and apoptotic stages in Chinese Hamster Ovary cells distinguished by Raman spectroscopy. Biotechnology Bioengineering , 115:401–412 . https://doi.org/10.1002/bit.26476
13. Davis, JG., Gierszal, KP., Wang, P., Ben-Amotz D. (2012) Water structural transformation at molecular hydrophobic interfaces.Nature 491:582–585 . https://doi.org/10.1038/nature11570
14. He, Kaiming., Zhang, Xiangyu., Ren, Shaoqing., Sun, Jian. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 770-778. https://doi.org/10.1109/CVPR.2016.90
15. Ioffe, S., (2017). Batch renormalization: Towards reducing minibatch dependence in batch-normalized models. Conference on Advances in neural information processing systems , 1945-1953, http://papers.nips.cc/paper/6790-batch-renormalization-towards-reducing-minibatch-dependence-in-batch-normalized-models.pdf.
16. Kingma, DP., Ba, J. (2015) Adam: A Method for Stochastic Optimization.3rd International conference on learning representations , ICLR, Conference track proceedings. https://arxiv.org/pdf/1412.6980.pdf
17. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks - IEEE Conference Publication. https://ieeexplore.ieee.org/document/8354201. Accessed 3 Jan 2020
18. Krizhevsky, A., Sutskever, I., Hinton, GE. (2012). ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1. Curran Associates Inc., Lake Tahoe, Nevada, pp 1097–1105
19. Teng, L., Wang, X., Wang, X., Gou, H., Ren, L., Wang, T., Wang, Y., Ji, Y., Huang, WE., Xu, J. (2016). Label-free, rapid and quantitative phenotyping of stress response in E. coli via ramanome. Scientific Reports , 6. https://doi.org/10.1038/srep34359
20. Ren, Y., Ji, Y., Teng, L., Zhang, H. (2017). Using Raman spectroscopy and chemometrics to identify the growth phase of Lactobacillus casei Zhang during batch culture at the single-cell level.Microbial Cell Factories , 16. https://doi.org/10.1186/s12934-017-0849-8
21. Naja, G., Bouvrette, P., Hrapovic, S., Luong, JHT. (2007). Raman-based detection of bacteria using silver nanoparticles conjugated with antibodies. Analyst 132, 679–686. https://doi.org/10.1039/B701160A
22. Razek, SA., Ayoub, AB., Swillam, MA., (2019) One Step Fabrication of Highly Absorptive and Surface Enhanced Raman Scattering (SERS) Silver Nano-trees on Silicon Substrate. Scientific Reports 9, 1–8 . https://doi.org/10.1038/s41598-019-49896-2
Table 1: List of microbes/cells used in this study.