Technical framework for enabling high-quality measurements in new approach methodologies (NAMs)

Main Article Content

Elijah J. Petersen
John T. Elliott
John Gordon
Nicole C. Kleinstreuer
Emily Reinke
Mattias Roesslein
Blaza Toman


New approach methodologies (NAMs) are in vitro, in chemico, and in silico or computational approaches that can potentially be used to reduce animal testing. For NAMs that require laboratory experiments, it is critical that they provide consistent and reliable results. While guidance has been provided on improving the reproducibility of NAMs that require laboratory experiments, there is not yet an overarching technical framework that details how to add measurement quality features into a protocol. In this manuscript, we discuss such a framework and provide a step-by step process describing how to refine a protocol using basic quality tools. The steps in this framework include 1) conceptual analysis of sources of technical variability in the assay, 2) within laboratory evaluation of assay performance, 3) statistical data analysis, and 4) determination of method transferability (if needed). While each of these steps has discrete components, they are all inter-related and insights from any step can influence the others. Following the steps in this framework can help reveal the advantages and limitations of different choices during the design of an assay such as which in-process control measurements to include and how many replicates to use for each control measurement and for each test substance. Overall, the use of this technical framework can support optimizing NAM reproducibility, thereby supporting meeting research and regulatory needs.

Article Details

How to Cite
Petersen, E. J., Elliott, J. T., Gordon, J., Kleinstreuer, N. C., Reinke, E., Roesslein, M. and Toman, B. (2022) “Technical framework for enabling high-quality measurements in new approach methodologies (NAMs)”, ALTEX - Alternatives to animal experimentation. doi: 10.14573/altex.2205081.

Borrel, A., Huang, R., Sakamuru, S. et al. (2020a). High-Throughput Screening to Predict Chemical-Assay Interference. Scientific Reports 10, 3986. doi:10.1038/s41598-020-60747-3

Borrel, A., Mansouri, K., Nolte, S. et al. (2020b). InterPred: a webtool to predict chemical autofluorescence and luminescence interference. Nucleic Acids Research 48, W586-W590. doi:10.1093/nar/gkaa378

Cassano, J. C., Roesslein, M., Kaufmann, R. et al. (2020). A novel approach to increase robustness, precision and high-throughput capacity of single cell gel electrophoresis ALTEX - Alternatives to animal experimentation 3, 95-109. doi:10.14573/altex.1906252

Coussens, N. P., Sittampalam, G. S., Guha, R. et al. (2018). Assay Guidance Manual: Quantitative Biology and Pharmacology in Preclinical Drug Discovery. Clin Transl Sci 11, 461-470. doi:10.1111/cts.12570

CPSC (2022). Guidance: Alternative test methods and integrated testing approaches.

Elliott, J. T., Rosslein, M., Song, N. W. et al. (2017). Toward Achieving Harmonization in a Nanocytotoxicity Assay Measurement Through an Interlaboratory Comparison Study. Altex-Alternatives to Animal Experimentation 34, 201-218. doi:10.14573/altex.1605021

EPA (2019). Directive to prioritize efforts to reduce animal testing.

European Union (EU). 2010. And of the council of 22 september 2010 on the protection of animals used

for scientific purposes. Official Journal of the European Union L 276, vol. 53: 33–79.

Gerloff, K., Albrecht, C., Boots, A. W. et al. (2009). Cytotoxicity and oxidative DNA damage by nanoparticles in human intestinal Caco-2 cells. Nanotoxicology 3, 355-364. doi:10.3109/17435390903276933

Guadagnini, R., Kenzaoui, B. H., Walker, L. et al. (2015). Toxicity screenings of nanomaterials: challenges due to interference with assay processes and components of classic in vitro tests. Nanotoxicology 9, 13-24. doi:10.3109/17435390.2013.829590

Hanna, S. K., Cooksey, G. A., Dong, S. et al. (2016). Feasibility of using a standardized Caenorhabditis elegans toxicity test to assess nanomaterial toxicity. Environmental Science-Nano 3, 1080-1089. doi:10.1039/c6en00105j

ICCVAM (2018). A Strategic Roadmap for Establishing New Approaches to Evaluate the Safety of Chemicals and Medical Products in the United States. doi:10.22427/NTP-ICCVAM-ROADMAP2018

Ishikawa, K. (1985). What Is Total Quality Control? The Japanese Way. Translated by Lu, David J. (1st ed.). Englewood Cliffs, New Jersey: Prentice-Hall. ISBN 978-0-13-952433-2.

ISO (2018). ISO 19007:2018: Nanotechnologies -- In vitro MTS assay for measuring the cytotoxic effect of nanoparticles.

Lacroix, G., Koch, W., Ritter, D. et al. (2018). Air–Liquid Interface In Vitro Models for Respiratory Toxicology Research: Consensus Workshop and Recommendations. Applied In Vitro Toxicology 4, 91-106. doi:10.1089/aivt.2017.0034

Leibrock, L., Jungnickel, H., Tentschert, J. et al. (2020). Parametric optimization of an air-liquid interface system for flow-through inhalation exposure to nanoparticles: assessing dosimetry and intracellular uptake of CeO2 nanoparticles. Nanomaterials 10, article number 2369.

NCad. 2016. Ncad opinion transition to non-animal research. doi:10.3390/nano10122369

Nelson, B. C., Petersen, E. J., Marquis, B. J. et al. (2013). NIST gold nanoparticle reference materials do not induce oxidative DNA damage. Nanotoxicology 7, 21-29. doi:10.3109/17435390.2011.626537

OECD (2014). Guidance Document on the Validation of (Quantitative) Structure-Activity Relationship [(Q)SAR] Models. doi:10.1787/9789264085442-en

OECD (2018a). Guidance Document on Good In Vitro Method Practices (GIVIMP). doi:10.1787/9789264304796-en

OECD (2018b). Guidance Document on Good In Vitro Method Practices (GIVIMP). Chapter 6. doi:10.1787/9789264304796-en

OECD (2021), Guideline No. 497: Defined Approaches on Skin Sensitisation, OECD Guidelines for the Testing of Chemicals, Section 4, OECD Publishing, Paris, doi:10.1787/b92879a4-en.

Ong, K. J., MacCormack, T. J., Clark, R. J. et al. (2014). Widespread Nanoparticle-Assay Interference: Implications for Nanotoxicity Testing. Plos One 9, doi:10.1371/journal.pone.0090650

Pamies, D., Leist, M., Coecke, S. et al. (2022). Guidance document on Good Cell and Tissue Culture Practice 2.0 (GCCP 2.0). ALTEX - Alternatives to animal experimentation 39, 30-70. doi:10.14573/altex.2111011

Parish, S. T., Aschner, M., Casey, W. et al. (2020). An evaluation framework for new approach methodologies (NAMs) for human health safety assessment. Regulatory Toxicology and Pharmacology 112, 104592. doi:10.1016/j.yrtph.2020.104592

Petersen, E. J., Reipa, V., Watson, S. S. et al. (2014). DNA Damaging Potential of Photoactivated P25 Titanium Dioxide Nanoparticles. Chemical Research in Toxicology 27, 1877-1884. doi:10.1021/tx500340v

Petersen, E. J., Hirsch, C., Elliott, J. T. et al. (2020). Cause-and-Effect Analysis as a Tool To Improve the Reproducibility of Nanobioassays: Four Case Studies. Chemical Research in Toxicology 33, 1039-1054. doi:10.1021/acs.chemrestox.9b00165

Petersen, E. J., Nguyen, A. D., Brown, J. et al. (2021a). Characteristics to consider when selecting a positive control material for an in vitro assay. ALTEX - Alternatives to animal experimentation 38, 365-376. doi:10.14573/altex.2102111

Petersen, E. J., Sharma, M., Clippinger, A. J. et al. (2021b). Use of Cause-and-Effect Analysis to Optimize the Reliability of In Vitro Inhalation Toxicity Measurements Using an Air–Liquid Interface. Chemical Research in Toxicology 34, 1370-1385. doi:10.1021/acs.chemrestox.1c00080

Petersen, E. J., Elliott, J. T., Bancos, S., et al. (2022a). Predictive alternative methods for assessing biocompatibility of dental materials: A NIST-NIDCR workshop report, ALTEX - Alternatives to animal experimentation, in press.

Petersen, E. J., Ceger, P., Allen, D. et al. (2022b). U.S. federal agency interests and key considerations for new approach methodologies for nanomaterials. ALTEX - Alternatives to animal experimentation 39, 182-206. doi:10.14573/altex.2105041

Petersen, E. J., Uhl, R., Toman, B. et al. (2022c). Development of a 96-well electrophilic allergen screening assay for skin sensitization using a measurement science approach. Toxics 10(5), article number 257. doi:10.3390/toxics10050257

Roesslein, M., Hirsch, C., Kaiser, J. P. et al. (2013). Comparability of in Vitro Tests for Bioactive Nanoparticles: A Common Assay to Detect Reactive Oxygen Species as an Example. International Journal of Molecular Sciences 14, 24320-24337. doi:10.3390/ijms141224320

Rösslein, M., Elliott, J. T., Salit, M. et al. (2015). Use of Cause-and-Effect Analysis to Design a High-Quality Nanocytotoxicology Assay. Chemical Research in Toxicology 28, 21-30. doi:10.1021/tx500327y

Sullivan, K. M., Aggarwal, M., Akins, J. M. et al. (2017). Dermal absorption for pesticide health risk assessment: Harmonization of study design and data reporting for North American Regulatory submissions. Regulatory Toxicology and Pharmacology 90, 197-205. doi:10.1016/j.yrtph.2017.09.012

Tanneberger, K., Rico-Rico, A., Kramer, N. I. et al. (2010). Effects of Solvents and Dosing Procedure on Chemical Toxicity in Cell-Based in Vitro Assays. Environmental Science & Technology 44, 4775-4781. doi:10.1021/es100045y

Tanneberger, K., Knobel, M., Busser, F. J. M. et al. (2013). Predicting Fish Acute Toxicity Using a Fish Gill Cell Line-Based Toxicity Assay. Environmental Science & Technology 47, 1110-1119. doi:10.1021/es303505z

United States. 2016. Frank r. Lautenberg chemical safety for the 21st century act - public law 114–182.

Urbisch, D., Mehling, A., Guth, K. et al. (2015). Assessing skin sensitization hazard in mice and men using non-animal test methods. Regulatory Toxicology and Pharmacology 71, 337-351. doi:10.1016/j.yrtph.2014.12.008

WHO (2001). Good laboratory practice training manual: Trainer.

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J. et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data 3, 160018. doi:10.1038/sdata.2016.18

Worle-Knirsch, J. M., Pulskamp, K. and Krug, H. F. (2006). Oops they did it again! Carbon nanotubes hoax scientists in viability assays. Nano Letters 6, 1261-1268. doi:10.1021/nl060177c

Zavala, J., Freedman, A., Szilagyi, J. et al. (2020). New Approach Methods to Evaluate Health Risks of Air Pollutants: Critical Design Considerations for In Vitro Exposure Testing. International Journal of Environmental Research and Public Health 17, 2124. doi:10.3390/ijerph17062124

Most read articles by the same author(s)