Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks
Article Sidebar
Main Article Content
Abstract
The workshop titled “Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks” was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA) and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development, and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and vice versa, the development and improvement of evidence mapping and text mining methods and tools, as well as a call for a fundamental change in chemical risk and uncertainty assessment methodology if to be conducted based on AOPs and new approach methodologies (NAM). The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders’ trust for implementation of NAM evidence and AOPs into chemical risk assessment.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles are distributed under the terms of the Creative Commons Attribution 4.0 International license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is appropriately cited (CC-BY). Copyright on any article in ALTEX is retained by the author(s).
Abbott, A. (2009). Toxicity testing gets a makeover. Nature 461, 158. doi:10.1038/461158a
Ågerstrand, M., Breitholtz, M. and Rudén, C. (2011a). Comparison of four different methods for reliability evaluation of ecotoxicity data: a case study of non-standard test data used in environmental risk assessments of pharmaceutical substances. Environ Sci Eur 23, 17. doi:10.1186/2190-4715-23-17
Ågerstrand, M., Küster, A., Bachmann, J. et al. (2011b). Reporting and evaluation criteria as means towards a transparent use of ecotoxicity data for environmental risk assessment of pharmaceuticals. Environ Poll 159, 2487-2492. doi:10.1016/j.envpol.2011.06.023
Alonso-Coello, P., Schünemann, H. J., Moberg, J. et al. (2016a). GRADE evidence to decision (EtD) frameworks: A systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 353, i2016. doi:10.1136/bmj.i2016
Alonso-Coello, P., Oxman, A. D., Moberg, J. et al. (2016b). GRADE evidence to decision (EtD) frameworks: A systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ 353, i2089. doi:10.1136/bmj.i2089
Anderson, L. M., Petticrew, M., Rehfuess, E. et al. (2011). Using logic models to capture complexity in systematic reviews. Res Syn Meth 2, 33-42. doi:10.1002/jrsm.32
Ankley, G. T., Bennett, R. S., Erickson, R. J. et al. (2010). Adverse outcome pathways: A conceptual framework to support ecotoxicology research and risk assessment. Environ Toxicol Chem 29, 730-741. doi:10.1002/etc.34
Ankley, G. T. and Edwards, S. W. (2018). The adverse outcome pathway: A multifaceted framework supporting 21st century toxicology. Curr Opin Toxicol 9, 1-7. doi:10.1016/j.cotox.2018.03.004
Arnesdotter, E., Spinu, N., Firman, J. et al. (2021). Derivation, characterisation and analysis of an adverse outcome pathway network for human hepatotoxicity. Toxicology 459, 152856. doi:10.1016/j.tox.2021.152856
Arzuaga, X., Smith, M. T., Gibbons, C. F. et al. (2019). Proposed key characteristics of male reproductive toxicants as an approach for organizing and evaluating mechanistic evidence in human health hazard assessments. Environ Health Perspect 127, 65001. doi:10.1289/EHP5045
Bal-Price, A. and Meek, M. B. (2017). Adverse outcome pathways: Application to enhance mechanistic understanding of neurotoxicity. Pharmacol Ther 179, 84-95. doi:10.1016/j.pharmthera.2017.05.006
Bal-Price, A., Lein, P. J., Keil, K. P. et al. (2017). Developing and applying the adverse outcome pathway concept for understanding and predicting neurotoxicity. Neurotoxicology 59, 240-255. doi:10.1016/j.neuro.2016.05.010
Becker, R. A., Ankley, G. T., Edwards, S. W. et al. (2015). Increasing scientific confidence in adverse outcome pathways: Application of tailored Bradford-Hill considerations for evaluating weight of evidence. Regul Toxicol Pharmacol 72, 514-537. doi:10.1016/j.yrtph.2015.04.004
Beronius, A., Molander, L., Zilliacus, J. et al. (2018). Testing and refining the science in risk assessment and policy (SciRAP) webbased platform for evaluating the reliability and relevance of in vivo toxicity studies. J Appl Toxicol 38, 1460-1470. doi:10.1002/jat.3648
Bozada, T. Jr., Borden, J., Workman, J. et al. (2021). Sysrev: A FAIR platform for data curation and systematic evidence review. Front Artif Intell 4, 685298. doi:10.3389/frai.2021.685298
Burnett, S. D., Blanchette, A. D., Grimm, F. A. et al. (2019). Population-based toxicity screening in human induced pluripotent stem cell-derived cardiomyocytes. Toxicol Appl Pharmacol 381, 114711. doi:10.1016/j.taap.2019.114711
Casati, S. (2018). Integrated approaches to testing and assessment. Basic Clin Pharmacol Toxicol 123, Suppl 5, 51-55. doi:10.1111/bcpt.13018
Darney, K., Lautz, L. S., Béchaux, C. et al. (2021). Human variability in polymorphic CYP2D6 metabolism: Implications for the risk assessment of chemicals in food and emerging designer drugs. Environ Int 156, 106760. doi:10.1016/j.envint.2021.106760
De Vries, R. and Whaley, P. (2018). In vitro critical appraisal tool (IV-CAT): Tool development protocol. Zenodo. doi:10.5281/zenodo.1493498
De Vries, R. B. M., Angrish, M., Browne, P. et al. (2021). Applying evidence-based methods to the development and use of adverse outcome pathways. ALTEX 38, 336-347. doi:10.14573/altex.2101211
Dishaw, L., Yost, E., Arzuaga, X. et al. (2020). A novel study evaluation strategy in the systematic review of animal toxicology studies for human health assessments of environmental chemicals. Environ Int 141, 105736. doi:10.1016/j.envint.2020.105736
Dornbos, P. and LaPres, J. J. (2018). Incorporating population-level genetic variability within laboratory models in toxicology: From the individual to the population. Toxicology 395, 1-8. doi:10.1016/j.tox.2017.12.007
Ede, J. D., Lobaskin, V., Vogel, U. et al. (2020). Translating scientific advances in the AOP framework to decision making for nanomaterials. Nanomaterials 10, 1229. doi:10.3390/nano10061229
EFSA (European Food Safety Authority) and EBTC (Evidence-Based Toxicology Collaboration) (2018). EFSA Scientific Colloquium 23: Evidence integration in risk assessment: The science of combining apples and oranges. EFSA Supporting Publication 16, 1396E. doi:10.2903/sp.efsa.2018.EN-1396
EFSA PPR Panel (EFSA Panel on Plant Protection Products and their Residues), Ockleford, C., Adriaanse, P. et al. (2017). Scientific Opinion on the investigation into experimental toxicological properties of plant protection products having a potential link to Parkinson’s disease and childhood leukaemia. EFSA J 15, e4691. doi:10.2903/j.efsa.2017.4691
Fox, D. M. (2010). The Convergence of Science and Governance: Research, Health Policy, and American States. University of California Press.
Gilbert, M. E., Rovet, J., Chen, Z. et al. (2012). Developmental thyroid hormone disruption: Prevalence, environmental contaminants and neurodevelopmental consequences. Neurotoxicology 33, 842-852. doi:10.1016/j.neuro.2011.11.005
Gilmour, N., Kern, P. S., Alépée, N. et al. (2020). Development of a next generation risk assessment framework for the evaluation of skin sensitisation of cosmetic ingredients. Regul Toxicol Pharmacol 116, 104721. doi:10.1016/j.yrtph.2020.104721
Guyatt, G. H., Oxman, A. D., Vist, G. E. et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336, 924-926. doi:10.1136/bmj.39489.470347.AD
Guyatt, G. H., Oxman, A. D., Vist, G. et al. (2011). GRADE guidelines: 4. Rating the quality of evidence – Study limitations (risk of bias). J Clin Epidemiol 64, 407-415. doi:10.1016/j.jclinepi.2010.07.017
Hartung, T., Hoffmann, S. and Stephens, M. (2013). Mechanistic validation. ALTEX 30, 119-130. doi:10.14573/altex.2013.2.119
Higgins, J. P. T., Chandler, T. J., Cumpston, M. et al. (2019). Cochrane Handbook for Systematic Reviews of Interventions. Chichester, UK: John Wiley & Sons. https://training.cochrane.org/handbook/current (accessed 14.02.2022).
Hoffmann, S., de Vries, R. B. M., Stephens, M. L. et al. (2017). A primer on systematic reviews in toxicology. Arch Toxicol 91, 2551-2575. doi:10.1007/s00204-017-1980-3
Horvat, T., Landesmann, B., Lostia, A. et al. (2017). Adverse outcome pathway development from protein alkylation to liver fibrosis. Arch Toxicol 91, 1523-1543. doi:10.1007/s00204-016-1814-8
Institute of Medicine (2011). Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC, USA: National Academies Press. doi:10.17226/13059
Ives, C., Campia, I., Wang, R. L. et al. (2017). Creating a structured AOP knowledgebase via ontology-based annotations. Appl In Vitro Toxicol 3, 298-311. doi:10.1089/aivt.2017.0017
Johnson, P. I., Koustas, E., Vesterinen, H. M. et al. (2016). Application of the navigation guide systematic review methodology to the evidence for developmental and reproductive toxicity of triclosan. Environ Int 92, 716-728. doi:10.1016/j.envint.2016.03.009
Judson, R. S., Houck, K. A., Kavlock, R. J. et al. (2010). In vitro screening of environmental chemicals for targeted testing prioritization: The ToxCast project. Environ Health Perspect 118, 485-492. doi:10.1289/ehp.0901392
Kavlock, R. J., Austin, C. P. and Tice, R. R. (2009). Toxicity testing in the 21st century: Implications for human health risk assessment. Risk Anal 29, 485-487. doi:10.1111/j.1539-6924.2008.01168.x
Klimisch, H. J., Andreae, M. and Tillmann, U. (1997). A systematic approach for evaluating the quality of experimental toxicological and ecotoxicological data. Regul Toxicol Pharmacol 25, 1-5. doi:10.1006/rtph.1996.1076
Knapen, D., Stinckens, E., Cavallin, J. E. et al. (2020). Toward an AOP network-based tiered testing strategy for the assessment of thyroid hormone disruption. Environ Sci Technol 54, 8491-8499. doi:10.1021/acs.est.9b07205
Knight, A. (2008). Non-animal methodologies within biomedical research and toxicity testing. ALTEX 25, 213-231. doi:10.14573/altex.2008.3.213
Krithikadatta, J., Gopikrishna, V. and Datta, M. (2014). CRIS guidelines (checklist for reporting in-vitro studies): A concept note on the need for standardized guidelines for improving quality and transparency in reporting in-vitro studies in experimental dental research. J Conserv Dent 17, 301-304. doi:10.4103/0972-0707.136338
La Merrill, M. A., Vandenberg, L. N., Smith, M. T. et al. (2019). Consensus on the key characteristics of endocrine-disrupting chemicals as a basis for hazard identification. Nat Rev Endocrinol 16, 45-57. doi:10.1038/s41574-019-0273-8
Lam, J., Koustas, E., Sutton, P. et al. (2014). The navigation guide – Evidence-based medicine meets environmental health: Integration of animal and human evidence for PFOA effects on fetal growth. Environ Health Perspect 122, 1040-1051. doi:10.1289/ehp.1307923
Lam, J., Sutton, P., Kalkbrenner, A. et al. (2016). A systematic review and meta-analysis of multiple airborne pollutants and autism spectrum disorder. PLoS One 11, e0161851. doi:10.1371/journal.pone.0161851
Lam, J., Lanphear, B. P., Bellinger, D. et al. (2017). Developmental PBDE exposure and IQ/ADHD in childhood: A systematic review and meta-analysis. Environ Health Perspect 125, 086001. doi:10.1289/EHP1632
Leist, M., Ghallab, A., Graepel, R. et al. (2017). Adverse outcome pathways: Opportunities, limitations and open questions. Arch Toxicol 91, 3477-3505. doi:10.1007/s00204-017-2045-3
Li, S. A., Alexander, P. E., Reljic, T. et al. (2018). Evidence to decision framework provides a structured “roadmap” for making GRADE guidelines recommendations. J Clin Epidemiol 104, 103-112. doi:10.1016/j.jclinepi.2018.09.007
Liu, Z., Shi, Q., Ding, D. et al. (2011). Translating clinical findings into knowledge in drug safety evaluation – Drug induced liver injury prediction system (DILIps). PLoS Comput Biol 7, e1002310. doi:10.1371/journal.pcbi.1002310
Luderer, U., Eskenazi, B., Hauser, R. et al. (2019). Proposed key characteristics of female reproductive toxicants as an approach for organizing and evaluating mechanistic data in hazard assessment. Environ Health Perspect 127, 075001. doi:10.1289/EHP4971
Marshall, I. J. and Wallace, B. C. (2019). Toward systematic review automation: A practical guide to using machine learning tools in research synthesis. Syst Rev 8, 163. doi:10.1186/s13643-019-1074-9
Martin, P., Bladier, C., Meek, B. et al. (2018). Weight of evidence for hazard identification: A critical review of the literature. Environ Health Perspect 126, 076001. doi:10.1289/EHP3067
Meek, M. E., Palermo, C. M., Bachman, A. N. et al. (2014). Mode of action human relevance (MOA/HR) framework – Evolution of the Bradford Hill considerations and comparative analysis of weight of evidence. J Appl Toxicol 34, 595-606. doi:10.1002/jat.2984
Mellor, C. L., Steinmetz, F. P. and Cronin, M. T. (2016). The identification of nuclear receptors associated with hepatic steatosis to develop and extend adverse outcome pathways. Crit Rev Toxicol 46, 138-152. doi:10.3109/10408444.2015.1089471
Moberg, J., Oxman, A. D., Rosenbaum, S. et al. (2018). The GRADE evidence to decision (EtD) framework for health system and public health decisions. Health Res Policy Syst 16, 45. doi:10.1186/s12961-018-0320-2
Morgan, R. L., Thayer, K. A., Bero, L. et al. (2016). GRADE: Assessing the quality of evidence in environmental and occupational health. Environ Int 92-93, 611-616. doi:10.1016/j.envint.2016.01.004
NRC – National Research Council (2007). Toxicity Testing in the 21st Century: A Vision and Strategy. Washington, DC, USA: National Academies Press. doi:10.17226/11970
NRC (2014). Review of EPA’s Integrated Risk Information System (IRIS) Process. Washington, DC, USA: National Academies Press. doi:10.17226/18764
NRC (2017). Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals. Washington, DC, USA: National Academies Press. doi:10.17226/24758
NTP – National Toxicology Program (2016). NTP Monograph on Immunotoxicity Associated with Exposure to Perfluorooctanoic acid (PFOA) and Perfluorooctane sulfonate (PFOS). U.S. Department of Health and Human Services. https://ntp.niehs.nih.gov/ntp/ohat/pfoa_pfos/pfoa_pfosmonograph_508.pdf (accessed 14.02.2022).
NTP (2019). NTP Monograph on the Systematic Review of Occupational Exposure to Cancer Chemotherapy and Adverse Health Outcomes. 2019. U.S. Department of Health and Human Services. https://ntp.niehs.nih.gov/ntp/ohat/oeccaaho/mgraph/occchemo_final20190300_508.pdf (accessed 20.12.2019).
O’Connor, A. M., Glasziou, P., Taylor, M. et al. (2020). A focus on cross-purpose tools, automated recognition of study design in multiple disciplines, and evaluation of automation tools: A summary of significant discussions at the fourth meeting of the international collaboration for automation of systematic reviews (ICASR). Syst Rev 9, 100. doi:10.1186/s13643-020-01351-4
OECD (2016). Guidance Document for the Use of Adverse Outcome Pathways in Developing Integrated Approaches to Testing and Assessment (IATA). OECD Series on Testing and Assessment, No. 260. OECD Publishing, Paris. doi:10.1787/44bb06c1-en
OECD (2018). Users’ Handbook Supplement to the Guidance Document for Developing and Assessing Adverse Outcome Pathways. OECD Series on Adverse Outcome Pathways, No. 1. OECD Publishing, Paris. doi:10.1787/5jlv1m9d1g32-en
Parmelli, E., Amato, L., Oxman, A. D. et al. (2017). Grade evidence to decision (EtD) framework for coverage decisions. Int J Technol Assess Health Care 33, 176-182. doi:10.1017/S0266462317000447
Perkins, E. J., Ashauer, R., Burgoon, L. et al. (2019). Building and applying quantitative adverse outcome pathway models for chemical hazard and risk assessment. Environ Toxicol Chem 38, 1850-1865. doi:10.1002/etc.4505
Pistollato, F., Madia, F., Corvi, R. et al. (2021). Current EU regulatory requirements for the assessment of chemicals and cosmetic products: Challenges and opportunities for introducing new approach methodologies. Arch Toxicol 95, 1867-1897. doi:10.1007/s00204-021-03034-y
Rehfuess, E. A., Booth, A., Brereton, L. et al. (2018). Towards a taxonomy of logic models in health technology assessments; A priori, staged and iterative approaches. Res Synth Methods 9, 13-24. doi:10.1002/jrsm.1254
Rhomberg, L. R., Goodman, J. E., Bailey, L. A. et al. (2013). A survey of frameworks for best practices in weight-of-evidence analyses. Crit Rev Toxicol 43, 753-784. doi:10.3109/10408444.2013.832727
Rooney, A. A., Boyles, A. L., Wolfe, M. S. et al. (2014). Systematic review and evidence integration for literature-based environmental health science assessments. Environ Health Perspect 122, 711-718. doi:10.1289/ehp.1307972
Rooney, A. A., Boyles, A. L. and Walker, V.R. (2015). Systematic review, an illustration of increased transparency in a framework for evaluating immunotoxicity associated with PFOA and PFOS exposure. In J. DeWitt (ed.), Toxicological Effects of Perfluoroalkyl and Polyfluoroalkyl Substances. Molecular and Integrative Toxicology (419-449). Cham, Switzerland: Humana Press. doi:10.1007/978-3-319-15518-0_16
Samuel, G. O., Hoffmann, S., Wright, R. A. et al. (2016). Guidance on assessing the methodological and reporting quality of toxicologically relevant studies: A scoping review. Environ Int 92-93, 630-646. doi:10.1016/j.envint.2016.03.010
Schneider, K., Schwarz, M., Burkholder, I. et al. (2009). “ToxRTool”, a new tool to assess the reliability of toxicological data. Toxicol Lett 189, 138-144. doi:10.1016/j.toxlet.2009.05.013
Schünemann, H., Hill, S., Guyatt, G. et al. (2011). The GRADE approach and Bradford Hill’s criteria for causation. J Epidemiol Community Health 65, 392-395. doi:10.1136/jech.2010.119933
Smith, M., T., Guyton, K. Z., Gibbons, C. F. et al. (2016). Key characteristics of carcinogens as a basis for organizing data on mechanisms of carcinogenesis. Environ Health Perspect 124, 713-721. doi:10.1289/ehp.1509912
Tabernilla, A., Dos Santos Rodrigues, B., Pieters, A. et al. (2021). In vitro liver toxicity testing of chemicals: A pragmatic approach. Int J Mol Sci 22, 5038. doi:10.3390/ijms22095038
Tollefsen, K. E., Scholz, S., Cronin, M. T. et al. (2014). Applying adverse outcome pathways (AOPs) to support integrated approaches to testing and assessment (IATA). Regul Toxicol Pharmacol 70, 629-640. doi:10.1016/j.yrtph.2014.09.009
Vesterinen, H. M., Johnson, P. I., Atchley, D. S. et al. (2015). Fetal growth and maternal glomerular filtration rate: A systematic review. J Matern Fetal Neonatal Med 28, 2176-2181. doi:10.3109/14767058.2014.980809
Villeneuve, D. L., Crump, D., Garcia-Reyero, N. et al. (2014). Adverse outcome pathway (AOP) development I: Strategies and principles. Toxicol Sci 142, 312-320. doi:10.1093/toxsci/kfu199
Vinken, M., Landesmann, B., Goumenou, M. et al. (2013). Development of an adverse outcome pathway from drug-mediated bile salt export pump inhibition to cholestatic liver injury. Toxicol Sci 136, 97-106. doi:10.1093/toxsci/kft177
Viswanathan, M., Ansari, M., Berkman, N. D. et al. (2012). Assessing the risk of bias of individual studies when comparing medical interventions. In Methods Guide for Effectiveness and Comparative Effectiveness Reviews (Publication No. 12-EHC047-EF). Rockville, MD, USA: Agency for Healthcare Research and Quality (AHRQ) . https://effectivehealthcare.ahrq.gov/products/methods-guidance-bias-individual-studies/methods (accessed 14.02.2022).
Whaley, P., Piggott, T., Morgan, R. L. et al. (2022). Biological plausibility in environmental health systematic reviews: A GRADE concept paper. J Clin Epidem 146, 32-46. doi:10.1016/j.jclinepi.2022.02.011
Wittwehr, C., Amorim, M. J., Clerbaux, L. A. et al. (2021). Understanding COVID-19 through adverse outcome pathways – 2nd CIAO AOP design workshop. ALTEX 38, 351-357. doi:10.14573/altex.2102221
Wolffe, T. A. M., Whaley, P., Halsall, C. et al. (2019). Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environ Int 130, 104871. doi:10.1016/j.envint.2019.05.065
Woodruff, T. J. and Sutton P. (2014). The Navigation guide systematic review methodology: A rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect 122, 1007-1014. doi:10.1289/ehp.1307175