Ethical and feasible stakeholder engagement in guideline development



      Stakeholder engagement impacts on the relevance and usability of guidelines. Consequently, guideline developers are advised to engage with a diverse group of stakeholders. One of the most critical and challenging phases of guideline development is determining the guideline scope, and there is currently scant guidance for how stakeholders can be engaged in feasible and ethical ways.


      This article aims to provide practical guidance for how diverse stakeholders can be engaged to determine the scope of a guideline.


      Supported by previous frameworks and by drawing on experiences from a research project aiming to develop a clinical practice guideline on non-pharmacological approaches for agitation in the intensive care unit, this paper describes a 7-step process for stakeholder engagement. Guideline developers need to consider the aim of their project, identify relevant stakeholders, decide on the level of engagement, plan how to engage in feasible and ethical ways, consider how to increase the trustworthiness of a project, and finally consider how to evaluate the project.


      Consultation of diverse stakeholder groups is feasible, but it is essential to plan the activities and be aware of the steps to take to ensure an effective and ethical process.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Collegian
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Adams A.M.N.
        • Chamberlain D.
        • Grønkjær M.
        • Thorup C.B.
        • Conroy T.
        Caring for patients displaying agitated behaviours in the intensive care unit–a mixed-methods systematic review.
        Australian Critical Care. 2022; 35: 454-465
      1. Alkhaffaf, B., Blazeby, J. M., Metryka, A., Glenny, A.-M., Adeyeye, A., Costa, P. M., et al. (2021). Methods for conducting international Delphi surveys to optimise global participation in Core Outcome Set development: a case study in gastric cancer informed by a comprehensive literature review.

        • Andrews D.
        • Fong G.
        • Hackam D.
        • Li L.
        • Lynam M.
        • Mathews M.
        • Strauss S.
        Guide to knowledge translation planning at CIHR: Integrated and end–of–grant approaches.
        Canadian Institute of Health Research, Ottawa, Canada2012
        • Armstrong M.J.
        • Bloom J.A.
        Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies.
        Research involvement and engagement. 2017; 3: 1-11
        • Armstrong M.J.
        • Mullins C.D.
        • Gronseth G.S.
        • Gagliardi A.R.
        Impact of patient involvement on clinical practice guideline development: a parallel group study.
        Implementation Science. 2018; 13: 55
        • Arnstein S.R.
        A ladder of citizen participation.
        Journal of the American Planning Association. 2019; 85: 24-34
      2. Australian Clinical Trials Alliance. (2021). Ethics review and involvement. Retrieved from (access date 10th April 2022).

        • Banner D.
        • Bains M.
        • Carroll S.
        • Kandola D.K.
        • Rolfe D.E.
        • Wong C.
        • et al.
        Patient and public engagement in integrated knowledge translation research: are we there yet?.
        Research involvement and engagement. 2019; 5: 1-14
        • Banner D.J.
        • Graham I.D.
        • Bains M.
        • Carroll S.
        • Aaron S.
        • Healey J.
        • et al.
        Ready to go: Using an integrated knowledge translation approach to support the development of a funding application to explore patient engagement in Canadian circulatory and respiratory diseases research.
        how we work together. 2020; : 19
        • Berk L.
        • Jorm A.F.
        • Kelly C.M.
        • Dodd S.
        • Berk M.
        Development of guidelines for caregivers of people with bipolar disorder: a Delphi expert consensus study.
        Bipolar disorders. 2011; 13: 556-570
      3. Boivin, A., & Abelson, J. Evaluation Toolkit. Retrieved from |all|1. (access date 10th April 2022)

        • Braithwaite J.
        • Glasziou P.
        • Westbrook J.
        The three numbers you need to know about healthcare: the 60-30-10 challenge.
        BMC Medicine. 2020; 18: 1-8
        • Cahill N.E.
        • Suurdt J.
        • Ouellette-Kuntz H.
        • Heyland D.K.
        Understanding adherence to guidelines in the intensive care unit: development of a comprehensive framework.
        Journal of parenteral and Enteral Nutrition. 2010; 34: 616-624
        • Cluzeau F.
        • Wedzicha J.A.
        • Kelson M.
        • Corn J.
        • Kunz R.
        • Walsh J.
        • et al.
        Stakeholder involvement: how to do it right: article 9 in integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report.
        Proceedings of the American Thoracic Society. 2012; 9: 269-273
      4. Cowl, J., Armstrong, M. J., Schaefer, C., & Fielding, J. (2021). GIN public Toolkit, Chapter 1 Consultation. Retrieved from (access date June 2021).

        • Creswell J.W.
        Research design: qualitative, quantitative, and mixed methods approaches.
        4th ed. SAGE Publications, California, United States of America2014
        • Creswell J.W.
        • Clark V.L.P.
        Designing and conducting mixed methods research.
        Sage publications, 2017
        • Del Campo P.D.
        • Gracia J.
        • Blasco J.
        • Andradas E.
        A strategy for patient involvement in clinical practice guidelines: methodological approaches.
        BMJ quality & safety. 2011; 20: 779-784
        • Doria N.
        • Condran B.
        • Boulos L.
        • Maillet D.G.C.
        • Dowling L.
        • Levy A.
        Sharpening the focus: differentiating between focus groups for patient engagement vs. qualitative research.
        Research involvement and engagement. 2018; 4: 1-8
        • Dusdal J.
        • Powell J.J.
        Benefits, motivations, and challenges of international collaborative research: A sociology of science case study.
        Science and Public Policy. 2021; 48: 235-245
        • French S.D.
        • Nielsen M.
        • Hall L.
        • Nicolson P.J.
        • van Tulder M.
        • Bennell K.L.
        • et al.
        Essential key messages about diagnosis, imaging, and self-care for people with low back pain: a modified Delphi study of consumer and expert opinions.
        Pain. 2019; 160: 2787-2797
        • Gawlik K.
        Focus group interviews.
        In Qualitative methodologies in organization studies. Springer, 2018: 97-126
        • Girlanda F.
        • Fiedler I.
        • Becker T.
        • Barbui C.
        • Koesters M.
        The evidence–practice gap in specialist mental healthcare: systematic review and meta-analysis of guideline implementation studies.
        The British Journal of Psychiatry. 2017; 210: 24-30
        • Grant S.
        • Hazlewood G.S.
        • Peay H.L.
        • Lucas A.
        • Coulter I.
        • Fink A.
        • et al.
        Practical considerations for using online methods to engage patients in guideline development.
        The Patient-Patient-Centered Outcomes Research. 2018; 11: 155-166
        • Gutman T.
        • Tong A.
        • Howell M.
        • Dansie K.
        • Hawley C.M.
        • Craig J.C.
        • Murphy L.
        Principles and strategies for involving patients in research in chronic kidney disease: report from national workshops.
        Nephrology Dialysis Transplantation. 2018; 35: 1585-1595
      5. Hardavella, G., Bjerg, A., Saad, N., Jacinto, T., & Powell, P. (2015). Doing science: how to optimise patient and public involvement in your research.

        • Hayes H.
        • Buckland S.
        • Tarpey M.
        Briefing notes for researchers: public involvement in NHS, public health and social care research.
        INVOLVE: Eastleigh. 2012; (Retrieved from)
      6. Health, N. I. f., & Excellence, C. (2015). Developing NICE guidelines: the manual: National Institute for Health and Care Excellence (NICE).

        • Heneghan C.
        • Perera R.
        • Mant D.
        • Glasziou P.
        Hypertension guideline recommendations in general practice: awareness, agreement, adoption, and adherence.
        British Journal of General Practice. 2007; 57: 948-952
        • Hoekstra F.
        • Trigo F.
        • Sibley K.M.
        • Graham I.D.
        • Kennefick M.
        • Mrklas K.J.
        • et al.
        Systematic overviews of partnership principles and strategies identified from health research about spinal cord injury and related health conditions: a scoping review.
        The Journal of Spinal Cord Medicine. 2022; : 1-18
        • Honey-Rosés J.
        • Canessa M.
        • Daitch S.
        • Gomes B.
        • García J.M.-B.
        • Xavier A.
        • Zapata O.
        Comparing Structured and Unstructured Facilitation Approaches in Consultation Workshops: A Field Experiment.
        Group Decision and Negotiation. 2020; 29: 949-967
      7. IAP2. iap2 International Association for Public Participation. Retrieved from (access date 24th Nov 2021).

      8. IAP2. (2017). Code of ethics. Retrieved from (8th July 2021).

      9. IAP2 Spectrum of Public Participation. (2018). Retrieved from

        • Ingoe H.M.
        • Eardley W.
        • Rangan A.
        • Hewitt C.
        • McDaid C.
        An international multi-stakeholder delphi consensus exercise to develop a core outcomes set (COS) for surgical fixation of rib fractures.
        Injury. 2020; 51: 224-229
      10. Involve. (2009). Patient and public involvement in research and research ethics committee review. Retrieved from (7th May 2021)

      11. INVOLVE. (2015). Public involvement in research: Values and principles framework. Eastleigh: INVOLVE Retrieved from (21 Nov 2021)

        • Kaushik V.
        • Walsh C.A.
        Pragmatism as a research paradigm and its implications for social work research.
        Social Sciences. 2019; 8: 255
        • Kothari A.
        • McCutcheon C.
        • Graham I.D.
        Defining integrated knowledge translation and moving forward: a response to recent commentaries.
        International journal of health policy and management. 2017; 6: 299
        • Kothari A.
        • Wathen C.N.
        A critical second look at integrated knowledge translation.
        Health Policy. 2013; 109: 187-191
        • Kothari A.
        • Wathen C.N.
        Integrated knowledge translation: digging deeper, moving forward.
        J Epidemiol Community Health. 2017; 71: 619-623
        • Landry R.
        • Amara N.
        • Pablos-Mendes A.
        • Shademani R.
        • Gold I.
        The knowledge-value chain: a conceptual framework for knowledge translation in health.
        Bulletin of the World Health Organization. 2006; 84: 597-602
        • Liamputtong P.
        Qualitative research methods.
        3rd ed. Oxford University Press, Melbourne, Victoria, Australia2009
        • McMillan S.S.
        • King M.
        • Tully M.P.
        How to use the nominal group and Delphi techniques.
        International Journal of Clinical Pharmacy. 2016; 38: 655-662
      12. National Health and Medical Research Council, N. (2020). Guidelines for Guidelines. Retrieved from (Yaccess date 10th April 2022).

        • National Health Medical Research Council
        Statement on consumer and community involvement in health and medical research.
        Australian Government Canberra, 2016
        • Nguyen T.
        • Graham I.D.
        • Mrklas K.J.
        • Bowen S.
        • Cargo M.
        • Estabrooks C.A.
        • et al.
        How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field.
        Health research policy and systems. 2020; 18: 1-20
        • Northway R.
        • Hurley K.
        • O’Connor C.
        • Thomas H.
        • Howarth J.
        • Langley E.
        • Bale S.
        Deciding what to research: an overview of a participatory workshop.
        British Journal of Learning Disabilities. 2014; 42: 323-327
        • Nowell L.
        Pragmatism and integrated knowledge translation: exploring the compatabilities and tensions.
        Nursing open. 2015; 2: 141-148
        • Ocloo J.
        • Matthews R.
        From tokenism to empowerment: progressing patient and public involvement in healthcare improvement.
        BMJ quality & safety. 2016; 25: 626-632
        • Pandya-Wood R.
        • Barron D.S.
        • Elliott J.
        A framework for public involvement at the design stage of NHS health and social care research: time to develop ethically conscious standards.
        Research involvement and engagement. 2017; 3: 1-21
        • Perez X.
        • Wisnivesky J.P.
        • Lurslurchachai L.
        • Kleinman L.C.
        • Kronish I.M.
        Barriers to adherence to COPD guidelines among primary care providers.
        Respiratory medicine. 2012; 106: 374-381
      13. QSR International, P. L. (2021). NVivo (version 12 Pro). Retrieved from (access date November 2021).

        • Rapport F.
        • Doel M.
        • Hutchings H.
        • Jones A.
        • Culley L.
        • Wright S.
        Consultation workshops with patients and professionals: developing a template of patient-centred professionalism in community nursing.
        Journal of Research in Nursing. 2014; 19: 146-160
        • Reinecke H.
        • Unrath M.
        • Freisinger E.
        • Bunzemeier H.
        • Meyborg M.
        • Lüders F.
        • et al.
        Peripheral arterial disease and critical limb ischaemia: still poor outcomes and lack of guideline adherence.
        European heart journal. 2015; 36: 932-938
        • Romsland G.I.
        • Milosavljevic K.L.
        • Andreassen T.A.
        Facilitating non-tokenistic user involvement in research.
        Research involvement and engagement. 2019; 5: 1-12
        • Russell J.
        • Fudge N.
        • Greenhalgh T.
        The impact of public involvement in health research: what are we measuring? Why are we measuring it? Should we stop measuring it?.
        Research involvement and engagement. 2020; 6: 1-8
        • Salerno J.
        • Coleman K.J.
        • Jones F.
        • Peters E.S.
        The ethical challenges and opportunities of implementing engagement strategies in health research.
        Annals of Epidemiology. 2021; 59: 37-43
        • Serrano-Aguilar P.
        • del Mar Trujillo-Martin M.
        • de la Rosa A.P.
        • Cuellar-Pompa L.
        • Saavedra-Medina H.
        • Linertova R.
        • et al.
        Patient participation in a clinical guideline development for systemic lupus erythematosus.
        Patient education and counseling. 2015; 98: 1156-1163
        • Spitaels D.
        • Vankrunkelsven P.
        • Desfosses J.
        • Luyten F.
        • Verschueren S.
        • Van Assche D.
        • et al.
        Barriers for guideline adherence in knee osteoarthritis care: a qualitative study from the patients' perspective.
        Journal of Evaluation in Clinical Practice. 2017; 23: 165-172
        • Steinberg E.
        • Greenfield S.
        • Wolman D.M.
        • Mancher M.
        • Graham R.
        Clinical practice guidelines we can trust.
        National Academies Press, 2011 (access date 10th April 2022)
        • Straus S.
        • Tetroe J.
        • Graham I.D.
        Knowledge translation in health care: moving from evidence to practice.
        John Wiley & Sons, 2013
      14. The University of British Columbia. IKT guiding principles. Retrieved from (10th April 2022).

        • Tong A.
        • Lopez-Vargas P.
        • Howell M.
        • Phoon R.
        • Johnson D.
        • Campbell D.
        • et al.
        Consumer involvement in topic and outcome selection in the development of clinical practice guidelines.
        Health Expectations. 2012; 15: 410-423
        • Trevelyan E.G.
        • Robinson N.
        Delphi methodology in health research: how to do it?.
        European Journal of Integrative Medicine. 2015; 7: 423-428
      15. World Health Organization. (2014). WHO handbook for guideline development. Retrieved from

        • Wright D.
        • Foster C.
        • Amir Z.
        • Elliott J.
        • Wilson R.
        Critical appraisal guidelines for assessing the quality and impact of user involvement in research.
        Health Expectations. 2010; 13: 359-368
        • Wyatt K.D.
        • Stuart L.M.
        • Brito J.P.
        • Leon B.C.
        • Domecq J.P.
        • Prutsky G.J.
        • et al.
        Out of context: clinical practice guidelines and patients with multiple chronic conditions: a systematic review.
        Medical care. 2014; : S92-S100