Research Paper|Articles in Press

A co-designed web-based Delirium Toolkit for carers: An eDelphi evaluation of usability and quality

Published:November 24, 2022DOI:



      Innovative co-designed strategies are required to support the integration of carers to partner with healthcare staff in delirium management.


      To validate a co-designed web-based Delirium Toolkit, a resource developed to engage and support carers of hospitalised older adults at risk of delirium.


      Expert group consensus on the Delirium Toolkit's healthcare information and resources for carers of hospitalised older adults at risk of delirium was achieved through eDelphi technique. Eight participants, seven with experience of caring with someone aged 65 years or older diagnosed with delirium and one carer with an interest in understanding delirium were recruited. A total of 23 items were rated using a five-point Likert scale over two rounds to achieve 75% consensus on the layout of the web-based Delirium Toolkit and the usability and quality of content.


      Data were collected and consensus reached after two rounds in May and June of 2022. Out of 23 items, consensus was reached on 19 items in Round one (rejecting main navigation, headings, delirium screening tool instructions, and ease of understanding of carer support resources). Consensus was achieved in all items in Round two after suggested improvements to the Delirium Toolkit were actioned.


      The web-based Delirium Toolkit has been developed with carers to support their integration as partners in the care of hospitalised older adults at risk of delirium.


      The study findings support the readiness for testing the Delirium Toolkit in clinical environments.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Collegian
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Anderson, M., & Perrin, A. (2017). Tech Adoption Climbs Among Older Adults. Internet, Science & Tech, Pew Research Centre. CID: 20.500.12592/rfkth9. Available from: [Accessed 25 August 2022].

        • Australian Bureau of Statistics
        Household use of information technology, Australia, 2014-15.
        Australia Government, Canberra, Australia2018 (Available from:) ([Accessed 25 August 2022])
        • Australian Commission on Safety and Quality in Health Care
        Delirium clinical care standard.
        ACSQHC, Sydney2021
        • Ayton D.
        • Moran C.
        • Berkovic D.
        • Bateman C.
        • Anderson K.
        • Blair A.
        • et al.
        The Volunteer Dementia and Delirium Care (VDDC)© program: an exploration of perceived barriers and enablers to implementation in an acute and subacute metropolitan hospital.
        Australasian Journal on Ageing. 2020; 39: e334-e343
        • Brefka S.
        • Eschweiler G.W.
        • Dallmeier D.
        • Denkinger M.
        • Leinert C.
        Comparison of delirium detection tools in acute care.
        Zeitschrift für Gerontologie und Geriatrie. 2022; 55: 105-115
        • Diamond I.R.
        • Grant R.C.
        • Feldman B.M.
        • Pencharz P.B.
        • Ling S.C.
        • Moore A.M.
        • et al.
        Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies.
        Journal of Clinical Epidemiology. 2014; 67: 401-409
      2. Digital Transformation Agency. (2020). Digital Transformation Strategy. Available from: [Accessed 25 August 2022].

      3. Donaldson, J., Baynes, C., & Donaldson, A. (2020). My Aged Care Explained. Available from: [Accessed 25 August 2022].

        • Evans C.
        The use of consensus methods and expert panels in pharmacoeconomic studies.
        Pharmacoeconomics. 1997; 12: 121-129
        • Gibb K.
        • Seeley A.
        • Quinn T.
        • Siddiqi N.
        • Shenkin S.
        • Rockwood K.
        • et al.
        The consistent burden in published estimates of delirium occurrence in medical inpatients over four decades: a systematic review and meta-analysis study.
        Age and Ageing. 2020; 49: 352-360
        • Gill F.J.
        • Leslie G.D.
        • Grech C.
        • Latour J.M.
        Using a web-based survey tool to undertake a Delphi study: application for nurse education research.
        Nurse Education Today. 2013; 33: 1322-1328
        • Hargittai E.
        • Piper A.M.
        • Morris M.R.
        From internet access to internet skills: digital inequality among older adults.
        Universal Access in the Information Society. 2019; 18: 881-890
        • Hill R.
        • Betts L.R.
        • Gardner S.E.
        Older adults’ experiences and perceptions of digital technology: (Dis) empowerment, wellbeing, and inclusion.
        Computers in Human Behavior. 2015; 48: 415-423
        • Iragorri N.
        • Spackman E.
        Assessing the value of screening tools: reviewing the challenges and opportunities of cost-effectiveness analysis.
        Public Health Reviews. 2018; 39: 1-27
        • Kincl T.
        • Štrach P.
        Measuring website quality: asymmetric effect of user satisfaction.
        Behaviour & Information Technology. 2012; 31: 647-657
        • Klompstra L.
        • Liljeroos M.
        • Lundgren J.
        • Ingadottir B.
        A clinical tool (CUE-tool) for health care professionals to assess the usability and quality of the content of medical information websites: electronic Delphi study.
        Journal of Medical Internet Research. 2021; 23: e22668
        • Larkin M.
        • Henwood M.
        • Milne A.
        Older carers and carers of people with dementia: improving and developing effective support.
        Social Policy and Society. 2022; 21: 242-256
        • Marston H.R.
        • Musselwhite C.B.
        Improving older people's lives through digital technology and practices.
        Gerontology and Geriatric Medicine. 2021; 7: 1-4
        • McKenzie J.
        • Joy A.
        Family intervention improves outcomes for patients with delirium: systematic review and meta-analysis.
        Australasian Journal on Ageing. 2020; 39: 21-30
        • Okoli C.
        • Pawlowski S.D.
        The Delphi method as a research tool: an example, design considerations and applications.
        Information & Management. 2004; 42: 15-29
        • Parsons Leigh J.
        • Krewulak K.D.
        • Zepeda N.
        • Farrier C.E.
        • Spence K.L.
        • Davidson J.E.
        • et al.
        Patients, family members and providers perceive family-administered delirium detection tools in the adult ICU as feasible and of value to patient care and family member coping: a qualitative focus group study.
        Canadian Journal of Anesthesia. 2021; 68: 358-366
        • Petrovic M.
        • Gaggioli A.
        Digital mental health tools for caregivers of older adults—a scoping review.
        Frontiers in Public Health. 2020; 28: 128
        • Piper A.M.
        • Garcia R.C.
        • Brewer R.N.
        Understanding the challenges and opportunities of smart mobile devices among the oldest old.
        International Journal of Mobile Human Computer Interaction. 2016; 8: 83-98
        • Racine A.M.
        • D'Aquila M.
        • Schmitt E.M.
        • Gallagher J.
        • Marcantonio E.R.
        • Jones R.N.
        • et al.
        Delirium burden in patients and family caregivers: development and testing of new instruments.
        The Gerontologist. 2019; 59: e393-e402
        • Rafe V.
        • Monfaredzadeh M.
        A qualitative framework to assess hospital/medical websites.
        Journal of Medical Systems. 2012; 36: 2927-2939
        • Redley B.
        • Richardson B.
        • Peel C.
        • Ockerby C.
        • Rawson H.
        • Tomlinson E.
        • et al.
        Co-development of “BRAIN-TRK”: qualitative examination of acceptability, usability and feasibility of an app to support nurses' care for patients with behavioural and psychological symptoms of neurocognitive disorders in hospital.
        Journal of Clinical Nursing. 2019; 28: 2868-2879
        • Shenkin S.D.
        • Fox C.
        • Godfrey M.
        • Siddiqi N.
        • Goodacre S.
        • Young J.
        • et al.
        Delirium detection in older acute medical inpatients: a multicentre prospective comparative diagnostic test accuracy study of the 4AT and the confusion assessment method.
        BMC Medicine. 2019; 17: 1-14
        • Shulman R.W.
        • Kalra S.
        • Jiang J.Z.
        Validation of the Sour Seven Questionnaire for screening delirium in hospitalized seniors by informal caregivers and untrained nurses.
        BMC Geriatrics. 2016; 16: 1-8
        • Teodorczuk A.
        • Billett S.
        Mediating workplace situational pressures: the role of artefacts in promoting effective interprofessional work and learning.
        Focus on Health Professional Education. 2017; 18: 80-91
        • Wilson J.
        • Heinsch M.
        • Betts D.
        • Booth D.
        • Kay-Lambkin F.
        Barriers and facilitators to the use of e-health by older adults: a scoping review.
        BMC Public Health [Electronic Resource]. 2021; 21: 1-12
        • Wong P.
        • Redley B.
        • Digby R.
        • Correya A.
        • Bucknall T.
        Families’ perspectives of participation in patient care in an adult intensive care unit: a qualitative study.
        Australian Critical Care. 2020; 33: 317-325
        • Zhao Y.C.
        • Zhao M.
        • Song S.
        Online health information seeking behaviors among older adults: systematic scoping review.
        Journal of Medical Internet Research. 2022; 24: e34790