Building Healthcare Professionals’ Capability for Responsible AI

Abstract

This Expert Talk explores how healthcare professionals can develop the judgement and practical capability to use generative and agentic AI responsibly. It considers workflow selection, context design, prototyping, evaluation, governance and human oversight, while distinguishing educational experimentation from clinical deployment. The discussion focuses on institutional conditions needed to translate enthusiasm into sustained, safe practice.

1. How should healthcare organisations define “AI capability” for healthcare professionals, beyond basic familiarity with AI tools and technologies?

AI capability is the ability to make sound professional decisions about when, why and how AI should be used within a workflow. It includes recognising suitable tasks, protecting information, providing relevant context, checking outputs and knowing when to stop or seek help. Capability also involves understanding how an individual’s use affects colleagues, patients and organisational accountability. I would assess it through observable practice: can a professional explain the intended benefit, identify foreseeable failures and demonstrate a workable review process? Familiarity with an interface is a starting point. Responsible capability requires judgement that transfers across tools and changing circumstances.

2. What foundational knowledge and practical skills should healthcare professionals develop to use generative and agentic AI effectively and responsibly?

Professionals need to understand that generative AI can produce convincing material containing errors, omissions or invented references. They should understand privacy, bias, uncertainty and the limits of performance claims. Practical skills include defining a task, selecting trustworthy sources, providing clear instructions and evaluating results against explicit criteria. Agentic AI, which can use tools to perform sequences of actions, also requires an understanding of permissions, approval points and interruption procedures. Professionals should practise identifying unsafe requests and escalating unexpected behaviour. The appropriate level of technical depth varies by role, but everyone needs to recognise the boundaries of their competence and authority.

3. How can organisations identify healthcare workflows where AI can add meaningful value while ensuring that its use remains appropriate to the clinical context?

Start with a problem that staff can describe and measure, such as repeated preparation of educational material or avoidable administrative rework. Map the existing process, including inputs, decisions, handovers and exceptions. Then ask whether simplification, a template or conventional software would solve it more reliably. Where AI has a plausible role, define a bounded task and assess the consequences of failure, data sensitivity and ease of verification. Clinical relevance depends on the intended use and downstream effects, including how an administrative output might influence care. Selection should involve frontline staff and governance colleagues, with patient input where appropriate.

4. What role does context design play in improving the quality, relevance, and reliability of AI outputs in healthcare settings?

Context design means specifying the purpose, audience, relevant information, permitted sources and limits of a task. For example, an educational draft should reflect agreed learning objectives and approved references, with instructions to identify missing information rather than invent it. Context also includes document versions, terminology and the structure needed for professional review. Good context can make outputs more relevant and easier to check, but it cannot guarantee accuracy. Teams must test whether the system follows instructions and preserves important qualifications. Reliable use depends on maintaining the information environment and verifying outputs, alongside the wording of an individual prompt.

5. How can healthcare professionals move from experimenting with AI tools to developing practical prototypes that address genuine workflow challenges?

A prototype should begin with a specific workflow problem, a named owner and a clear description of what success would look like. Professionals can map the current process, choose one bounded activity and test an alternative using synthetic information or appropriately approved data. The prototype should include the review step, escalation route and fallback process from the outset. Testing should capture failures and additional work, rather than only attractive outputs. Collaboration with technical and governance colleagues helps identify constraints early. Progression to operational use should depend on documented evidence and appropriate approval; a successful demonstration alone does not establish readiness.

6. What evaluation approaches can organisations use to assess the usefulness, accuracy, safety, and usability of AI-supported workflows before wider adoption?

Evaluation should compare the complete AI-supported workflow with the existing process. Measure preparation, generation, checking, correction and exception handling, rather than generation speed alone. Use representative cases alongside difficult, incomplete and conflicting inputs. Domain experts should assess factual accuracy, consequential omissions and suitability for the intended audience. Staff observation can reveal whether outputs are understandable and whether review is realistic under working conditions. Acceptance criteria and stopping rules should be agreed in advance, with assurance proportionate to potential harm. Evaluation must also examine the combined performance of people and system, followed by monitoring during any approved pilot.

7. How should healthcare organisations distinguish educational experimentation with AI from applications intended for clinical practice?

The distinction should be based on intended use, actual use and consequences. An exercise using fictional cases to teach critical appraisal differs from an application that influences a patient’s diagnosis, treatment or clinical record. Educational environments need clear boundaries, approved data practices and visible identification of unvalidated outputs. Material should not migrate into care simply because it looks polished or has performed well in a workshop. Clinical applications require assurance appropriate to their purpose, institutional approval and consideration of applicable regulatory requirements. Organisations should define the transition explicitly, including who can authorise it and what additional evidence is required.

8. What forms of human oversight are important when AI is incorporated into clinical, administrative, or educational workflows?

Oversight must identify who checks the output, what they check and whether they have the expertise, information and time to do so. Reviewers need authority to reject a result, request clarification and escalate concerns. Clinical outputs require scrutiny appropriate to their potential consequences; administrative and educational outputs also need checks for accuracy, confidentiality and inappropriate downstream use. For systems that can take actions, organisations should define permissions and approval points before execution. Monitoring, incident review and a workable fallback process remain necessary after deployment. A nominal human approval step offers little protection if staff cannot detect or interrupt errors.

9. How can medical educators and clinical leaders incorporate responsible AI competencies into continuing professional development and health professions education?

Responsible AI should be taught through realistic professional tasks, with learning outcomes matched to the learner’s role. Participants might evaluate an inaccurate summary, identify a privacy problem or redesign a workflow with clear review points. Assessment should examine their reasoning, source checking and decisions about escalation, alongside the quality of the final product. Educators can introduce foundational concepts early and revisit them in increasingly complex clinical or organisational settings. Continuing professional development should include protected practice, feedback and opportunities to discuss failures. Faculty also need support, so that teaching remains grounded in professional responsibilities as technologies and institutional policies change.

10. What institutional policies, governance structures, and support systems are needed to help healthcare professionals adopt AI in a safe and sustainable way?

Organisations need clear rules on approved environments, permitted data, access, retention and acceptable uses. Each AI-supported workflow should have an accountable owner, documented limits and an approval route proportionate to risk. Governance should connect clinical leadership, education, information security, data protection, technical services and relevant patient perspectives. Practical support matters: staff need accessible advice, training, testing facilities and a straightforward incident-reporting route. An inventory should record deployed workflows, evaluation findings and review responsibilities. Procurement and change management should address updates, supplier dependencies and withdrawal arrangements. Governance becomes useful when it enables clear decisions and continuing oversight throughout a workflow’s life.

11. How can organisations create an environment where healthcare professionals can experiment with AI while maintaining appropriate standards of privacy, accountability, and professional responsibility?

Provide a supported space with approved tools, clear task boundaries and synthetic or appropriately authorised information. Staff should know which activities they may explore independently and which require additional review. A simple project record can capture the purpose, data used, responsible person, findings and unresolved risks. Protected learning time and access to expert advice help staff raise concerns before work becomes operational. Leaders should encourage candid discussion of mistakes while maintaining accountability for agreed safeguards. Experimentation should produce inspectable learning, including reasons to abandon an approach. Moving into real service delivery requires a separate decision supported by appropriate evidence.

12. Looking ahead, what capabilities should healthcare organisations prioritise to ensure that their workforce can adapt as generative and agentic AI technologies continue to evolve?

Organisations should prioritise capabilities that remain useful when tools change: workflow analysis, critical appraisal, information stewardship, evaluation and professional judgement. Staff also need to collaborate across clinical, educational, technical and governance roles. As systems gain the ability to act, understanding permissions, approval boundaries, monitoring and recovery will become increasingly important. Institutions should maintain reusable test cases and review learning when models, information sources or workflows change. Workforce development needs continuing investment rather than a single introductory course. The aim is a workforce able to explain and evaluate AI-supported work, recognise changing risks and retain meaningful responsibility for its outcomes.