EXPLORING THE KNOWLEDGE BASE FOR HEALTH INFORMATICS: The outcomes of the Otley (2005) meeting.


Prof. Graham Wright
Dr. Peter Murray
Dr. Helen Betts

CHIRAD, Centre for Health Informatics Research and Development (CHIRAD),
The University of Winchester, United Kingdom
Email: profwright@gmail.com

Education - Aims and objectives; Educational psychology: education; Health Informatics.



Indice general


Este artículo describe un proyecto en ejecución, encaminado a definir la base cognitiva de la disciplina Informática en Salud. Lo que inicialmente era una iniciativa de investigación a pequeña escala se ha convertido en parte de un proyecto mas amplio y de mayor alcance que es financiado conjuntamente por la International Medical Informatics Association (IMIA) y el British Computer Society Health Informatics Forum (BCHIF). Las salidas del proyecto ayudarán a definir el núcleo de conocimientos en que se centra el plan estratégico de IMIA. El artículo explora los fundamentos y el contexto del enfoque aprobado por el equipo de investigadores. El proceso de investigación y las tácticas usadas en el taller de tanques pensantes en Otley, 2005; se exploran en detalle y finalmente el artículo conduce al lector a los resultados de la reunión de Otley, algunas implicaciones derivadas de la misma, y el flujo de trabajo que subsigue.

Palabras Clave:
Educación – Fines y objetivos; Psicología educacional; Educación, Informática en salud.


This paper describes an ongoing project that aims to define the cognitive basis of the discipline of Health Informatics. What was originally a small scale research initiative has become part of a larger and wider ranging research project that is being jointly funded by the International Medical Informatics Association (IMIA) and the British Computer Society Health Informatics Forum (BCHIF). The outputs of the project will help to define the knowledge core at the centre of IMIA's Strategic Plan. The paper explores the background and context of the approach taken by the research team. The research process and tactics used at the2005 Otley Think Tank Workshop is explored in detail, and finally the paper points the reader to the outcomes of the Otley meeting, some implications, and to further ongoing work.




The Centre for Health Informatics Research and Development (CHIRAD) research team have been attempting to build a mapping of the discipline of Health Informatics. The team recognised the valuable work undertaken by previous researchers in the field and also realised the enormity of the task they were facing, and so focused in the first instance on the cognitive mapping exercise which is the focus of this paper.
The Education Steps think-tank meeting of 2005 in Otley, was the first stage of this project initiated by the Health Informatics Forum (HIF) of the British Computer Society (BCS) and conducted by the CHIRAD.
The original aim of the project was to explore the theoretical constructs underpinning health informatics, and from this, identify and investigate the educational issues. The further aims of the project were also to consider the evidence base of health informatics, and examine the existence of, or need for, and so content of, different levels of education in health informatics. It is only now that the Otley meeting has taken place, and as a result of considerable reflection on the process and initial outcomes that the authors are in a position to reveal the direction of the intended research agenda to be covered over the next years, building on the first phase of the overall project.
In essence this will consist of triangulation, expert opinion, document analysis and literature review, to complete the mapping.
The present paper will cover:

  • the context of the project, in terms of pre-existing work and underpinning issues and concerns;
  • the educational framework selected for the project, i.e. Bloom's taxonomy;
  • the approaches used for conducting the workshop;
  • the workshop itself and a summary of the discussions and involvement of participants;
  • plans for further work to build on the Otley meeting.

Background to and context of the Otley meeting

In the UK Health Informatics community there has been a growing concern about the pre-dominant modules/competencies approach, with work-based skills-oriented delivery, to health informatics education in the United Kingdom.
While it was recognised that this work in itself was useful, and had a valuable role to play, it seemed, to some people, to be resulting in a 'lowest common denominator' approach that did not provide an atmosphere for the higher level scientific and theoretical development of the discipline of health informatics. Indeed, it could be argued that such an approach potentially threatened an atmosphere in which one could nurture the exploration of the higher level scientific development.
It was recognised, by the research team and by BCS HIF, that there existed, from developments over the previous 5-10 years, and primarily funded through the NHS Information Authority (NHSIA), much good work relating to skills and competency frameworks for health informatics. It was felt that this area had been well rehearsed, and so it was not the intention of the project to revisit this work; it was recognised, though, that this work would need to be taken into consideration. And it is the intention to do so in the next phase of the research.
It was also not the aim of the project to seek to try and define health informatics or explore the most appropriate name for the field. Rather, the intention was through essentially a mapping exercise that relied on the views of health informatics experts, to examine the core components of the discipline by collating the scientific elements of the subjects and thematic areas within the domain.
The participants were excited by the possible uses of the outcomes which they listed as including

  • trying to answer the question 'is there such a thing as health informatics as a subject?;
  • producing a position statement about the levels of elements;
  • assisting in the maturing of the identity of the health informatics profession;
  • providing the basis for a transferable credit rating system;
  • bringing together education and training elements of health informatics;
  • contributing to the academic rigor of the UKCHIP (www.ukchip.org ) framework;

Literature and pre-existing work

It could be argued that much of the work envisaged within the project had already been undertaken, both within the UK and internationally. The detailed descriptions of competency frameworks referred to above, and work such as the development of the IMIA Scientific Content map could be seen as overlapping work. As will be discussed in the next sections, the focus of the Education Steps project is on education, as opposed to training, and in particular addressing the cognitive as opposed to psychomotor domains within learning.
In the UK, the NHS Information Authority has developed checklists of competencies for a wide range of health specialists, much of this work being done during the late 1990’s and early 2000’s. These focused largely on IT skills but also included; security, confidentiality, data quality etc. which were considered vital or useful for different staff groups.
These approaches formed a considerable part of the human resource strategy document “Making Information Count: A Human Resources Strategy for Health Informatics Professionals in 2002”. This contained both Information Technology and Information Management competencies for medics, nurses, and records staff, etc., and provided the basis for Professional Qualifications in IM&T More recently, the adoption of the European Computer Driving Licence as the minimum standard in IT skills for NHS staff has perpetuated the competency driven approach. Some other health informatics competencies have been created as part of the knowledge and skills framework, which was developed in conjunction with skills for Health as part of the National Occupational Standards Database and underpins career progression and payment under the UK government Agenda for Change initiative which relates skills to pay.

Why use Bloom's taxonomy?

The Otley meeting used the cognitive domain of Bloom's taxonomy as a theoretical framework within which to situate discussions, and aimed to collate the conceptual basis of health informatics, with the focus on education rather than training and competencies, through using a mixture of small group work and plenary discussions.
As the members of the team had a strong background within the academic world, and had many years' experience in delivering education, designing curricula, and associated academic and scientific activities, including research and publication, they felt that the project should be situated within an appropriate academic context. It was therefore felt that a theoretical framework would be useful within which to situate the work of the project. Bloom's taxonomy 1, 2 was selected, as it is well known from health, medical and nursing curricula, and provides a hierarchical framework for categorising levels of abstraction for objectives within educational settings, and maps well against other academic levels, eg progression from undergraduate to postgraduate levels.
Bloom identified three domains of educational activity 2: the cognitive, relating to knowledge and mental skills, the affective, relating to attitude, feelings and emotions, and the psychomotor, relating to manual or physical skills. Within the cognitive domain, Bloom recognises six levels of educational objectives – from the lowest, knowledge, through comprehension, application, analysis, synthesis, to evaluation, the highest level. Using the cognitive domain of Bloom's taxonomy, it was intended that participants in the Otley meeting would be able to collate the conceptual basis of health informatics to elucidate the elements that can subsequently be categorised as knowledge, comprehension, application, analysis, synthesis and evaluation.
The research team also felt that the spiral curriculum 3, 4, another construct widely used within health curricula, was an additional part of the theoretical framework within which the project work would be situated. Bruner 3 describes the concept of a spiral curriculum wherein basic ideas within a curriculum are revisited at intervals, in such a way that earlier knowledge is added to and built on as one progresses up the spiral, either within a given course or as one moves to higher level qualifications.
As a basis for this work, we asked participants to familiarize themselves with documents such as the IMIA Scientific Content map, and any other relevant documents to which they had access. We asked, in addition, that, in preparation for this workshop, participants should bring with them either:

  • the module descriptions from any health informatics programme that they were involved with, which should include the aims and objectives or learning outcomes, and indicative content; or
  • a list of the models, theories and laws, constructs and concepts that framed their understanding of the discipline of health informatics.

We also emphasised to potential participants that the focus would be on education rather than training, and that we were going to explore the cognitive rather than psycho-motor aspects of the health informatics discipline.

The ducks and ponds metaphor

The workshop aimed to capture all the elements of the discipline of health informatics, and also the broad themes or subject areas into which these elements can be grouped. It was at this point that the analogy of ducks (for the individual elements) and ponds (for the broad themes of the framework) was felt to be an appropriate way of helping participants to visualize the tasks they would be undertaking. The six levels of the cognitive domain of Bloom’s taxonomy were seen to form the basis of the grouping of the ‘ducks’ within the ‘ponds’, and it was felt that they might contribute to understanding the curriculum of knowledge for the syllabus or curriculum at different academic levels.
Participants in the Otley meeting would be asked to identify 'ducks', but at the same time, to think 'what does a duck look like?' This meant that in describing them, there was a need to include an active verb from one of Bloom's cognitive levels, plus a generic principle. It was also felt that we would need to identify as many 'ducks' as possible, that is, capture the fullest size of the 'flock of ducks' (total numbers) and try not to become enmired in the detail of a few examples. The role of facilitators in encouraging the discussion groups to pursue this was seen to be crucial. It was recognised by the team that this would potentially be a very challenging exercise, with the risk of discussion focusing on a few examples if different participants had differing views on the nature or importance of particular ducks; however, we also felt that the breadth of experience of participants would help in capturing a large number of different 'ducks'.
The broad themes (ponds) also needed to be identified. The order of the workshop, i.e. whether to first identify ducks and then group them into ponds, or whether to first identify ponds and then to identify ducks, generated considerable discussion within the research team, which eventually resulted in the 'ponds first' decision.

The mechanisms at Otley

The Otley meeting was an intensive 24-hour think-tank, of workshop format and involving small group and plenary discussions, with participants and the team in residence overnight. There were 24 participants, who came from a number of health informatics groups, including BCS Health Informatics Forum, BCS Health Informatics Specialist Groups, ASSIST (The Association of ICT Professionals in Health and Social Care) and IMIA (International Medical Informatics Association), the world body for health/medical informatics. Most of the participants were from the UK (and covered all four home countries), whilst others came from Europe, Australia, South Africa and the USA.

A summary of the process:

1. Within small groups, participants identified the main subject areas (ponds) from their own lists, curricula or knowledge and experience.
2. Participants identified elements (ducks) of subject areas within small groups.
3. Participants assigned each duck to a subject area and, where possible, a level from Bloom's cognitive domain within the whole group.

The first set of ducks and ponds

Using the metaphor of 'ducks' and 'ponds' to represent respectively the finer elements of the discipline and the broad themes within which those elements could be clustered, the discussions resulted in a first set comprising 221 ducks in total, grouped into 13 ponds.
The ponds varied in size, with the smallest containing 6 ducks and the largest 37, although it was already recognised by participants that the largest pond would likely be divided following further discussions.
The 13 ponds are currently named as follows, although the names, again, may be subject to change as a result of ongoing discussions: Health and social care – care processes; Health (care) records; Health informatics standards; Computer Science for Health Informatics (ICT for Health); Health and Social care Industry; Knowledge Domains & Knowledge Discovery; Legal & Ethical; People in organisations; Politics and policy; Terminology, classification and grouping; Toolkit (systems); Uses of clinical information; and Uses of informatics to support clinical healthcare governance. A complete list of the ducks and ponds developed at the Otley meeting is given in Appendix 1.

Validating and disseminating the Otley outputs

This first listing resulted from a time-limited discussion among a relatively small group, albeit of nationally and internationally recognised experts. All participants recognise the provisional nature of this first list and the need for further reflection, and refinement of the list. Through an iterative process of refinement, it was hoped that a generally acceptable list of ducks and ponds could be generated.
The Otley participants were invited to comment on the list, and to propose amendments, through a mixture of individual commentary and online group discussions. There was a general feeling among participants that the set of 'ducks', and particularly of 'ponds' that had been developed and agreed was not complete, and that others would emerge as a result of further discussions following the workshop. While there was a general consensus on the first listing, it had been evident in the discussions that some people felt that what had been identified as 'ponds' might, as a result of further discussion and reflection, be seen to be simply 'ducks'.

EFMI and AMIA workshops
In 2005 two workshops were held at conferences in Europe (MIE Geneva) and America (AMIA Washington) which were only of short duration and thus explored a narrow focused area. In both workshops the authors explored the overall concept and the clinical informatics theme with the participants.

Belfast meeting
In March 2006 there will be a 24 hour workshop to further refine the technical and computing themes developed at Otley.

Early applications
Dr Simon de Lusignan, working at St Georges Medical School in London, has used the outputs from Otley to help formulate an undergraduate Health Informatics degree programme.

Reflections on the process: did we achieve anything worthwhile?

The question that naturally arises after any project or meeting is 'would we do the same again, and in the same way?' Bringing together a group of 30 health informatics experts, from a range of practical and theoretical backgrounds, with a wide range of special interests, and collectively with in excess of 500 years experience in health informatics, and asking them to explore, and possibly challenge, the fundamental basis of their discipline, was always going to be a potentially risky undertaking. It is a tribute to the professionalism and enthusiasm for their areas of expertise that the participants at the Otley meeting engaged in the process so enthusiastically and that they felt that the beginnings of something useful resulted from the time and effort.
The overall structure of the workshop was felt to work well, by both participants and the Planning Team. The greatest complaint was the lack of time devoted to each stage of the process, and in particular to some of the discussions and sessions when there was an attempt to reach consensus where several disparate views clearly existed. The level of knowledge displayed by participants, the passionate defences of their views, and the enthusiasm displayed at all stages of the workshop clearly showed that we had the right types and mix of people involved.
Participants in the Otley meeting found, after some persuasion and initial scepticism, that although the metaphor has limits and can be stretched and mangled in many ways, it was a useful way of starting to think about what are the essential broad themes (ponds) within health informatics as a discipline, and what are the essential finer-grained components (ducks) that are the parts of those themes. As might be expected from experts with strongly-held views and coming from a wide range of practical and theoretical backgrounds and interests, the discussions were wide-ranging. It was quickly acknowledged that the scope of the tasks was huge and that we could only make a small start. The attempt to focus on the cognitive domain of the discipline, and to break free of the constraints of thinking only about psychomotor skills and competencies, caused problems for some participants. Yet it is remarkable that, even within a short period of intense discussion, we managed to make some progress towards agreement on some of the broad themes.



The team have had many wide ranging and interesting discussions since presenting the initial outcomes and have revisited the overall methodology. It is the team’s intent to invite a wider group of academics to be involved in the next stages of this research. It is proposed to triangulate the Otley outcomes to a discourse analysis of Health Informatics peer review articles and the published work of others on competencies.
BCSHIF continues to support the project financially, and with many members providing invaluable input. It is hoped that the Health Informatics community at large through the International Medical Informatics Association can be engaged in the next stages of the project. The next stages of the work, from which we hope to be able to provide some preliminary results by Informatica2007, are jointly funded by BCSHIF and IMIA.
The next steps of this work will include:

I. A literature analysis examining the emerging themes and high level descriptors using discourse analysis software and methods. This will be based in an analysis of available electronic literature, and using established and novel indexing and analysis techniques.

II. A consensus workshop to explore the commonalities and differences between the Otley outputs with the document analysis.

III. Review of existing models and frameworks, particularly those being used nationally and internationally.

It is the intention of the research team to have phase three completed in time to report at the Medinfo conference in 2007.
The Outcomes from all the workshops can be found through the CHIRAD website on the Education Steps project website at http://www.differance-engine.net/educationsteps/


The Otley meeting was funded by a research project grant from the BCS Health Informatics Forum.



  1. Bloom B. Taxonomy of educational objectives. BostonMA: Allyn and Bacon, 1984.

  2. Clark D. Learning domains, or Bloom's Taxonomy. 1999. Ref Type: Internet Communication (mention the www).

  3. Bruner J. The process of education. CambridgeMA: HarvardUniversity Press, 1960.

  4. Steinaker N, BellRM. The experiential taxonomy:a new approach to teaching and learning. New York: Academic Press, 1979.



Appendix 1 First set of ducks and ponds from Otley meeting.

  Health and social care – care processes Analyses issues of consumerism in Health Informatics
    Analyses pros & cons of existing systems for primary clinical uses
    Apply telehealth solutions to the elderly population
    Asses the suitability of a package <x> to do <y>
    Clinical pathways and disease management
    Communities of practice
    Consumer Health Information
    Critiques information for patients
    Describe decision making processes in clinical decision making
    Describe stages of communication from source to recipient and what influences
    Describe systems currently used by clinicians to gather clinical information
    Evaluate the need for specialty-specific clinical information systems
    Evaluates how Health Informatics affects outcomes
    Evaluates Patient Journey
    Gives examples of patient outcomes and how to measure them
    Identify patient information needs and range of delivery methods
    Interpret / evaluate clinical information needs
    Knows how delivery of care differs in primary and secondary care
    Knows the method of primary to secondary to tertiary care referral and the flows of information
    Select an effective implementation strategy for a clinical system
    The care pathway
    Understanding the clinical consultation
    Understand conflict in the interface between the technology and caring philosophies
    Understanding the healthcare process
    Understanding the treatment process
  Understanding the diagnostic process
  Health (care) records Appreciates the history of methods of gathering information in the clinical workplace
    Audit trails
    Case note tracking
    Clean clinical data
    Clinical communication from message to meaning
    Data structures
    Decision support
    Define EHR
    Explains the usefulness of clinical recordkeeping standards
    Patient: clinician consultation
    Patient Access to records
    Patient and provider identification
    Patient safety
    Transition to paperless
  Understands the importance of maintaining the integrity of the Master Patient Index and Patient Numbering Systems
  Health Informatics standards Assess quality of draft message design process standard
    Assess utility of a clinical interface
    Clinical coding systems
    Describes Technical Infrastructures
    Unique Identifiers Design & develop
    Technical standards - Design & Implementation
    Design of specific clinical message
    Explains the needs for technical standards
    Interoperability standards
    Ontologies & data definitions
    Patient Safety
    Standards for coding terminology and communication
  Uniques architectures & standards development
  Computer Science for Health Informatics (ICT for Health) Apply / Fix any integrity issues in EHR content
    Benefits realization (should this be here?)
    Biomedical equipment interfaces
    Build a prototype system for a department
    Collaborative Internet architectures
    Computer literacy (ECDL)
    Data structures
    Decision support tools
    Demystify IT for users
    Design databases (per se & in health i.e. for a clinic)
    Design of IS/IT systems for Health
    Differentiate between technical, syntactic & semantic interoperability
    EPR: architecture, content, views & use
    Evaluation of implementations
    Explain / comprehend Messaging Standards
    Explain engineering
    Explain the ?operation of a Picture Archiving Systems (PACS)
    Human Computer Interaction (HCI) principles
    Information Retreival
    Knowledge management
    Mobile technology
    Natural Language Processing
    Safety Critical Software
    Systems desgn & the application of systems theory to design
    Systems Implementation
    Technical skills - understand It architectures, PC desktop applications, EHR concepts, structures and infrastructure required
    Use of office / database / web applications
    Visualisation (how to, using HCI, 2D/3D graphics / commercial products)
  Why DSS do / do no work and are / are not cost-effective
  Health and Social care Industry Assess Health Informatics readiness of an organisation
    Healthcare models: economic and financial
    Healthcare supply chain
    Know history of NPfIT and international equivalents
    Payment by Results etc (part of H.Econ Econ drivers)
    Summarises health care systems economics
    Understand the health care organisation
  Understands how organisations work (incl H&SC organisations)
  Knowledge Domains & Knowledge Discovery Benefits realisation
    Clinical content production
    Clinical Data structures
    Compare clinical (and other) outcomes (?benefits realization)
    Data mining
    Data Structures
    Decision support
    Evaluate integrity of decision support system
    Evaluate the impact of a health informatics intervention
    Information sources
    Knowledge of performance & process improvement
    Measuring (evaluating) performance of organisations
  Produce knowledge domain constraint models (Tom Beale)
  Legal & Ethical Apply Health Informatics ethics
    Applies principles of security in a health context
    Current relevant legislation
    Data security
    Data security, privacy, confidentiality, access, integrity and standards
    Defends telemedicine & telecare
    Describe how Health Informatics benefits patients
    Ethics & governance
    Implement safe & secure IS/IT systems
    Justifies eHealth
    Knowledge of HH policy regulatory Requirements around medical record documentation
    Medical ethics
    Patient safety
    Principles of Freedom of Information and other legislation
    Understand component parts of 1998 Data Protection Act & other privacy legislation
  Understand the impact of breaches in confidentiality on patient care
  People in organisations Champion benefits of health informatics
    Change management
    Compose educational resources for healthcare professionals
    Comprehends psychology / sociology
    Contract management
    Contrast management Theories
    Educate and support IT/IS users
    Explains Health Informatics
    Going paperless
    Implement new information systems
    Learning Skills (Learning to Learn)
    Project governance
    Understanding healthcare professional roles
    Understanding the needs of patients / clients / professionals
  Use communication & teaching / learning strategy to present information to patients / clients
  Politics and policy Information Systems Strategies - Alignment with External forces
    Analyse differences in strategy when comparing ehealth / IT programmes within UK
    Describe likely trends in Health Policy and IT
    Differentiates professional organizations
    Discuss the issues facing the NPfIT programme
    Healthcare relationship with social conditions
    Interpret policy & strategy
    Local healthcare strategy
    National IT policies & strategies
    Understand Payment by Results (part of Health economics)
  Understand the economic forces driving the healthcare industry
  Terminology, classification and grouping Describe limitations of coding systems
    Describe the significance of accurate data
    Describe types of coding, terming, classification
    Evaluate competence of a terminology for a clinical task
    Evaluate different terminology systems that describe health
    Explain clinical coding systems
    Ontologies - Data definitions
  Standards for coding, terminology & communication
  Toolkit (systems) Ability to do Current State Analysis to Future State work processes using IT
    Ability to summarise information requests
    Ability to undertake audit
    Appraise health information
    Appraise options for IS / IT solutions
    Benefits realisation
    Build a model of a hospital department
    Compare the effects of alternative system designs
    Construct systems
    Data modelling to support analysis : warehouse / retrieval / EBP
    Decision Support
    Develop & implement information / ICT strategy
    Evaluate health system information flow & system analysis
    Evaluate healthcare
    Evaluate IS/IT
    Evaluate proposed systems
    Find & appraise appropriate research method
    Formulating questions
    How to apply tools
    How to do Computational modeling
    Manage information systems - the management of IS
    Manage projects &/or services
    Messaging standards
    Modelling of processes - by various methods
    Political influencing
    Process Analysis & redesign
    Process optimisation
    Produce a business case for a system purchase
    Project / Contract management tools
    Project planning
    Provide Business requirements for system
    Reconstruction of Clinical Guidelines in a form for a CDS sys
    Research methods of information science and healthcare
    Statistical methods
    Systems implementation
    Understanding the different models (of HCI)
  Write the elements of care planning
  Uses of clinical information Analyse (clinical) data
    Carry out statistical tests (simple)
    Data analysis & statistical presentation
    Data for population management
    Data mining
    Data quality
    Data structures
    Data warehousing
    Describe the significance of accurate data
    Describe the uses of clinical information in secondary and primary care
    Differentiate relative and actual risk
    Drive improvements in data quality
    Explain clinical coding systems
    Geographical Information systems
    Identify data sources to support clinical risk management
    Interpret clinical data
    Know examples of information systems used to gather clinical data for secondary care purposes
    Present information
    Techniques to maximize data quality
    Understand the clinical audit cycle
  Understand uses of HES data
  Using informatics to support clinical healthcare governance Adverse Event Management
    Clinical audit & effectiveness
    Organisational Learning
    Patient Information i.e. Health Education
    Patient safety
  Risk Management