How can healthcare organizations build a solid data foundation to leverage on the potential of Artificial Intelligence?

To see what it means to be ready for artificial intelligence (AI), I’d like to stand on the shoulders of a data science pioneer who laid the foundation for health information management.

In 1837, England established the national register of marriages, births and deaths, later expanded to include occupation, age, and cause of death. Dr. William Farr headed this up as the first “compiler of abstracts”, realizing early on that this was not just a numbers game but a critical research tool. He was among the first to classify morbidity and mortality, link labor conditions with health, and study major epidemics.

However, despite being a meticulous researcher, Dr. Farr was also wrong when it came to researching the cause of cholera epidemics that ravaged London from 1848 to 1854. The good news is that he had the humility and the wisdom to continue to learn. He not only got it right later but provided us with some valuable lessons.

So, what can we learn from this medical data pioneer as we prepare for AI in healthcare today?

1. Understand Your Objectives

Most healthcare organizations probably don’t plan to build AIs and large language models (LLMs). However, they will want to benefit from AI solutions. For these to work, they will have to train and test AI solutions against their own datasets.

You might also want to employ machine learning on healthcare data to add predictive models to an electronic medical record (EMR) system. Some organizations may also want to build new, AI-native digital health solutions or do cutting-edge clinical research using AI.

Whatever your objectives, as Dr. Farr discovered, they are all dependent on good data.

2. Identify Information

Dr. Farr recognized that many factors contributed to health and well-being and extended his observations beyond mortality and morbidity. He also collected what we would today term “social determinants of health” and environmental data, including day of week, weather, geography, property ownership, population density, and elevation.

Healthcare organizations looking to benefit from AI must also consider what information is available and relevant to their objectives. Sources could include health, business, genomics, demographic, and environmental data.

Your data should also be consistent across sources, complete, appropriately structured, and sufficiently diverse. Of course, I’m talking about more than just identifying the scope and scale of information. You need to be able to manage it as well, which brings us to our third lesson.

3. Collect and Normalize

Dr. Farr got paid to collect data but he realized that it was useless if it wasn’t normalized or complete. If we think our unstructured data today is bad, consider what it would have been like in the nineteenth century. So, Dr. Farr developed a system which we now know as the International Classification of Diseases (ICD), and still use today, to make the data meaningful.

Things have come a long way since then. Modern standards like HL7® FHIR® support the normalization of healthcare data and interoperability between systems to make data consistent and meaningful. That is important for EMR or analytic systems and critical if you want to get good results from AI.

Here’s a great example. The U.S. State of New Mexico asked our customer TriCore and its Rhodes Group subsidiary to find undiagnosed hepatitis C patients and get them into treatment. TriCore was tasked with automating this process and creating a treatment dashboard for primary care providers.

That meant collecting data from 26 clinical laboratories with little interest in sharing data let alone developing individual system interfaces. TriCore was stumped until they realized that they could collect and normalize the data using FHIR APIs, which didn’t require the laboratories to share data with one another or to develop costly interfaces. The project was so successful it got 34,000 citizens into treatment for a deadly and costly infectious disease.

As the Rhodes Group CEO said at the time: “You can read about FHIR. You can talk about FHIR. But until you have one of those ‘Aha’ moments, you don’t really appreciate what it delivers.”

 
4. Recognize and Overcome Bias

Interpreting the data about the 1848-49 and 1854 London cholera epidemics, Dr. Farr rejected a hypothesis that had been proposed by Dr. John Snow, associating cholera mortality and water. Farr instead stratified mortality by elevation to support the prevailing ‘miasma’ theory of the time that bad air was spreading the disease. His biased conclusions delayed interventions in the water supply and led to needless deaths.

Biased data or interpretation is also a huge problem with AI. Missing or inaccurate data can skew AI predictions, favoring outcomes based on the flawed datasets. Training data reflecting past viewpoints can perpetuate or amplify biases in AI decisions. Data not representative of the population can result in models biased toward the sampled group.

You don’t have to look very far to find biased medical data. The commonly used Body Mass Index (BMI) was developed in the 1830s by a mathematician using measurements of Belgian men; pulse oximeter readings are less reliable for people with darker skin.

So, if you are using AI models trained in a different environment, you will want to train them on your own data and validate the results. If you use AI for clinical research, you will need datasets for the target population.

5. Learn from One Another

Interestingly, Dr. Snow based his own work on the data gathered by Dr. Farr. And Farr, while initially disagreeing, was wise enough to expand his data gathering and to continue his own analysis, within a few years, he too recognized the connection between water and cholera transmission, and was able to work together with Snow and others to control the deadly disease.

In preparing for AI, we must also learn from one another. The King Khaled Eye Specialist Hospital (KKESH) in Saudi Arabia is a good example. They were plagued by no-shows; a high percentage of patients simply did not turn up for appointments. KKESH tried all the usual tricks – text messages, email reminders, and voicemail notifications – to no avail.

We worked with KKESH to develop a machine learning model to predict no-shows using the InterSystems IRIS for Health™ platform data on data from their InterSystems TrakCare® EMR system. They now use the information to proactively contact those patients, reducing the no-show rate by 40%, freeing those appointments for others on the wait list, and avoiding countless hours of lost clinician productivity.

Another InterSystems customer, Mirus Australia, provides advisory services and applications for aged care. It now uses machine learning and generative AI to populate its standardized OMOP database for collaborative research and development. Leveraging AI, the solution ingests unstructured data and normalizes it, replacing a costly manual chart abstraction process costing US$250 million.

We can also learn from startup companies like Singapore-based Jonda Health, which recently released the JondaX low-code health data transformation engine. Using AI and machine learning models and the InterSystems IRIS data platform, JondaX converts medical data in diverse formats, such as PDFs, into standardized formats including HL7 and FHIR. This enables more equitable access to healthcare data, while streamlining processes.

Inventing the Future with AI

A 2004 study showed that if logistic regression techniques had been available to Dr. William Farr, its application to the 1852 data set would probably have changed his faulty conclusion that a district’s elevation above high water was the most important explanatory variable for cholera mortality.

Likewise, new AI solutions and techniques will address problems we cannot solve today. Like Dr. Farr, we must prepare by building a solid data foundation. Healthcare organizations that work with InterSystems are building that solid data foundation so they are already ready for the data management opportunities and challenges of tomorrow.  These healthcare leaders are among those that are inventing the future at breakneck speed. We need to learn from them – and those who went before – to help chart the right course with AI.

To learn more about this topic, join InterSystems on-demand webinar: Are You and Your Data AI Ready?