GenAI on 🔥 (FHIR)
We developed an advanced healthcare interoperability application using GenAI and FHIR
Blanc Labs has spent the last five years working with healthcare companies like Reliq Health Technologies and Shannex as well as partners like Smile Digital Health and AWS, on building healthcare interoperability platforms and technologies on the latest standards such as FHIR and HL7.
Given the excitement around Generative AI and Large Language Models (LLMs) and their potential to transform entire swaths of how we build and interact with technology, our healthcare team wanted to experiment with combining Gen AI tools with an FHIR standard to see if we could leverage it for a practical healthcare scenario.
As a result, our team developed a proof of concept that is meant to showcase the power of conversational AI tools to query an interoperable database of medical information.
For this PoC, we used OpenAI to query Health Samurai‘s FHIR platform, Aidbox.
Blanc Labs has spent the last five years working with healthcare companies like Reliq Health Technologies and Shannex as well as partners like Smile Digital Health and AWS, on building healthcare interoperability platforms and technologies on the latest standards such as FHIR and HL7.
Given the excitement around Generative AI and Large Language Models (LLMs) and their potential to transform entire swaths of how we build and interact with technology, our healthcare team wanted to experiment with combining Gen AI tools with an FHIR standard to see if we could leverage it for a practical healthcare scenario.
As a result, our team developed a proof of concept that is meant to showcase the power of conversational AI tools to query an interoperable database of medical information.
For this PoC, we used OpenAI to query Health Samurai‘s FHIR platform, Aidbox.
Why we did it
In our opinion, providing a conversational UI/UX to the end user will dramatically enhance the healthcare practitioner’s ability to access and act upon patient data on-demand instead of being limited by traditional web UI/UX.
The goal of building this prototype is to evaluate the strengths and weakness of Generative AI technologies to convert natural language instructions to corresponding FHIR API calls.
Source 1 and 2: Canada Health Infoway – Pan Canadian Interoperability Roadmap
Solution Approach Overview
We built a prototype that allows healthcare practitioners to access any arbitrary patient data simply by asking questions in natural language.
To develop this prototype, we leveraged OpenAI’s GPT3.5 APIs as the foundational LLM to convert natural language to a FHIR query that would be run on Health Samurai’s Aidbox FHIR platform. We fine-tuned the standard GPT3.5 LLM model using prompt-engineering techniques to better understand healthcare-based user instructions and matched it with corresponding standard FHIR APIs to retrieve the required data. The model was also fine-tuned to output the data in a non-technical, natural language format.
While it is built on Health Samurai’s Aidbox with OpenAI APIs, this application can be customized for other FHIR providers and other Gen AI APIs without making major changes to the fundamentals to the application.
Examples of questions we can ask the AI–Powered Knowledge Assistant
What is the number of patients in the system?
- Give me information about patient number XX*.
- What’s the name of patient number XXZ?
- Tell me about the last medical record of patient number YYT.
- I want to know about the conditions that patient number XYZ is presenting.
*In a real-life scenario, XX would be replaced with the patient ID
Blanc Labs has developed an innovative Proof of Concept that leverages Generative AI and our FHIR platform, Aidbox, to enable natural language access to patient data through FHIR APIs. We believe this could have
mainstream application around healthcare data accessibility leading to enhanced patient care.
Gen AI on 🔥 Techstack
Lessons Learned
Training is both Art, and Science
- With this prototype, we have proven that quick and low-cost techniques like fine-tuning generic LLM models, refer Prompt Engineering techniques using OpenAI (e.g. GPT3.5) work in most cases in converting natural language to FHIR standards. This is a good alternative to the intensive and expensive approach of building foundational models from scratch or re-training existing LLMs for specific use-cases.
- Since FHIR is an extensive standard, the process to fine-tune LLMs for every scenario would be time-consuming. Hence developers can start by focusing on their specific use-case rather than fine-tuning for all possible scenarios.
Conversational Queries Do Not Equal Structured Query Language
- As we were fine-tuning the models using prompt engineering, we observed if the prompts were not detailed and precise, we would end up with ambiguous answers. In our attempt to make the UI easy to use by non-technical users who are not aware of FHIR standards, we spent a considerable amount of time in perfecting our prompt engineering techniques for specific use-cases.
- This approach works well for a layman approach to asking questions to present information from the dataset, but it is interesting to consider how a “power user” ie. A clinician with a more informed understanding of what data they were looking for, as well as the taxonomy of the dataset they are querying could yield different outcomes for users.
- One of the limitations of the GPT 3.5 LLM model was the limit of characters that can be returned. Some queries can’t be completed because their response would be larger than what is supported by the LLM model. For such scenarios, we would have to break a single prompt into multiple prompts. Their responses would need to be intelligently merged to provide a sensible output to the user.
Give Me the Data, But Make It Usable
- Extracting raw data from a FHIR platform can be very useful, but often the data is one dimensional or very dense. There is a big opportunity around health informatics3 to present the data in various ways to make it easier to consume, more relevant or easier to pinpoint the “so what” of what is being presented.
- Future improvements to the prototype’s functionality can be made by enhancing it to include responses in other data formats like tables and graphs to help users better visualize certain kinds of health data.
What’s next?
Our intention is to enhance the demo to support more complex queries involving more than one FHIR API call as well as supporting richer output formats like charts & tables.
We’d also like to hear from the FHIR community on their thoughts about how to take this to the next level. Please participate in the brief questionnaire below to provide you input and feedback.
What did you think of this initiative? Give us your feedback in the form below.
Latest Insights
Lenders Transformation Playbook: Bridging Strategy and Execution
AI’s Mid-Market Makeover in Financial Services
Mid-sized financial services institutions (FIs) are facing significant challenges during this period of rapid technological change, particularly with the rise of artificial intelligence (AI). As customer expectations grow, smaller banks and lenders must stay competitive and responsive. Canada’s largest financial institutions are already advancing in AI, while many others remain in ‘observer’ mode, hesitant to invest and experiment. Yet, mid-sized FIs that adopt the right strategy have unique agility, allowing them to adapt swiftly and efficiently to technological disruptions—even more so than their larger counterparts.
Process Improvement and Automation Support the Mission at Trez Capital 🚀
Trez distributes capital based on very specific criteria. But with over 300 investments in their portfolio, they process numerous payment requests and deal with documents in varied data formats. They saw an opportunity to enhance efficiency, improve task management, and better utilize data insights for strategic decision-making.
Align, Assemble, Assure: A Framework for AI Adoption
An in-depth guide for adopting and scaling AI in the enterprise using actionable and measurable steps.
Blanc Labs Welcomes Two New Leaders to Advance AI Innovation and Enhance Tech Advisory Services for Financial Institutions Across North America
Blanc Labs and TCG Process have partnered to transform lending operations with innovative automation solutions, using the DocProStar platform to enhance efficiency, compliance, and customer satisfaction in the Canadian lending market.
Blanc Labs Partners with TCG Process to Integrate their Automation and Orchestration Platform and deliver Advanced Intelligent Workflow Automation to Financial Institutions
Blanc Labs and TCG Process have partnered to transform lending operations with innovative automation solutions, using the DocProStar platform to enhance efficiency, compliance, and customer satisfaction in the Canadian lending market.
BPI in Banking and Financial Services in the US & Canada
Banking and financial services are changing fast. Moving from old, paper methods to new, digital ones is key to staying in business. It’s important to think about how business process improvement (BPI) can help.
Business Process Improvement vs Business Process Reengineering
Business process improvement vs. reengineering is a tough choice. In this guide, we help you choose between the two based on four factors.
What is the role of a Business Process Improvement Specialist?
A business process improvement specialist identifies bottlenecks and inefficiencies in your workflows, allowing you to focus efforts on automating the right processes.
Open Banking Technology Architecture Whitepaper
We’ve developed this resource to help technical teams adopt an Open Banking approach by explaining a high-level solution architecture that is organization agnostic.
Winning the Open Banking Race: A Challenger’s Path to Entering the Ecosystem Economy
Learn about the steps you can take in forming an open banking strategy and executing on it.
Canadian IT services firms offer a strategic opportunity for US Banks and FIs
Discover the strategic advantage for U.S. banks embracing innovation with cost-effective Canadian nearshore IT support.
These are not your grandmother’s models: the impact of LLM’s on Document Processing
Explore the transformative influence of large language models (LLMs) on document processing in this insightful article. Discover how these cutting-edge models are reshaping traditional approaches, unlocking new possibilities in data analysis, and revolutionizing the way we interact with information.
From Chaos to Clarity: Achieving Operational Excellence through Business Process Improvement
Discover transformative insights and strategies to streamline operations, enhance efficiency, and drive success.
Digital Transformation Vs. IT Modernization: What’s the Difference?
Healthcare Interoperability: Challenges and Benefits
Navigating the Healthcare Interoperability Journey
Finding the right API Management Platform
Lenders Transformation Playbook: Bridging Strategy and Execution
AI’s Mid-Market Makeover in Financial Services
Mid-sized financial services institutions (FIs) are facing significant challenges during this period of rapid technological change, particularly with the rise of artificial intelligence (AI). As customer expectations grow, smaller banks and lenders must stay competitive and responsive. Canada’s largest financial institutions are already advancing in AI, while many others remain in ‘observer’ mode, hesitant to invest and experiment. Yet, mid-sized FIs that adopt the right strategy have unique agility, allowing them to adapt swiftly and efficiently to technological disruptions—even more so than their larger counterparts.
Process Improvement and Automation Support the Mission at Trez Capital 🚀
Trez distributes capital based on very specific criteria. But with over 300 investments in their portfolio, they process numerous payment requests and deal with documents in varied data formats. They saw an opportunity to enhance efficiency, improve task management, and better utilize data insights for strategic decision-making.
Align, Assemble, Assure: A Framework for AI Adoption
An in-depth guide for adopting and scaling AI in the enterprise using actionable and measurable steps.
Blanc Labs Welcomes Two New Leaders to Advance AI Innovation and Enhance Tech Advisory Services for Financial Institutions Across North America
Blanc Labs and TCG Process have partnered to transform lending operations with innovative automation solutions, using the DocProStar platform to enhance efficiency, compliance, and customer satisfaction in the Canadian lending market.
Blanc Labs Partners with TCG Process to Integrate their Automation and Orchestration Platform and deliver Advanced Intelligent Workflow Automation to Financial Institutions
Blanc Labs and TCG Process have partnered to transform lending operations with innovative automation solutions, using the DocProStar platform to enhance efficiency, compliance, and customer satisfaction in the Canadian lending market.
BPI in Banking and Financial Services in the US & Canada
Banking and financial services are changing fast. Moving from old, paper methods to new, digital ones is key to staying in business. It’s important to think about how business process improvement (BPI) can help.
Business Process Improvement vs Business Process Reengineering
Business process improvement vs. reengineering is a tough choice. In this guide, we help you choose between the two based on four factors.
What is the role of a Business Process Improvement Specialist?
A business process improvement specialist identifies bottlenecks and inefficiencies in your workflows, allowing you to focus efforts on automating the right processes.
Open Banking Technology Architecture Whitepaper
We’ve developed this resource to help technical teams adopt an Open Banking approach by explaining a high-level solution architecture that is organization agnostic.
Winning the Open Banking Race: A Challenger’s Path to Entering the Ecosystem Economy
Learn about the steps you can take in forming an open banking strategy and executing on it.
Canadian IT services firms offer a strategic opportunity for US Banks and FIs
Discover the strategic advantage for U.S. banks embracing innovation with cost-effective Canadian nearshore IT support.
These are not your grandmother’s models: the impact of LLM’s on Document Processing
Explore the transformative influence of large language models (LLMs) on document processing in this insightful article. Discover how these cutting-edge models are reshaping traditional approaches, unlocking new possibilities in data analysis, and revolutionizing the way we interact with information.
From Chaos to Clarity: Achieving Operational Excellence through Business Process Improvement
Discover transformative insights and strategies to streamline operations, enhance efficiency, and drive success.
Digital Transformation Vs. IT Modernization: What’s the Difference?
Healthcare Interoperability: Challenges and Benefits
Navigating the Healthcare Interoperability Journey
Finding the right API Management Platform
Lenders Transformation Playbook: Bridging Strategy and Execution
AI’s Mid-Market Makeover in Financial Services
Mid-sized financial services institutions (FIs) are facing significant challenges during this period of rapid technological change, particularly with the rise of artificial intelligence (AI). As customer expectations grow, smaller banks and lenders must stay competitive and responsive. Canada’s largest financial institutions are already advancing in AI, while many others remain in ‘observer’ mode, hesitant to invest and experiment. Yet, mid-sized FIs that adopt the right strategy have unique agility, allowing them to adapt swiftly and efficiently to technological disruptions—even more so than their larger counterparts.
Process Improvement and Automation Support the Mission at Trez Capital 🚀
Trez distributes capital based on very specific criteria. But with over 300 investments in their portfolio, they process numerous payment requests and deal with documents in varied data formats. They saw an opportunity to enhance efficiency, improve task management, and better utilize data insights for strategic decision-making.
Align, Assemble, Assure: A Framework for AI Adoption
An in-depth guide for adopting and scaling AI in the enterprise using actionable and measurable steps.
Blanc Labs Welcomes Two New Leaders to Advance AI Innovation and Enhance Tech Advisory Services for Financial Institutions Across North America
Blanc Labs and TCG Process have partnered to transform lending operations with innovative automation solutions, using the DocProStar platform to enhance efficiency, compliance, and customer satisfaction in the Canadian lending market.
Blanc Labs Partners with TCG Process to Integrate their Automation and Orchestration Platform and deliver Advanced Intelligent Workflow Automation to Financial Institutions
Blanc Labs and TCG Process have partnered to transform lending operations with innovative automation solutions, using the DocProStar platform to enhance efficiency, compliance, and customer satisfaction in the Canadian lending market.
BPI in Banking and Financial Services in the US & Canada
Banking and financial services are changing fast. Moving from old, paper methods to new, digital ones is key to staying in business. It’s important to think about how business process improvement (BPI) can help.
Business Process Improvement vs Business Process Reengineering
Business process improvement vs. reengineering is a tough choice. In this guide, we help you choose between the two based on four factors.
What is the role of a Business Process Improvement Specialist?
A business process improvement specialist identifies bottlenecks and inefficiencies in your workflows, allowing you to focus efforts on automating the right processes.
Open Banking Technology Architecture Whitepaper
We’ve developed this resource to help technical teams adopt an Open Banking approach by explaining a high-level solution architecture that is organization agnostic.
Winning the Open Banking Race: A Challenger’s Path to Entering the Ecosystem Economy
Learn about the steps you can take in forming an open banking strategy and executing on it.
Canadian IT services firms offer a strategic opportunity for US Banks and FIs
Discover the strategic advantage for U.S. banks embracing innovation with cost-effective Canadian nearshore IT support.
These are not your grandmother’s models: the impact of LLM’s on Document Processing
Explore the transformative influence of large language models (LLMs) on document processing in this insightful article. Discover how these cutting-edge models are reshaping traditional approaches, unlocking new possibilities in data analysis, and revolutionizing the way we interact with information.
From Chaos to Clarity: Achieving Operational Excellence through Business Process Improvement
Discover transformative insights and strategies to streamline operations, enhance efficiency, and drive success.