EHR Assistant

Our project addresses the challenge that medical caregivers (doctors, nurses, assistants) face when they would like to extract medical information from electronic health records (EHR), which is a time-consuming, error-prone and inconsistent process today. On the other hand, we believe that voice is the perfect modality for doctor-EHR interactions and modern AI tools provide the capabilities to support such interactions. In our solution the medical caregivers can ask questions in natural speech, which is converted to text via an ElevenLabs model. This transcript is then passed to an Anthropic Claude LLM, along with the EHR data as context and then the LLM answers the original question based on the medical history of the patient. The textual answer is then converted back to speech via an ElevenLabs model again which is then played for the medical caregivers.

We previously carried out research for AI-powered anamnesis generation, but only with text data.