Gemini AI Hackathon

Hackathon Kick-Off 

We'll onboard you in the Gemini AI technology, newly released by Deep Mind, with Fabian Stehle, Data Scientist at New Native.

Introduction to Gemini AI

Join for the tech session with Shayak Sen, CTO of TruEra. We'll dive deeper in integration of TruLense into Gemini AI-based application.  

Tech Workshop with TruEra

Join experts Shayak from TruEra and Jerry Liu from LlamaIndex as they delve into the world of multimodal AI applications.

Building and Evaluating Multimodal GenAI Apps: RAG Triad is All you Need

Q&A Session with lablab.ai mentors on Discord

Submissions Deadline

A dynamic team of tech enthusiasts and creative problem solvers

Shav

Shavvimal

# Problem

From discussions, there seems to be some degree of diagnostic uncertainty in medicine, once the history is taken. Sometimes, when things are very busy, some investigations are missed out. Hospitals are short staffed as they are, and sometimes, even when things are busy, F1's, F2's and IMT1's need to run diagnoses by the registrar (who may not even be available) if there is any uncertainty. An AI could fit in here to validate diagnoses.

On a shift, doctors don't have time to go through every potential diagnosis. One Doctor Interviewed said "Sometimes I have no idea whats going on". He went on to say, "The amount of times I've taken a history and forgotten to check one thing and had to go back to check". This lack of efficiency is the pain point that Diagnosys AI aims to address.

# Proposed Solution

Use a MultiModal Retrieval-Augmented Generation (RAG) Stack, contexualised with best practice and resources (listed below) with an LLM (Gemini) to provide a comprehensive list of potential diagnoses, investigations and treatments. This will be a tool for Junior doctors, and will be input with the History taken from a patient. In the above process, it will fit in-between stage 3 and 4. In detail:

- Input (LLM be prompted with) History, Examinations done and (in the future) Patient Records
- Generate potential diagnoses and further investigation
- Specific things that could have been be missed out, as well as potential treatments
- Suggest potential examinations and what might be found in them i.e. What are red flags from suggested examinations to look out for
- Trust in the AI is a risk factor. This is mitigated by providing citations to the contextual information, such as Oxford Handbook of Clinical Medicine, Kumar & Clark's Clinical Medicine, etc.
- Can utilise multi=modal Models with Images, with rashes for example. These will refernece images from DermNet NZ

Diagnosys AI is designed to alleviate the challenges of diagnostic uncertainty and efficiency that junior doctors frequently encounter, aimed at the UK's junior doctors - F1's, F2's, and IMT1's (first stage of specialty training). 

"Shav team on Gemini AI Hackathon Hackathon"

Team Idea

Submission