Beyond QA: A Multifaceted Evaluation of John Snow Labs’ Medical Chatbot
Chat with a Medical Generative AI that explains its answers, cites references, updates medical knowledge daily, and lets you add and choose knowledge bases: https://www.johnsnowlabs.com/medical-...
Install NLP Libraries https://www.johnsnowlabs.com/install/
Watch all NLP Summit 2023 sessions: https://www.nlpsummit.org/nlp-summit-...
With more than one million new biomedical research papers published per year, providing everyone with fast & reliable answers based on current scientific knowledge is paramount. This session by Veysel Kocaman, the Head of Data Science at John Snow Labs, presents a comprehensive evaluation of how well the company’s state-of-the-art Medical Chatbot meets that goal. The evaluated tasks include question answering on:
• Prebuilt knowledge bases such as Pubmed, MedArxiv, and Clinical Trials.
• User-specific documents, such as internal & confidential files.
• Structured data in relational databases, without users resorting to SQL or BI tools.
The evaluation covers these dimensions:
• Truthfulness: Ensuring answer fidelity, prioritizing trustworthy sources, and negating hallucinations.
• Accuracy: Delivering higher accuracy compared to general-purpose LLMs and vector databases with bespoke document splitting, reranking, post-filtering, and similarity search.
• Explain-ability: Citing resources while generating answers.
• Security: Supporting air-gapped deployment, functioning securely on-premise without requiring internet connectivity or external API calls.
• Expert Preference: A team of medical doctors applied a specialized methodology to evaluate the generated answers on relevance, style, consistency, and appropriateness.
• Recall: Medical doctors evaluated not only the generated answers, but also whether there exists additional research that can provide a more recent or complete answer.
• Latency: The speed of building & updating the body of knowledge, calculating embeddings, and running inference to answer user questions.
Gain insights into the capabilities, benchmarks, and applications of the Medical Chatbot, shaping the future of healthcare data retrieval and decision-making support.
Connect with us:
Our website: https://www.johnsnowlabs.com/
LinkedIn: / johnsnowlabs
Facebook: / johnsnowlabsinc
Watch video Beyond QA: A Multifaceted Evaluation of John Snow Labs’ Medical Chatbot online without registration, duration hours minute second in high quality. This video was added by user John Snow Labs 09 October 2023, don't forget to share it with your friends and acquaintances, it has been viewed on our site 552 once and liked it 8 people.