Beyond QA: A Multifaceted Evaluation of John Snow Labs’ Medical Chatbot
Chat with a Medical Generative AI that explains its answers, cites references, updates medical knowledge daily, and lets you add and choose knowledge bases: https://www.johnsnowlabs.com/medical-...
Install NLP Libraries https://www.johnsnowlabs.com/install/
Watch all NLP Summit 2023 sessions: https://www.nlpsummit.org/nlp-summit-...
With more than one million new biomedical research papers published per year, providing everyone with fast & reliable answers based on current scientific knowledge is paramount. This session by Veysel Kocaman, the Head of Data Science at John Snow Labs, presents a comprehensive evaluation of how well the company’s state-of-the-art Medical Chatbot meets that goal. The evaluated tasks include question answering on:
• Prebuilt knowledge bases such as Pubmed, MedArxiv, and Clinical Trials.
• User-specific documents, such as internal & confidential files.
• Structured data in relational databases, without users resorting to SQL or BI tools.
The evaluation covers these dimensions:
• Truthfulness: Ensuring answer fidelity, prioritizing trustworthy sources, and negating hallucinations.
• Accuracy: Delivering higher accuracy compared to general-purpose LLMs and vector databases with bespoke document splitting, reranking, post-filtering, and similarity search.
• Explain-ability: Citing resources while generating answers.
• Security: Supporting air-gapped deployment, functioning securely on-premise without requiring internet connectivity or external API calls.
• Expert Preference: A team of medical doctors applied a specialized methodology to evaluate the generated answers on relevance, style, consistency, and appropriateness.
• Recall: Medical doctors evaluated not only the generated answers, but also whether there exists additional research that can provide a more recent or complete answer.
• Latency: The speed of building & updating the body of knowledge, calculating embeddings, and running inference to answer user questions.
Gain insights into the capabilities, benchmarks, and applications of the Medical Chatbot, shaping the future of healthcare data retrieval and decision-making support.
Connect with us:
Our website: https://www.johnsnowlabs.com/
LinkedIn: / johnsnowlabs
Facebook: / johnsnowlabsinc
Смотрите видео Beyond QA: A Multifaceted Evaluation of John Snow Labs’ Medical Chatbot онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь John Snow Labs 09 Октябрь 2023, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 552 раз и оно понравилось 8 людям.