|
|
|
|
|
|
|
The event was especially meaningful because David Seung U Lee and Yu Jin Kim delivered oral presentations there.
The theme of this year’s conference was "From Vision to Value: AI's Role in Shaping Modern Healthcare." It focused on how artificial intelligence (AI) is reshaping the future of healthcare and transforming itself from a mere vision into tangible value. The discussions delved deeply into AI's roles in this transformative process. The conference provided a platform to share the latest research achievements and explored how AI contributes to patient care and system improvement in real-world medical settings.
As for the oral presentations by our members, David Seung U Lee presented the study entitled “Improving Few-shot Performance of Large Language Models to Extract Clinical Information from Real World Clinical Notes.” His research addressed the challenges of extracting information from unstructured and noisy clinical notes using large language models (LLMs). The study proposed a cost-effective strategy to enhance LLM performance by optimizing factors such as model size, example selection, and arrangement strategies. Using 2,110 annotated clinical notes from Seoul National University Hospital, the study demonstrated the trade-offs between prompt design and performance, with optimal results achieved using 15 examples and BM25-based example selection. Additionally, the research explored the impact of data drift on LLM robustness, offering insights into devising better prompting strategies for real-world applications.
Aligned with the focus of the conference, Yu Jin Kim also presented the study entitled “Adverse Event Extraction from EMR-Based Real-World Data in Pediatric Acute Lymphoblastic Leukemia Patients Receiving Chemotherapy.” The study introduced a sequential strategy for efficiently extracting adverse events (AEs) from electronic medical records (EMR), combining structured data analysis with minimal manual review. The method effectively captured clinically relevant AEs with reasonable accuracy, demonstrating its potential to enhance real-world data utilization for evaluating treatment safety in pediatric patients with acute lymphoblastic leukemia.
|
|
|
|
|
The conference was organized around the theme "Clinical Pharmacology in Innovative Therapeutics" and featured various sessions on latest topics, including Live Biotherapeutic Products, ICT in Clinical Trials, Voice from the Clinical Trial Field, Innovative Sites for Early Phase Clinical Trials, Omics with Clinical Pharmacology, and GCP Updates.
In the Live Biotherapeutic Products session, insights were shared on the discovery, CMC, non-clinical, and clinical studies of microbiome therapeutics, a promising new treatment modality. The ICT in Clinical Trials session addressed the latest developments in Real-World Evidence-based clinical trials and Decentralized Clinical Trials.The Voice from the Clinical Trial Field session brought together experts from institutions, SMOs, CROs, and clinical trial sites to share practical insights.
More importantly, CCADD members introduced their works through poster presentations. David Seung U Lee, as the lead presenter, introduced the study "Improving Few-Shot Performance of Large Language Models to Extract Clinical Information from Real-World Clinical Notes." This research evaluated how large language models (LLMs) like Llama-3.18 perform on clinical information extraction tasks using annotated notes from Seoul National University Hospital. The study demonstrated the importance of optimized prompting strategies and entity distributions, achieving high accuracy and robustness. These findings highlight the potential of LLMs in advancing clinical text analysis for real-world applications. Meanwhile Jung-Hyu Won and Yun Jin Choi presented their study titled "Comparing the Therapeutic Efficacy of Tisagenlecleucel and Salvage Chemotherapy in Pediatric Relapsed/Refractory B-Cell Acute Lymphoblastic Leukemia: A MAIC Approach." This research retrospectively analyzed outcomes using the Matching-Adjusted Indirect Comparison (MAIC) method to evaluate Tisagenlecleucel, a CD19-targeting CAR-T cell therapy, against salvage/reinduction chemotherapy. Results showed that while no significant differences were observed in overall remission rates (ORR) or overall survival (OS), Tisagenlecleucel achieved a significantly higher complete remission (CR) rate. These findings highlight the potential of CAR-T therapy as an effective option for pediatric relapsed/refractory B-ALL.
Through their participation in this conference, CCADD members gained a deeper understanding of the latest advancements in clinical pharmacology and therapeutics. |
|
|
|
|
The conference featured several noteworthy sessions. Discussions on drug adverse reactions focused on improving patient safety through better detection and reporting mechanisms. The DUR platform sessions explored advancements in drug utilization review systems that help prevent medication errors. Presentations on natural language processing (NLP) showcased its application in extracting meaningful data from unstructured clinical notes, enhancing decision-making in clinical settings. Meanwhile, sessions on the limitations of multi-center Common Data Model (CDM) research highlighted challenges such as data standardization and integration across different institutions.
Lively discussions ensued on these topics, with various studies showcasing the innovative use of medical data to improve healthcare outcomes. The conference emphasized the dynamic and evolving nature of medical informatics, driven by continuous advancements in technology. A key theme throughout the conference was the importance of collaboration. Speakers and participants highlighted how interdisciplinary teamwork and partnerships between medical institutions, academic bodies, and industry are crucial for overcoming challenges and driving innovation in medical informatics.
This achievement stems from a collaborative effort involving CCADD, the Medical Informatics Lab of the College of Medicine at The Catholic University of Korea, led by Prof. In Young Choi, and hemato-oncologists from both hospitals. Dr. Choi's presentation provided an in-depth evaluation of the algorithm developed to extract information automatically from CDWs into the registry, highlighting its practical applications and significant implications for pediatric ALL research.
The session chair lauded Dr. Choi’s presentation, commending its excellence and the potential impact of this innovative work on future medical research and patient care. |
|
|
|
|
Professor Howard and David attended the
84th Scientific Sessions of the American Diabetes Association (ADA). It was
held from June 21 to 24 in Orlando, Florida, hosting experts in diabetes research
from all over the world. The event brought together leading scientists,
healthcare professionals, and stakeholders to discuss critical issues,
including advancements in diabetes technology. It also showcased cutting-edge
research and innovations in diabetes treatment and management, ranging from obesity
and GLP-1 receptor agonists to autoantibody testing and early diabetes detection.
The event also provided a memorable venue for
Prof. Howard and David, where they presented their work on “Developing Machine Learning
Model for Predicting Acute Coronary Syndrome in Type 2 Diabetes Mellitus Patients
through Substitution of Propensity Scores for Binary Variables”. During the poster
presentation session, questions were asked and answered, ideas were exchanged, and
constructive comments were offered. The event, along with the opportunity to present
the work, paved the way for improving the study and provided a valuable experience
to learn from world-renowned researchers.
|
|
|
|
|
The EACL conference, hosted in Malta from March 17th to 22nd, was not just a venue for presenting their findings; it also provided enriching learning opportunities. The event covered a wide range of topics, from the challenges posed by large language models (LLMs), as seen in studies like "Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs," to their innovative applications in healthcare, as exemplified by "Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study." These discussions spurred our team to consider how we could integrate these insights into our research.
Additionally, the conference featured thought-provoking keynote speeches that challenged prevailing views on language models. Dr. Lapata, in particular, argued against the notion that LLMs are a solution-for-all. Instead, she advocated for a modular approach, employing specialized, smaller language models to efficiently and sustainably address various natural language processing tasks, such as summarizing and crosslingual semantic parsing.
The main conference, spanning three days, was followed by workshops that offered practical learning experiences. "Computational Approaches to Language Data Pseudonymization" was notably relevant to our work, focusing on the anonymization of sensitive text data, a critical consideration when dealing with clinical texts. A standout session, "When Is a Name Sensitive? Eponyms in Clinical Text and Implications for De-Identification," delved into how transformer-based models could minimize false positives in de-identifying clinical texts, a concern often raised by the presence of eponyms, and it got us thinking deeply about how to properly deidentify personal health information in clinical notes.
All in all, the week-long EACL conference served dual purposes: it was an avenue for sharing our contributions with the NLP community and a fertile ground for acquiring new knowledge. Such opportunities are invaluable, affirming our progress and direction in the field.
|
|
|
|
|
We are pleased to announce that our research paper, "CReSE: Benchmark Data and Automatic Evaluation Framework for Recommending Eligibility Criteria from Clinical Trial Information", authored by Dr. Siun Kim (PhD graduate of CCADD), Jung-Hyun Won, David Lee (both current CCADD PhD students), Renqian Luo, Lijun Wu, Tao Qin (collaborator researchers at Microsoft Research Asia), and Prof. Howard Lee, has been accepted for presentation at the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2024).
The paper introduces the CReSE model (Contrastive learning and Rephrasing-based and Clinical Relevance-preserving Sentence Embedding model), a new framework aimed at improving the recommendation of eligibility criteria (EC) in clinical trials. In addition, CCADD introduces a comprehensive benchmark dataset, consisting of over 3.2 million high-quality EC-title pairs from approximately 270,000 clinical trials listed on ClinicalTrials.gov.
The acceptance of our paper at EACL 2024 is a significant milestone for CCADD, demonstrating our progress in reshaping drug development through the integration of computational linguistics and interdisciplinary innovation. |
|
|
|
|
On
December 27th, 2023, CCADD held the year-end workshop at Front1, a startup hub at the heart of Seoul, where PMACH is based.
This year, Prof. Howard Lee and the members of CCADD were joined by PMATCH and
Dr. Siun Kim. The workshop went on in a dynamic atmosphere, welcoming both
Prof. Howard’s part-time student and the interns for the upcoming internship program.
After
a brief ice-breaking time, the main session started with a recap of the
accomplishments of 2023, reflecting on the overall trajectory. Prof. Howard
commended CCADD’s continuous growth in both research and publications. As
preparation for 2024, the following session outlined a roadmap, encompassing
detailed research ideas and plans for conference to attend in 2024.
The
spotlight then turned to CCADD interns. Three passionate interns—Tang Chao, Seorin Choi, and Yunha Noh—who will
join the 2024 winter internship program introduced themselves through "Who
am I" presentations. Following this, Yujin expressed her aspirations for
the path ahead as a new CCADD graduate research student. The workshop proceeded
with a lecture on the challenges faced by postdoctoral researchers and
strategies for maintaining a healthy mindset among graduate students.
Dr.
Siun Kim took a special session to share insights into his post-doctoral
journey following graduation. Delivering a lecture on the trending topic of
"LLMs in medicine and drug development", he provided valuable
perspectives and briefly introduced his ongoing research projects. He candidly
shared reflections on the first-year post doc., emphasizing the professional
commitment required in the field of research.
Finally,
PMATCH summarized their achievements in 2023 and anticipated the upcoming
year's developments. With the successful launch of the 'BATTERY' app and a
triumphant debut at CES® 2023, PMATCH envisions a promising future with new team members.
Yoomin Jeon, the Chief Technology Officer of PMATCH, expressed her commitment
to meeting user’s needs as a must-have app.
The
workshop concluded with a keynote address by Prof. Howard. Introducing new trends
in 2024, he emphasized the significance of adjusting our research focus and
domain knowledge to take another leap forward. Prof. Howard’s central message
highlighted 2024 as a pivotal year for CCADD's rebound and PMATCH's inaugural
year of growth.
|
|
|
|
|
Yoona Choi and Jung-Hyun Won attended the 65th Annual Meeting & Exposition of the American Society of Hematology (ASH), which took place from December 9 to 12, 2023, in San Diego, USA. Annually, the ASH conference brings together healthcare professionals, researchers, clinicians, and academics from around the globe to explore advancements in hematology. This prestigious event not only focuses on the study, diagnosis, treatment, and prevention of blood disorders but also features a variety of social events and opportunities for collaboration and the sharing of insights among experts in the field. Official photos from ASH are available here. 
At the 'Outcomes Research—Lymphoid Malignancies: Poster II' session, Yoona Choi, serving as the first author and representing the entire research team, presented a study titled 'Evaluating the Effectiveness and Safety of Prephase Steroid Treatment before Remission Induction Chemotherapy in Pediatric Acute Lymphoblastic Leukemia: A Common Data Model-Based Real-World Data Approach.' This study investigated steroid pretreatment in pediatric acute lymphoblastic leukemia (ALL) patients undergoing remission induction chemotherapy (RIC), with the aim of reducing complications such as tumor lysis syndrome (TLS). Analyzing data from Seoul National University Hospital and Catholic University Seoul St. Mary’s Hospital, the study compared patients who received steroid pretreatment at least seven days before RIC with those who did not. The findings indicated a significant reduction in TLS incidence in the pretreated group. While overall survival rates were higher in this group, the difference was not statistically significant. These results provide important real-world evidence for the effectiveness of steroid pretreatment in reducing TLS risk in pediatric ALL patients.
This study was part of a larger project, funded by a grant from the Ministry of Health and Welfare, aiming to develop a disease registry database for pediatric ALL using real-world data in the clinical data warehouse (CDW) of Seoul National University Hospital and Catholic University Seoul St. Mary’s Hospital, thereby generating real-world evidence for pediatric ALL. 
|
|
|
|
|
Prof. Howard Lee gave a talk at the 17th JSCPT-KSCPT Joint Symposium, Kobe, Japan, on December 15, 2023. In his presentation entitled 'Real-World Data Analysis as a Clinical Pharmacologist: Experience and Some Thoughts', Prof. Lee. introduce the audience to the RWD research works CCADD has performed for the past 3 or so years, all of which have been published in reputable journals or under review shown below. Prof. Lee stressed, "With RWD landscape fast growing, clinical pharmacologists should leverage the potentials of RWD/RWE in drug development by furthering its scientific innovation!"
Prof. Lee's speech was well received by the audience followed by a welcome reception by JSCPT colleagues, whom he and other delegates have not seen in person for the past three years because of the COVID-19 pandemic. The night was full of talking, drinking, and singing! Both teams promised to meet again next year in Jeju, Korea, for the Spring Conference of 2024 KSCPT.
|
|
|
|
|
CCADD attended the 2023 Fall Conference of the Korean Society of Medical Informatics (KoSMI) held at HIP(Healthcare Innovation Park) of Seoul National University Bundang Hospital, South Korea. The conference spanned three days from November 29th to December 1st, both online and offline. Aligned with the theme "Beyond Data: Actionable Health AI", the conference featured 27 symposiums covering various topics, from artificial intelligence in the healthcare to standardization of medical data and digital health. The first day was conducted online. Particularly, ”Symposium 6: Development of Predictive Models for Cardiovascular Diseases Using Medical Big Data” was noteworthy, where Prof. Howard Lee served as the session chair and speaker, attracting significant attention from the audience with the presentation on ”Improving the Performance of Machine Learning Models for Predicting Acute Coronary Syndrome in Type 2 Diabetes Patients through Substitution of Propensity Scores for Binary Variables”. Also, throughout the conference, the poster presentation was led by David Seung U Lee. On the second day, each symposium delved into topics of medical data and artificial intelligence applications to the healthcare domain. Themes included "Strategies for the Activation and Commercialization of AI-Based Drug Development" and "Smart Curation of Medical Data: Approaches for Preparing for the Utilization of Medical AI." This encompassed insights into the industry's latest strategies for drug target exploration. In Symposium 9, Dr. Siun Kim delivered a presentation on ”Addressing the challenges of data scarcity and bias of AI-based drug development”. The discourse extensively covered specific manifestations of data scarcity and proposed strategies for mitigation within drug development.
During the final day, the symposiums provided an opportunity to approach the trends in the utilization and data processing methods of developing LLM. Participants delved into the practical approaches for utilizing Real-World Data (RWD) in regulatory decision-making, replicating Randomized Controlled Trials (RCT), and employing common data models. CCADD members exchanged ideas with fellow researchers studying similar research topics, offering a valuable opportunity to comprehend ongoing research trends. It was an insightful symposium for healthcare information researchers to share and discuss the latest trends in research. |
|
|
|
|
Prof. Howard Lee gave a talk at an invited symposium in association with the Annual Conference of the Korean Society of Medical Informatics (KoSMI), held both online and offline on November 29, 2023. The title of his presentation was ‘Performance Increase of the Machine Learning Model to Predict Acute Coronary Syndrome in Patients with Type 2 Diabetes Mellitus by Replacing Binary Variables with Propesnsity Score’. This presentation was a culmination of a research project with the same title supported by the Seokcheon Foundation, 2023, where the current and former CCADD graduate students, namely David Seung U Lee, Jung-Hyun Won, and YeSol Hong collectively contributed by proposing a new novel way to address the falseness of binary feature in the electronic medical records. Prof. Lee's speech was well received by the audience evidenced by 6 follow-up questions! The more detailed methodological aspect of Prof. Lee' talk was presented in an accepted poster led by David Seung U Lee.
|
|
HOME
