A research article entitled “Beyond Fine-Tuning: Leveraging Domain-Aware In-Context Learning with Large Language Models for Clinical Named Entity Recognition” has been published in the Journal of Biomedical Informatics (Volume 174, February 2026). The Journal of Biomedical Informatics is a leading international journal in the field of biomedical and clinical informatics that publish high-impact research at the intersection of medicine, data science, and artificial intelligence. Our study presents a comprehensive evaluation of large language model (LLM) based in-context learning (ICL) for clinical named entity recognition (NER) under realistic clinical constraints.

The study compared ICL with conventional encoder-based fine-tuning approaches using manually annotated real-world clinical notes. ICL configurations were optimized across task instructions, output formats, demonstration selection methods, sorting strategies, and demonstration pool sizes using the open-source LLaMA-3.3-70B model, while encoder-based fine-tuning was performed with RoBERTa-large as the strongest baseline.

The results showed that optimized demonstration selection played a major role in determining ICL performance, and improved macro F1 scores compared to random selection. In moderate-resource settings, ICL outperformed encoder-based fine-tuning and remained competitive across a wide range of labeled data sizes. Moreover, ICL demonstrated superior data efficiency and achieved substantial gains in cross-institutional transfer without requiring any parameter updates.

Overall, the study highlights the potential of domain-aware ICL as a flexible and data-efficient alternative to encoder-based fine-tuning for clinical NER. By enabling rapid adaptation through demonstration pools rather than retraining, the proposed approach offers a practical solution for real-world clinical natural language processing in dynamic and resource-limited healthcare environments.

This research was supported by grants from the MSIT (IITP), the Ministry of Health and Welfare (KHIDI), and the Ministry of Education (NRF) of the Republic of Korea.

Suhyun An is a Master’s student at the Department of Molecular Medicine and Biopharmaceutical Science (MMBS), Graduate School of Convergence Science, Seoul National University.

Her research centers around Medical Decision Making and AI-based drug discovery using large language models and deep neural networks, with a focus on adverse drug reactions and drug–drug interactions.

Suhyun An participated in CCADD during the 2024 Winter Internship Program and gained experience in real world data preprocessing, and machine learning. After the internship, Suhyun An continued as an undergraduate researcher at CCADD and participated in projects analyzing real-world data, including electronic medical records and clinical notes. These projects improved skills in natural language processing, applications of Large Language Models, and programming languages like R, Python, and Structured Query Language (SQL).

Yunha Noh is currently an undergraduate research student at CCADD, following her successful completion of the 2023 Winter Internship Program. She has received dual Bachelor’s degrees in Biotechnology and Computer Engineering from Duksung Women's University. Her primary research interests include applying machine learning to drug repurposing and leveraging real world data (RWD) to real world evidence in medical science.

Prof. Howard Lee and some CCADD members including Yoomin Jeon, Yoona Choi, and YeSol Hong visited AstraZeneca Korea (AZK) on December 3, 2020. AZ is a research-intensive pharmaceutical company based in the UK and has recently developed a vaccine against COVID-19. AZK invited Prof. Lee to a big data seminar series, and he talked about the role of medical big data including real world data and artificial intelligence (AI) in drug discovery and development. Prof. Lee also discussed how pharmaceutical companies could prepare themselves for the active and efficient adoption of big data and AI technologies for the day-to-day drug development activities. 

Yoona and YeSol are actually a former employee to AZK, in the departments of medical affairs and marketing, respectively. Thus, this visit was a kind of homecoming, and everyone welcomed them.The visit was also a great opportunity for CCADD to introduce its research fields and capabilities to pharmaceutical companies. This opportunity will enable CCADD for exploring many collaboration projects with those who are involved in actual drug development, which will certainly fill a gap for CCADD such that academic research is well aligned with the reality.     

Prof. Howard Lee proposed health insurance reimbursement for the combination therapy of DPP4 inhibitors and SGLT2 inhibitors, irrespective of their labels as approved by the Korea Ministry Food and Drug Safety (MFDS), at a special session organized by the Korean Diabetes Association (KDA), in association with its Spring Conference 2019, held on May 9-11, 2019, at the HICO, Kyeongju, Korea. Unlike other countries including the USA, EU, and Japan, MFDS in Korea has approved the combination therapy of SGLT2 inhibitors with other oral hypoglycemic agents, particularly with DPP4 inhibitors, only on the basis individual drugs, not on the therapeutic class. This unique regulatory approval has contributed to the delayed reimbursement of various combination therapies involving SGLT2 inhibitors since their approvals. 

The delayed reimbursement has caused a lot of debates not only between the pharmaceutical industry and the Health Insurance Review and Assessment Service or HIRA, but also among clinicians. With this background, the KDA asked Prof. Lee to share his perspectives from the points of drug regulatory science if the reimbursement for the combination therapy of DPP4 inhibitors and SGLT2 inhibitors on the basis of therapeutic class, not on the individual drug, has any scientific and clinical rationale. Prof. Lee, based on an intensive systematic literature review focusing on the pharmacokinetics, drug-drug interactions, clinical efficacy, and safety profiles of the combination treatment with DPP4 inhibitors and SGLT2 inhibitors, discussed that it is highly unlikely any combination of the two classes would interact in a clinically meaningful and significant way. Furthermore, Prof. Lee argued that it is very likely for every combination of a DPP4 inhibitor with an SGLT2 inhibitor to result in an additional, significant, and clinically meaningful decrease in HbA1c or glycated hemoglobin. However, since not every combination has been studied for its safety profiles although it is likely to be similar to those of monotherapy, Prof. Lee proposed to perform a real world data study under the auspice of KDA, sponsored by the pharmaceutical industry, to explore and confirm the safety and efficacy profiles of the combination therapy.      

Reflecting the interest in the reimbursement issue, the conference room was full of clinicians, industry representative, and journalists even though the session was presented early in the morning on Saturday. More detailed information on the session can be found here. 

Five CCADD members (Prof. Howard Lee, Dr. Soohyun Kim, Yoomin Jeon, Tae Chung, Serin Lee) have attended the Fall conference of The Korean Society of Medical Informatics (KOSMI) held at Chonbuk National University on November 22-23, 2018. The theme of the conference was 'Real World Data to optimize clinical trials', which reflects a growing role and value of big data and artificial intelligence for clinical trials. Over 700 people from academia, public institutions, and the pharmaceutical industry gathered to share their experiences, practices, insights, and perspectives on the more active use of data and information sciences in clinical trials.

Prof. Howard Lee spoke about AI-based patient recruitment in clinical trial in an invited session, entitled 'A Dimensionality Reduction Model to increase the Efficiency and Accuracy of Clinical Trial Feasibility Assessment using Electronic Medical Records(EMR)'. Prof. Lee emphasized the application of AI technology to clinical trial feasibility such as feature selection algorithms in machine learning to identify common sensitive eligibility features (cSEFs). 

Yoomin Jeon and Tae Chung presented a poster at the conference too. Yoomin's poster was about the actual application of what Prof. Lee introduced in his talk. Tae examined the extent of mapping of ICD-10 to MeSH to crosswalk and synchronize different terms collected from different data sources. (Analysis of agreement status between the diagnostic code of MeSH and ICD-10).

Meanwhile, Dr. Soohyun Kim led a tutorial session, where she taught several essential statistical methods for biomedical informatics research using R. 

For more details, you may want to contact Yoomin Jeon or Tae Chung.

Three CCADD members (Dr. Jeong An Gim, Yoomin Jeon, and Tae Kyu Chung) attended the Pharma AI & IoT 2018 conference, held in London, UK, from July 11 through 12, 2018. Many invited from pharmaceutical companies, software firms, and universities spoke at the conference. The conference provided a comprehensive outlook on the key issues about artificial intelligence (AI), which has been actively adopted at the pharma and healthcare industry. AI-augmented human decision-making was the central theme of the conference. Dr. Margaret McDonald from Pfizer told the audience machine learning or AI would increase efficiencies in regulatory decision-making. She added, "Unsupervised AI may integrate enormous amounts of data and those real world data support regulatory decision-making about drug safety." Likewise, many attendants emphasized the importance of AI in their field of specialization. During the conference, attendants discussed and shared best practices in furthering AI technology development specific to the pharma and healthcare industry through inspiring presentations and network opportunities.

After the conference, the CCADD attendants visited several landmarks in London such as the Greenwich Observatory, Tate Modern museum, and London Eye. Amazing weather and the scenic view of London was a big plus. Visiting London was a great refreshing and learning opportunity.