Return to search

Medical domain knowledge in domain-agnostic generative AI

The text-guided diffusion model GLIDE (Guided Language to Image Diffusion for Generation and Editing) is the state of the art in text-to-image generative artificial intelligence (AI). GLIDE has rich representations, but medical applications of this model have not been systematically explored. If GLIDE had useful medical knowledge, it could be used for medical image analysis tasks, a domain in which AI systems are still highly engineered towards a single use-case. Here we show that the publicly available GLIDE model has reasonably strong representations of key topics in cancer research and oncology, in particular the general style of histopathology images and multiple facets of diseases, pathological processes and laboratory assays. However, GLIDE seems to lack useful representations of the style and content of radiology data. Our findings demonstrate that domain-agnostic generative AI models can learn relevant medical concepts without explicit training. Thus, GLIDE and similar models might be useful for medical image processing tasks in the future - particularly with additional domain-specific fine-tuning.

Identiferoai:union.ndltd.org:DRESDEN/oai:qucosa:de:qucosa:91780
Date31 May 2024
CreatorsKather, Jakob Nikolas, Ghaffari Laleh, Narmin, Foersch, Sebastian, Truhn, Daniel
PublisherMacmillan Publishers Limited
Source SetsHochschulschriftenserver (HSSS) der SLUB Dresden
LanguageEnglish
Detected LanguageEnglish
Typeinfo:eu-repo/semantics/publishedVersion, doc-type:article, info:eu-repo/semantics/article, doc-type:Text
Rightsinfo:eu-repo/semantics/openAccess
Relation2398-6352, 90, 10.1038/s41746-022-00634-5, info:eu-repo/grantAgreement/Bundesministerium für Gesundheit/DEEP LIVER/ZMVI1- 2520DAT111/, info:eu-repo/grantAgreement/Deutschen Krebshilfe/Max-Eder-Programm/#70113864/

Page generated in 0.0021 seconds