{"id":{"repo_id":"usf","oai_identifier":"oai:digitalcommons.usf.edu:etd-11010"},"canonical_url":"https://search.ndltd.org/etd/usf/oai:digitalcommons.usf.edu:etd-11010","repository":{"repo_id":"usf","name":"University of South Florida","base_url":"https://digitalcommons.usf.edu/do/oai/"},"display":{"title":"Generative Spatio-Temporal and Multimodal Analysis of Neonatal Pain","abstract":"Neonates can not express their pain like an adult person. Due to the lacking of proper muscle growth and inability to express non-verbally, it is difficult to understand their emotional status. In addition, if the neonates are under any treatment or left monitored after any major surgeries (post-operative), it is more difficult to understand their pain due to the side effect of medications and the caring system (i.e. intubated, masked face, covered body with blanket, etc.). In a clinical environment, usually, bedside nurses routinely observe the neonate and measure the pain status following any standard clinical pain scale. But current practice is intermittent and subjective.In this dissertation, we investigate developing an automated neonatal monitoring system that can replace the current manual system in a clinical environment. We explore the availability of current neonatal pain datasets and the challenges of neonatal pain data collection. We further collect and present a multimodal neonatal pain dataset, especially focusing on postoperative pain that mimics a real clinical environment. To develop the automated system, at first, we explore the effectiveness of individual modalities (i.e., facial expression, body movement, crying sound, vital signs) and propose novel spatio-temporal methods to analyze neonatal pain. Later, we develop a full multimodal system that is capable of using all modalities together to provide the pain signal. We also want to make sure that the automated system can effectively function in a real clinical environment. In a real clinical system, missing modality is very common due to different clinical factors (i.e., masked face, covered body, etc.) or algorithms' limitations (i.e., low light condition, side face view, etc.). We also propose a multimodal system that can effectively regenerate pseudo-features for missing modalities and continue assessing neonatal pain.","abstract_html":"Neonates can not express their pain like an adult person. Due to the lacking of proper muscle growth and inability to express non-verbally, it is difficult to understand their emotional status. In addition, if the neonates are under any treatment or left monitored after any major surgeries (post-operative), it is more difficult to understand their pain due to the side effect of medications and the caring system (i.e. intubated, masked face, covered body with blanket, etc.). In a clinical environment, usually, bedside nurses routinely observe the neonate and measure the pain status following any standard clinical pain scale. But current practice is intermittent and subjective.In this dissertation, we investigate developing an automated neonatal monitoring system that can replace the current manual system in a clinical environment. We explore the availability of current neonatal pain datasets and the challenges of neonatal pain data collection. We further collect and present a multimodal neonatal pain dataset, especially focusing on postoperative pain that mimics a real clinical environment. To develop the automated system, at first, we explore the effectiveness of individual modalities (i.e., facial expression, body movement, crying sound, vital signs) and propose novel spatio-temporal methods to analyze neonatal pain. Later, we develop a full multimodal system that is capable of using all modalities together to provide the pain signal. We also want to make sure that the automated system can effectively function in a real clinical environment. In a real clinical system, missing modality is very common due to different clinical factors (i.e., masked face, covered body, etc.) or algorithms&#x27; limitations (i.e., low light condition, side face view, etc.). We also propose a multimodal system that can effectively regenerate pseudo-features for missing modalities and continue assessing neonatal pain.","abstract_has_math":false,"creators":["Salekin, Md Sirajus"],"institution":"University of South Florida","degree_name":"Doctor of Philosophy (Ph.D.)","degree_level":"Dissertation","degree_discipline":"Computer Science and Engineering","degree_department":null,"school":null,"contributors":["Yu Y. Sun, Ph.D.","Dmitry Goldgof, Ph.D.","Ghada Zamzmi, Ph.D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-11-05T07:00:00Z","date_published":"2022-11-05T07:00:00Z","updated_at":"2026-11-19T08:48:23Z","subjects":["Computer Assisted Intervention","Computer Vision","Deep Learning","Machine Learning","Medical Application","Computer Engineering","Computer Sciences"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usf.edu/etd/9813","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usf.edu/etd/9813","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usf.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usf.edu%3Aetd-11010","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Yu Y. Sun, Ph.D.","Dmitry Goldgof, Ph.D.","Ghada Zamzmi, Ph.D."]},{"key":"dc:creator","label":"Author","values":["Salekin, Md Sirajus"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-11-05T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science and Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation","Ph.D."]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy (Ph.D.)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of South Florida"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer Assisted Intervention","Computer Vision","Deep Learning","Machine Learning","Medical Application","Computer Engineering","Computer Sciences"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usf.edu/etd/9813"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Neonates can not express their pain like an adult person. Due to the lacking of proper muscle growth and inability to express non-verbally, it is difficult to understand their emotional status. In addition, if the neonates are under any treatment or left monitored after any major surgeries (post-operative), it is more difficult to understand their pain due to the side effect of medications and the caring system (i.e. intubated, masked face, covered body with blanket, etc.). In a clinical environment, usually, bedside nurses routinely observe the neonate and measure the pain status following any standard clinical pain scale. But current practice is intermittent and subjective.In this dissertation, we investigate developing an automated neonatal monitoring system that can replace the current manual system in a clinical environment. We explore the availability of current neonatal pain datasets and the challenges of neonatal pain data collection. We further collect and present a multimodal neonatal pain dataset, especially focusing on postoperative pain that mimics a real clinical environment. To develop the automated system, at first, we explore the effectiveness of individual modalities (i.e., facial expression, body movement, crying sound, vital signs) and propose novel spatio-temporal methods to analyze neonatal pain. Later, we develop a full multimodal system that is capable of using all modalities together to provide the pain signal. We also want to make sure that the automated system can effectively function in a real clinical environment. In a real clinical system, missing modality is very common due to different clinical factors (i.e., masked face, covered body, etc.) or algorithms' limitations (i.e., low light condition, side face view, etc.). We also propose a multimodal system that can effectively regenerate pseudo-features for missing modalities and continue assessing neonatal pain."]},{"key":"dc:title","label":"Title","values":["Generative Spatio-Temporal and Multimodal Analysis of Neonatal Pain"]}]}],"canonical_facts":{"dc:contributor":["Yu Y. Sun, Ph.D.","Dmitry Goldgof, Ph.D.","Ghada Zamzmi, Ph.D."],"dc:creator":["Salekin, Md Sirajus"],"dc:date":["2022-11-05T07:00:00Z"],"dc:description":["Neonates can not express their pain like an adult person. Due to the lacking of proper muscle growth and inability to express non-verbally, it is difficult to understand their emotional status. In addition, if the neonates are under any treatment or left monitored after any major surgeries (post-operative), it is more difficult to understand their pain due to the side effect of medications and the caring system (i.e. intubated, masked face, covered body with blanket, etc.). In a clinical environment, usually, bedside nurses routinely observe the neonate and measure the pain status following any standard clinical pain scale. But current practice is intermittent and subjective.In this dissertation, we investigate developing an automated neonatal monitoring system that can replace the current manual system in a clinical environment. We explore the availability of current neonatal pain datasets and the challenges of neonatal pain data collection. We further collect and present a multimodal neonatal pain dataset, especially focusing on postoperative pain that mimics a real clinical environment. To develop the automated system, at first, we explore the effectiveness of individual modalities (i.e., facial expression, body movement, crying sound, vital signs) and propose novel spatio-temporal methods to analyze neonatal pain. Later, we develop a full multimodal system that is capable of using all modalities together to provide the pain signal. We also want to make sure that the automated system can effectively function in a real clinical environment. In a real clinical system, missing modality is very common due to different clinical factors (i.e., masked face, covered body, etc.) or algorithms' limitations (i.e., low light condition, side face view, etc.). We also propose a multimodal system that can effectively regenerate pseudo-features for missing modalities and continue assessing neonatal pain."],"dc:identifier":["https://digitalcommons.usf.edu/etd/9813"],"dc:subject":["Computer Assisted Intervention","Computer Vision","Deep Learning","Machine Learning","Medical Application","Computer Engineering","Computer Sciences"],"dc:title":["Generative Spatio-Temporal and Multimodal Analysis of Neonatal Pain"],"thesis:degree_discipline":["Computer Science and Engineering"],"thesis:degree_level":["Dissertation","Ph.D."],"thesis:degree_name":["Doctor of Philosophy (Ph.D.)"],"thesis:institution_name":["University of South Florida"]},"updated_at":"2026-11-19T08:48:23Z"}