{"id":{"repo_id":"usf","oai_identifier":"oai:digitalcommons.usf.edu:etd-11464"},"canonical_url":"https://search.ndltd.org/etd/usf/oai:digitalcommons.usf.edu:etd-11464","repository":{"repo_id":"usf","name":"University of South Florida","base_url":"https://digitalcommons.usf.edu/do/oai/"},"display":{"title":"Automatic Image-Based Nutritional Calculator App","abstract":"Nutrition plays a pivotal role in shaping an individuals’ health and quality of life, making the evaluation of dietary intake crucial for promoting healthier lifestyle choices. Various solutions, particularly mobile apps, have been developed to facilitate the process of dietary estimation. Accurate nutritional intake assessment relies on two key components: ingredient recognition and food portion estimation. For a mobile app to offer a comprehensive solution for automatic nutritional assessment, it must address both components. In this work, we focus on a mobile app pipeline: the semi-automatic pipeline which focuses on automatic food ingredient recognition. This pipeline integrates state-of-the-art models for ingredient recognition. We demonstrate that models such as BLIP-2 and GPT-3.5, when combined, can deliver precise ingredient recognition and great generalization capabilities. A fine-tuned GPT3.5 model calculates the nutritional value of meals based on the recognized ingredients and portion sizes provided by users manually. Since the implementation of this app focuses more on the automatic ingredient recognition the paper will primarily focus on that component. The performance and outcomes of the semi-automatic pipeline are benchmarked against existing mobile apps. Our findings reveal that the semi-automatic pipeline holds significant promise for generalization across different cuisines and enabling individuals to record their nutritional intake accurately and efficiently.","abstract_html":"Nutrition plays a pivotal role in shaping an individuals’ health and quality of life, making the evaluation of dietary intake crucial for promoting healthier lifestyle choices. Various solutions, particularly mobile apps, have been developed to facilitate the process of dietary estimation. Accurate nutritional intake assessment relies on two key components: ingredient recognition and food portion estimation. For a mobile app to offer a comprehensive solution for automatic nutritional assessment, it must address both components. In this work, we focus on a mobile app pipeline: the semi-automatic pipeline which focuses on automatic food ingredient recognition. This pipeline integrates state-of-the-art models for ingredient recognition. We demonstrate that models such as BLIP-2 and GPT-3.5, when combined, can deliver precise ingredient recognition and great generalization capabilities. A fine-tuned GPT3.5 model calculates the nutritional value of meals based on the recognized ingredients and portion sizes provided by users manually. Since the implementation of this app focuses more on the automatic ingredient recognition the paper will primarily focus on that component. The performance and outcomes of the semi-automatic pipeline are benchmarked against existing mobile apps. Our findings reveal that the semi-automatic pipeline holds significant promise for generalization across different cuisines and enabling individuals to record their nutritional intake accurately and efficiently.","abstract_has_math":false,"creators":["Cupa, Kejvi"],"institution":"University of South Florida","degree_name":"Master of Science (M.S.)","degree_level":"Thesis","degree_discipline":"Computer Science and Engineering","degree_department":null,"school":null,"contributors":["Yu Sun, Ph.D.","Hariom Yadav, Ph.D.","John Templeton, Ph.D.","Zhao Han, Ph.D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-03-15T07:00:00Z","date_published":"2024-03-15T07:00:00Z","updated_at":"2026-11-19T08:49:07Z","subjects":["Computer Vision","Ingredient recognition","Nutrition","Portion estimation","Transformers","Computer Sciences"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usf.edu/etd/10177","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usf.edu/etd/10177","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usf.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usf.edu%3Aetd-11464","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Yu Sun, Ph.D.","Hariom Yadav, Ph.D.","John Templeton, Ph.D.","Zhao Han, Ph.D."]},{"key":"dc:creator","label":"Author","values":["Cupa, Kejvi"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-03-15T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science and Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis","M.S."]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science (M.S.)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of South Florida"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer Vision","Ingredient recognition","Nutrition","Portion estimation","Transformers","Computer Sciences"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usf.edu/etd/10177"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Nutrition plays a pivotal role in shaping an individuals’ health and quality of life, making the evaluation of dietary intake crucial for promoting healthier lifestyle choices. Various solutions, particularly mobile apps, have been developed to facilitate the process of dietary estimation. Accurate nutritional intake assessment relies on two key components: ingredient recognition and food portion estimation. For a mobile app to offer a comprehensive solution for automatic nutritional assessment, it must address both components. In this work, we focus on a mobile app pipeline: the semi-automatic pipeline which focuses on automatic food ingredient recognition. This pipeline integrates state-of-the-art models for ingredient recognition. We demonstrate that models such as BLIP-2 and GPT-3.5, when combined, can deliver precise ingredient recognition and great generalization capabilities. A fine-tuned GPT3.5 model calculates the nutritional value of meals based on the recognized ingredients and portion sizes provided by users manually. Since the implementation of this app focuses more on the automatic ingredient recognition the paper will primarily focus on that component. The performance and outcomes of the semi-automatic pipeline are benchmarked against existing mobile apps. Our findings reveal that the semi-automatic pipeline holds significant promise for generalization across different cuisines and enabling individuals to record their nutritional intake accurately and efficiently."]},{"key":"dc:title","label":"Title","values":["Automatic Image-Based Nutritional Calculator App"]}]}],"canonical_facts":{"dc:contributor":["Yu Sun, Ph.D.","Hariom Yadav, Ph.D.","John Templeton, Ph.D.","Zhao Han, Ph.D."],"dc:creator":["Cupa, Kejvi"],"dc:date":["2024-03-15T07:00:00Z"],"dc:description":["Nutrition plays a pivotal role in shaping an individuals’ health and quality of life, making the evaluation of dietary intake crucial for promoting healthier lifestyle choices. Various solutions, particularly mobile apps, have been developed to facilitate the process of dietary estimation. Accurate nutritional intake assessment relies on two key components: ingredient recognition and food portion estimation. For a mobile app to offer a comprehensive solution for automatic nutritional assessment, it must address both components. In this work, we focus on a mobile app pipeline: the semi-automatic pipeline which focuses on automatic food ingredient recognition. This pipeline integrates state-of-the-art models for ingredient recognition. We demonstrate that models such as BLIP-2 and GPT-3.5, when combined, can deliver precise ingredient recognition and great generalization capabilities. A fine-tuned GPT3.5 model calculates the nutritional value of meals based on the recognized ingredients and portion sizes provided by users manually. Since the implementation of this app focuses more on the automatic ingredient recognition the paper will primarily focus on that component. The performance and outcomes of the semi-automatic pipeline are benchmarked against existing mobile apps. Our findings reveal that the semi-automatic pipeline holds significant promise for generalization across different cuisines and enabling individuals to record their nutritional intake accurately and efficiently."],"dc:identifier":["https://digitalcommons.usf.edu/etd/10177"],"dc:subject":["Computer Vision","Ingredient recognition","Nutrition","Portion estimation","Transformers","Computer Sciences"],"dc:title":["Automatic Image-Based Nutritional Calculator App"],"thesis:degree_discipline":["Computer Science and Engineering"],"thesis:degree_level":["Thesis","M.S."],"thesis:degree_name":["Master of Science (M.S.)"],"thesis:institution_name":["University of South Florida"]},"updated_at":"2026-11-19T08:49:07Z"}