{"id":{"repo_id":"bu","oai_identifier":"oai:open.bu.edu:2144/48442"},"canonical_url":"https://search.ndltd.org/etd/bu/oai:open.bu.edu:2144/48442","repository":{"repo_id":"bu","name":"Boston University","base_url":"https://open.bu.edu/oai/request"},"display":{"title":"From coarse to fine-grained concept based discrimination for phrase detection","abstract":"Phrase Detection is a vision and language task where the goal is to determine if a phrase is relevant to an image and localize it, if applicable. The task has many important downstream applications such as Assistive Robotics where a Robot needs to detect/localize an object in an environment. However, training discriminative Phrase Detection models is difficult due to two main challenges 1) sparse training labels: ground truth regions are not exhaustively labeled with all applicable phrases which makes determining negative phrases challenging 2) the training distribution is heavily imbalanced; a small subgroup of the phrases constitute most of the training data. We address these problems through a novel coarse style discrimination: Negative Coarse Concepts (NCC). The method involves grouping visually coherent phrases (concepts) and using the concepts as negative samples which effectively expose the model to a wide and diverse portion of the distribution while minimizing the chance of false negatives. Furthermore, we supplement the coarse discrimination method with a fine grained module (FGM) which effectively discriminates between mutually exclusive tokens of diverse groups such as colors, sizes, etc. Finally, we combine these two novel modules into one model: CFCD-Net, which improves the state-of-the-art performance of Phrase Detection models on Flickr30K Entities and RefCOCO+ Datasets by 1.5-2 points. We further demonstrate how these improvements directly translate to improvements in the downstream task of Binary Image Selection (BISON).","abstract_html":"Phrase Detection is a vision and language task where the goal is to determine if a phrase is relevant to an image and localize it, if applicable. The task has many important downstream applications such as Assistive Robotics where a Robot needs to detect/localize an object in an environment. However, training discriminative Phrase Detection models is difficult due to two main challenges 1) sparse training labels: ground truth regions are not exhaustively labeled with all applicable phrases which makes determining negative phrases challenging 2) the training distribution is heavily imbalanced; a small subgroup of the phrases constitute most of the training data. We address these problems through a novel coarse style discrimination: Negative Coarse Concepts (NCC). The method involves grouping visually coherent phrases (concepts) and using the concepts as negative samples which effectively expose the model to a wide and diverse portion of the distribution while minimizing the chance of false negatives. Furthermore, we supplement the coarse discrimination method with a fine grained module (FGM) which effectively discriminates between mutually exclusive tokens of diverse groups such as colors, sizes, etc. Finally, we combine these two novel modules into one model: CFCD-Net, which improves the state-of-the-art performance of Phrase Detection models on Flickr30K Entities and RefCOCO+ Datasets by 1.5-2 points. We further demonstrate how these improvements directly translate to improvements in the downstream task of Binary Image Selection (BISON).","abstract_has_math":false,"creators":["Qraitem, Maan"],"institution":"Boston University","degree_name":"Master of Science","degree_level":"masters","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":[],"advisors":["Plummer, Bryan A."],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023","date_published":"2023","updated_at":"2026-11-18T19:48:23Z","subjects":["Computer science"],"languages":["en_US"],"rights":["Attribution 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by/4.0/"],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2144/48442","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"https://hdl.handle.net/2144/48442","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://open.bu.edu/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Aopen.bu.edu%3A2144%2F48442","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Plummer, Bryan A."]},{"key":"dc:creator","label":"Author","values":["Qraitem, Maan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2024-03-20T14:31:45Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2024-03-20T14:31:45Z"]},{"key":"dc:date.issued","label":"Date","values":["2023"]},{"key":"dc:date.updated","label":"Dc Date Updated","values":["2024-03-19T16:04:42Z"]},{"key":"dc:type","label":"Dc Type","values":["Thesis/Dissertation"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["masters"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Boston University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer science"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]},{"key":"dc:rights","label":"Dc Rights","values":["Attribution 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/2144/48442"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Phrase Detection is a vision and language task where the goal is to determine if a phrase is relevant to an image and localize it, if applicable. The task has many important downstream applications such as Assistive Robotics where a Robot needs to detect/localize an object in an environment. However, training discriminative Phrase Detection models is difficult due to two main challenges 1) sparse training labels: ground truth regions are not exhaustively labeled with all applicable phrases which makes determining negative phrases challenging 2) the training distribution is heavily imbalanced; a small subgroup of the phrases constitute most of the training data. We address these problems through a novel coarse style discrimination: Negative Coarse Concepts (NCC). The method involves grouping visually coherent phrases (concepts) and using the concepts as negative samples which effectively expose the model to a wide and diverse portion of the distribution while minimizing the chance of false negatives. Furthermore, we supplement the coarse discrimination method with a fine grained module (FGM) which effectively discriminates between mutually exclusive tokens of diverse groups such as colors, sizes, etc. Finally, we combine these two novel modules into one model: CFCD-Net, which improves the state-of-the-art performance of Phrase Detection models on Flickr30K Entities and RefCOCO+ Datasets by 1.5-2 points. We further demonstrate how these improvements directly translate to improvements in the downstream task of Binary Image Selection (BISON)."]},{"key":"dc:title","label":"Title","values":["From coarse to fine-grained concept based discrimination for phrase detection"]}]}],"canonical_facts":{"dc:contributor.advisor":["Plummer, Bryan A."],"dc:creator":["Qraitem, Maan"],"dc:date.accessioned":["2024-03-20T14:31:45Z"],"dc:date.available":["2024-03-20T14:31:45Z"],"dc:date.issued":["2023"],"dc:date.updated":["2024-03-19T16:04:42Z"],"dc:description.abstract":["Phrase Detection is a vision and language task where the goal is to determine if a phrase is relevant to an image and localize it, if applicable. The task has many important downstream applications such as Assistive Robotics where a Robot needs to detect/localize an object in an environment. However, training discriminative Phrase Detection models is difficult due to two main challenges 1) sparse training labels: ground truth regions are not exhaustively labeled with all applicable phrases which makes determining negative phrases challenging 2) the training distribution is heavily imbalanced; a small subgroup of the phrases constitute most of the training data. We address these problems through a novel coarse style discrimination: Negative Coarse Concepts (NCC). The method involves grouping visually coherent phrases (concepts) and using the concepts as negative samples which effectively expose the model to a wide and diverse portion of the distribution while minimizing the chance of false negatives. Furthermore, we supplement the coarse discrimination method with a fine grained module (FGM) which effectively discriminates between mutually exclusive tokens of diverse groups such as colors, sizes, etc. Finally, we combine these two novel modules into one model: CFCD-Net, which improves the state-of-the-art performance of Phrase Detection models on Flickr30K Entities and RefCOCO+ Datasets by 1.5-2 points. We further demonstrate how these improvements directly translate to improvements in the downstream task of Binary Image Selection (BISON)."],"dc:identifier.uri":["https://hdl.handle.net/2144/48442"],"dc:language.iso":["en_US"],"dc:rights":["Attribution 4.0 International"],"dc:rights.uri":["http://creativecommons.org/licenses/by/4.0/"],"dc:subject":["Computer science"],"dc:title":["From coarse to fine-grained concept based discrimination for phrase detection"],"dc:type":["Thesis/Dissertation"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["masters"],"thesis:degree_name":["Master of Science"],"thesis:institution_name":["Boston University"]},"updated_at":"2026-11-18T19:48:23Z"}