{"id":{"repo_id":"msu","oai_identifier":"oai:d.lib.msu.edu:etd_52554"},"canonical_url":"https://search.ndltd.org/etd/msu/oai:d.lib.msu.edu:etd_52554","repository":{"repo_id":"msu","name":"Michigan State University","base_url":"https://d.lib.msu.edu/oai"},"display":{"title":"Face Modeling Under Diverse Illuminations","abstract":"Thesis (Ph.D.)--Michigan State University. Computer Science - Doctor of Philosophy, 2025","abstract_html":"Thesis (Ph.D.)--Michigan State University. Computer Science - Doctor of Philosophy, 2025","abstract_has_math":false,"creators":["Hou, Andrew"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":["Liu, Xiaoming","Tong, Yiying","Ross, Arun","Morris, Daniel","Sarkis, Michel"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025","date_published":"2025","updated_at":"2026-11-18T18:59:06Z","subjects":["Computer science"],"languages":["eng"],"rights":["In Copyright"],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["etd:52554","local:Hou_grad.msu_0128D_21351"],"render_values":[{"text":"etd:52554","href":null,"code":true},{"text":"local:Hou_grad.msu_0128D_21351","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://doi.org/doi:10.25335/daan-ck96","outbound_label":"DOI","outbound_source":"dc:identifier","record_url":"https://doi.org/doi:10.25335/daan-ck96","record_label":"Repository record","record_kind":"doi","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://d.lib.msu.edu/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Ad.lib.msu.edu%3Aetd_52554","prefix":"oai_dc"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Liu, Xiaoming","Tong, Yiying","Ross, Arun","Morris, Daniel","Sarkis, Michel"]},{"key":"dc:creator","label":"Author","values":["Hou, Andrew"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025"]},{"key":"dc:relation","label":"Dc Relation","values":["Electronic Theses & Dissertations"]},{"key":"dc:type","label":"Dc Type","values":["text","Electronic dissertations"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer science"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["In Copyright"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["etd:52554","local:Hou_grad.msu_0128D_21351","https://doi.org/doi:10.25335/daan-ck96"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Thesis (Ph.D.)--Michigan State University. Computer Science - Doctor of Philosophy, 2025","3D face modeling is a longstanding problem within the computer vision and computer graphicscommunities with widespread applications in AR/VR, movie production, and entertainment that involves a vast array of problems including modeling facial appearance, shape, expressions, lighting, and pose. One prominent issue in this domain is the inability to properly model non-Lambertian lighting effects, including hard shadows caused by directional lights. To this end, we propose two methods to better handle facial hard shadows. The first leverages physically-inspired supervision along the hard shadow boundaries to encourage the model to focus on properly synthesizing these regions. The second method pushes one step further and directly leverages the 3D face to physically constrain the generated hard shadows and guarantees shadow geometric consistency with respect to the face. Beyond facial shadows, which follow the geometric structure of the face, foreign shadows caused by external objects (e.g. hats, sunglasses, and leaves) are another consideration when modeling in-the-wild faces. We thus propose a self-supervised foreign shadow removal method that formulates the problem as an image-inpainting problem and properly removes foreign shadows while preserving the subject’s facial details. However, all of these proposed methods do not have full control over facial illumination. For example, our shadow removal method, while able to completely remove shadows, is not able to control the degree of shadow softening and cannot perform the converse shadow intensifying operation. Moreover, our directional relighting methods represent light as a direction and are not able to control light size, rendering them unable to model the associated lighting effects. We thus propose COMPOSE, the first single-image portrait shadow editing method with full control over all shadow attributes, including shadow intensity, shape, and position that also preserves other attributes of the portrait’s original environmental lighting. This is achieved by decomposing the problem into an environment-preserving diffuse image estimation and subsequently estimating a shadowed image where the shadow parameters are varied. By performing image compositing between the diffuse and shadowed images, COMPOSE precisely controls all shadow attributes during illumination editing. Moving beyond illumination editing, we investigate the remainder of the face modeling problem in editing appearance, shape, expressions, and pose. We first propose INFAMOUS-NeRF, an implicit face modeling method that introduces hypernetworks to estimate subject-specific model weights, thus alleviating the burden of the NeRF MLP to encode all subject information in a single shared set of weights. While improving representation power with subject-specific weights, INFAMOUS-NeRF also maintains editability by encouraging subjects with similar attributes (e.g. same expression) to share the same latent code for those attributes, which maintains semantic alignment of latent spaces. Finally, we investigate the tradeoff between representation power and efficiency. Gaussian splatting methods attempt to resolve this tradeoff by offering both high representation power and efficiency with real-time rendering. However, their efficiency nonetheless depends on the number of rasterized gaussians. Moreover, the time needed to optimize/prepare the avatar (enrollment time) is often high. We therefore propose EGGHead: a novel face modeling method that, given a fixed gaussian budget, efficiently assigns gaussians to different semantic regions of the avatar. Our algorithm reduces the total number of gaussians needed while minimizing the impact on representation power. Our pipeline operates in a single forward pass, and achieves SoTA reconstruction quality, novel view synthesis quality, and enrollment/rendering speeds compared to other single-image face modeling methods.","Description based on online resource. Title from PDF t.p. (Michigan State University Fedora Repository, viewed ).","Includes bibliographical references."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf","1 online resource (179 p.) : ill."]},{"key":"dc:title","label":"Title","values":["Face Modeling Under Diverse Illuminations"]}]}],"canonical_facts":{"dc:contributor":["Liu, Xiaoming","Tong, Yiying","Ross, Arun","Morris, Daniel","Sarkis, Michel"],"dc:creator":["Hou, Andrew"],"dc:date":["2025"],"dc:description":["Thesis (Ph.D.)--Michigan State University. Computer Science - Doctor of Philosophy, 2025","3D face modeling is a longstanding problem within the computer vision and computer graphicscommunities with widespread applications in AR/VR, movie production, and entertainment that involves a vast array of problems including modeling facial appearance, shape, expressions, lighting, and pose. One prominent issue in this domain is the inability to properly model non-Lambertian lighting effects, including hard shadows caused by directional lights. To this end, we propose two methods to better handle facial hard shadows. The first leverages physically-inspired supervision along the hard shadow boundaries to encourage the model to focus on properly synthesizing these regions. The second method pushes one step further and directly leverages the 3D face to physically constrain the generated hard shadows and guarantees shadow geometric consistency with respect to the face. Beyond facial shadows, which follow the geometric structure of the face, foreign shadows caused by external objects (e.g. hats, sunglasses, and leaves) are another consideration when modeling in-the-wild faces. We thus propose a self-supervised foreign shadow removal method that formulates the problem as an image-inpainting problem and properly removes foreign shadows while preserving the subject’s facial details. However, all of these proposed methods do not have full control over facial illumination. For example, our shadow removal method, while able to completely remove shadows, is not able to control the degree of shadow softening and cannot perform the converse shadow intensifying operation. Moreover, our directional relighting methods represent light as a direction and are not able to control light size, rendering them unable to model the associated lighting effects. We thus propose COMPOSE, the first single-image portrait shadow editing method with full control over all shadow attributes, including shadow intensity, shape, and position that also preserves other attributes of the portrait’s original environmental lighting. This is achieved by decomposing the problem into an environment-preserving diffuse image estimation and subsequently estimating a shadowed image where the shadow parameters are varied. By performing image compositing between the diffuse and shadowed images, COMPOSE precisely controls all shadow attributes during illumination editing. Moving beyond illumination editing, we investigate the remainder of the face modeling problem in editing appearance, shape, expressions, and pose. We first propose INFAMOUS-NeRF, an implicit face modeling method that introduces hypernetworks to estimate subject-specific model weights, thus alleviating the burden of the NeRF MLP to encode all subject information in a single shared set of weights. While improving representation power with subject-specific weights, INFAMOUS-NeRF also maintains editability by encouraging subjects with similar attributes (e.g. same expression) to share the same latent code for those attributes, which maintains semantic alignment of latent spaces. Finally, we investigate the tradeoff between representation power and efficiency. Gaussian splatting methods attempt to resolve this tradeoff by offering both high representation power and efficiency with real-time rendering. However, their efficiency nonetheless depends on the number of rasterized gaussians. Moreover, the time needed to optimize/prepare the avatar (enrollment time) is often high. We therefore propose EGGHead: a novel face modeling method that, given a fixed gaussian budget, efficiently assigns gaussians to different semantic regions of the avatar. Our algorithm reduces the total number of gaussians needed while minimizing the impact on representation power. Our pipeline operates in a single forward pass, and achieves SoTA reconstruction quality, novel view synthesis quality, and enrollment/rendering speeds compared to other single-image face modeling methods.","Description based on online resource. Title from PDF t.p. (Michigan State University Fedora Repository, viewed ).","Includes bibliographical references."],"dc:format":["application/pdf","1 online resource (179 p.) : ill."],"dc:identifier":["etd:52554","local:Hou_grad.msu_0128D_21351","https://doi.org/doi:10.25335/daan-ck96"],"dc:language":["eng"],"dc:relation":["Electronic Theses & Dissertations"],"dc:rights":["In Copyright"],"dc:subject":["Computer science"],"dc:title":["Face Modeling Under Diverse Illuminations"],"dc:type":["text","Electronic dissertations"]},"updated_at":"2026-11-18T18:59:06Z"}