{"id":{"repo_id":"purdue-thes","oai_identifier":"oai:docs.lib.purdue.edu:open_access_dissertations-1218"},"canonical_url":"https://search.ndltd.org/etd/purdue-thes/oai:docs.lib.purdue.edu:open_access_dissertations-1218","repository":{"repo_id":"purdue-thes","name":"Purdue University","base_url":"https://docs.lib.purdue.edu/do/oai/"},"display":{"title":"Optimal \"Big Data\" Aggregation Systems - From Theory to Practical Application","abstract":"The integration of computers into many facets of our lives has made the collection and storage of staggering amounts of data feasible. However, the data on its own is not so useful to us as the analysis and manipulation which allows manageable descriptive information to be extracted. New tools to extract this information from ever growing repositories of data are required. Some of these analyses can take the form of a two phase problem which is easily distributed to take advantage of available computing power. The first phase involves computing some descriptive partial result from some subset of the original data, and the second phase involves aggregating all the partial results to create a combined output. We formalize this compute-aggregate model for a rigorous performance analysis in an effort to minimize the latency of the aggregation phase with minimal intrusive analysis or modification. Based on our model we find an aggregation overlay attribute which highly affects aggregation latency and its dependence on an easily findable trait of aggregation. We rigorously prove the dependence and find optimal overlays for aggregation. We use the proven optima to create simple heuristics and build a system, NOAH, to take advantage of the findings. NOAH can be used by big data analysis systems. We also study an individual problem, top-k matching, to explore the effects of optimizing the computation phase separately from aggregation and create a complete distributed system to fulfill an economically relevant task.","abstract_html":"The integration of computers into many facets of our lives has made the collection and storage of staggering amounts of data feasible. However, the data on its own is not so useful to us as the analysis and manipulation which allows manageable descriptive information to be extracted. New tools to extract this information from ever growing repositories of data are required. Some of these analyses can take the form of a two phase problem which is easily distributed to take advantage of available computing power. The first phase involves computing some descriptive partial result from some subset of the original data, and the second phase involves aggregating all the partial results to create a combined output. We formalize this compute-aggregate model for a rigorous performance analysis in an effort to minimize the latency of the aggregation phase with minimal intrusive analysis or modification. Based on our model we find an aggregation overlay attribute which highly affects aggregation latency and its dependence on an easily findable trait of aggregation. We rigorously prove the dependence and find optimal overlays for aggregation. We use the proven optima to create simple heuristics and build a system, NOAH, to take advantage of the findings. NOAH can be used by big data analysis systems. We also study an individual problem, top-k matching, to explore the effects of optimizing the computation phase separately from aggregation and create a complete distributed system to fulfill an economically relevant task.","abstract_has_math":false,"creators":["Culhane, William J, IV"],"institution":null,"degree_name":"Doctor of Philosophy (PhD)","degree_level":"Dissertation","degree_discipline":"Philosophy","degree_department":null,"school":null,"contributors":["Patrick Eugster","Charles Killian","Sonia Fahmy","Dongyan Xu"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:51:58Z","subjects":["Big data","Cloud computing","Aggregation","Top-k","Matching","Algorithms","Distributed Systems","Systems","Computer Sciences","Theory and Algorithms"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://docs.lib.purdue.edu/open_access_dissertations/216","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://docs.lib.purdue.edu/open_access_dissertations/216","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://docs.lib.purdue.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adocs.lib.purdue.edu%3Aopen_access_dissertations-1218","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Patrick Eugster","Charles Killian","Sonia Fahmy","Dongyan Xu"]},{"key":"dc:creator","label":"Author","values":["Culhane, William J, IV"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.created","label":"Dc Date Created","values":["2015-05-01T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Philosophy"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Big data","Cloud computing","Aggregation","Top-k","Matching","Algorithms","Distributed Systems","Systems","Computer Sciences","Theory and Algorithms"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://docs.lib.purdue.edu/open_access_dissertations/216"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The integration of computers into many facets of our lives has made the collection and storage of staggering amounts of data feasible. However, the data on its own is not so useful to us as the analysis and manipulation which allows manageable descriptive information to be extracted. New tools to extract this information from ever growing repositories of data are required. Some of these analyses can take the form of a two phase problem which is easily distributed to take advantage of available computing power. The first phase involves computing some descriptive partial result from some subset of the original data, and the second phase involves aggregating all the partial results to create a combined output. We formalize this compute-aggregate model for a rigorous performance analysis in an effort to minimize the latency of the aggregation phase with minimal intrusive analysis or modification. Based on our model we find an aggregation overlay attribute which highly affects aggregation latency and its dependence on an easily findable trait of aggregation. We rigorously prove the dependence and find optimal overlays for aggregation. We use the proven optima to create simple heuristics and build a system, NOAH, to take advantage of the findings. NOAH can be used by big data analysis systems. We also study an individual problem, top-k matching, to explore the effects of optimizing the computation phase separately from aggregation and create a complete distributed system to fulfill an economically relevant task."]},{"key":"dc:description.note","label":"Dc Description Note","values":["<p>© 2015 IEEE. Reprinted, with permission, from Culhane, William and Kogan, Kirill and Jayalath, Chamikara and Eugster, Patrick. Optimal Communication Structures for Big Data Aggregation. INFOCOM '15. DOI: <a href=\"http://dx.doi.org/10.1109/INFOCOM.2015.7218544\">10.1109/INFOCOM.2015.7218544</a></p> <p>In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Purdue University's products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to<a href=\"https://owa.purdue.edu/owa/redir.aspx?SURL=vP_5bU4G3NlDfVO2uZWFSInAiNEH7AsRh_UMltHudRu84BWmrLDSCGgAdAB0AHAAOgAvAC8AdwB3AHcALgBpAGUAZQBlAC4AbwByAGcALwBwAHUAYgBsAGkAYwBhAHQAaQBvAG4AcwBfAHMAdABhAG4AZABhAHIAZABzAC8AcAB1AGIAbABpAGMAYQB0AGkAbwBuAHMALwByAGkAZwBoAHQAcwAvAHIAaQBnAGgAdABzAF8AbABpAG4AawAuAGgAdABtAGwA&URL=http%3a%2f%2fwww.ieee.org%2fpublications_standards%2fpublications%2frights%2frights_link.html\" target=\"_blank\">http://www.ieee.org/publications_standards/publications/rights/rights_link.html</a> to learn how to obtain a License from RightsLink.</p>"]},{"key":"dc:title","label":"Title","values":["Optimal \"Big Data\" Aggregation Systems - From Theory to Practical Application"]}]}],"canonical_facts":{"dc:contributor":["Patrick Eugster","Charles Killian","Sonia Fahmy","Dongyan Xu"],"dc:creator":["Culhane, William J, IV"],"dc:date.created":["2015-05-01T07:00:00Z"],"dc:description.abstract":["The integration of computers into many facets of our lives has made the collection and storage of staggering amounts of data feasible. However, the data on its own is not so useful to us as the analysis and manipulation which allows manageable descriptive information to be extracted. New tools to extract this information from ever growing repositories of data are required. Some of these analyses can take the form of a two phase problem which is easily distributed to take advantage of available computing power. The first phase involves computing some descriptive partial result from some subset of the original data, and the second phase involves aggregating all the partial results to create a combined output. We formalize this compute-aggregate model for a rigorous performance analysis in an effort to minimize the latency of the aggregation phase with minimal intrusive analysis or modification. Based on our model we find an aggregation overlay attribute which highly affects aggregation latency and its dependence on an easily findable trait of aggregation. We rigorously prove the dependence and find optimal overlays for aggregation. We use the proven optima to create simple heuristics and build a system, NOAH, to take advantage of the findings. NOAH can be used by big data analysis systems. We also study an individual problem, top-k matching, to explore the effects of optimizing the computation phase separately from aggregation and create a complete distributed system to fulfill an economically relevant task."],"dc:description.note":["<p>© 2015 IEEE. Reprinted, with permission, from Culhane, William and Kogan, Kirill and Jayalath, Chamikara and Eugster, Patrick. Optimal Communication Structures for Big Data Aggregation. INFOCOM '15. DOI: <a href=\"http://dx.doi.org/10.1109/INFOCOM.2015.7218544\">10.1109/INFOCOM.2015.7218544</a></p> <p>In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Purdue University's products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to<a href=\"https://owa.purdue.edu/owa/redir.aspx?SURL=vP_5bU4G3NlDfVO2uZWFSInAiNEH7AsRh_UMltHudRu84BWmrLDSCGgAdAB0AHAAOgAvAC8AdwB3AHcALgBpAGUAZQBlAC4AbwByAGcALwBwAHUAYgBsAGkAYwBhAHQAaQBvAG4AcwBfAHMAdABhAG4AZABhAHIAZABzAC8AcAB1AGIAbABpAGMAYQB0AGkAbwBuAHMALwByAGkAZwBoAHQAcwAvAHIAaQBnAGgAdABzAF8AbABpAG4AawAuAGgAdABtAGwA&URL=http%3a%2f%2fwww.ieee.org%2fpublications_standards%2fpublications%2frights%2frights_link.html\" target=\"_blank\">http://www.ieee.org/publications_standards/publications/rights/rights_link.html</a> to learn how to obtain a License from RightsLink.</p>"],"dc:identifier":["https://docs.lib.purdue.edu/open_access_dissertations/216"],"dc:subject":["Big data","Cloud computing","Aggregation","Top-k","Matching","Algorithms","Distributed Systems","Systems","Computer Sciences","Theory and Algorithms"],"dc:title":["Optimal \"Big Data\" Aggregation Systems - From Theory to Practical Application"],"thesis:degree_discipline":["Philosophy"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Doctor of Philosophy (PhD)"]},"updated_at":"2026-11-19T08:51:58Z"}