Back to search

Georgia Institute of Technology

Exploiting Memory-Level Parallelism to Mitigate the Impact of Critical Loads on Performance and Energy Efficiency

Abstract

dc:description.abstract

Long latency loads that stall the Re-Order Buffer (ROB) while holding execution resources are critical to processor performance. In the scenario of multi-threaded coroutines, the context switching cost is only a few nanoseconds which is the same order of magnitude as the stalled cycles. This opens up an opportunity to hide the latency of these critical loads behind useful work from other threads (thread-level parallelism – TLP) by context switching after encountering critical loads and paying the context-switching cost. The context switch can be triggered after a load stalls the ROB (reactive approach), but this limits gains from TLP. Instead, we propose predicting critical loads early in the pipeline and triggering a switch (proactive approach). We find that the gains from TLP alone are not always sufficient to balance the context-switching cost. In addition to TLP, there can be multiple independent loads within each thread (memory-level parallelism – MLP), which are free of cost. After predicting a critical load, the optimal number of instructions to expose intra-thread MLP is predicted using an MLP predictor. These optimal instructions are then fetched in the runahead mode before triggering a context switch so that they don’t stall the ROB. In this way, our proposed design maximizes the gain per context switch decision, by exposing both TLP as well as MLP. Further, our design takes the frequency of the processor into consideration to optimize for energy efficiency. The criticality predictor and MLP predictor are crucial building blocks for our design. Existing MLP predictors proposed in prior literature do not consider load criticality and end up overestimating available MLP. We propose a new MLP metric – “MLP Density,” which better represents available MLP. Existing criticality predictors only identify critical loads but do not quantify how critical they are. We propose a new criticality metric – “Criticality Impact,” which captures the degree of criticality of loads and can directly be compared against the context-switching cost for making optimal context-switching decisions. Our design achieves a 10-15% average speedup over the single-thread baseline as well as an 8% speedup compared to MLP-agnostic policies. Our frequency-aware context-switching policy achieves a 9% average improvement in EDP.

Degree

thesis:*
Level thesis:degree_level
Masters
Department dc:contributor.department
Electrical and Computer Engineering
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Jabade, Shreyan Siddharth
Advisor dc:contributor.advisor
  • Daglis, Alexandros
Committee members dc:contributor.committeemember
  • Conte, Tom
  • Kim, Hyesoon

Subjects

dc:subject × 3

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/76973

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Jabade, Shreyan Siddharth. Exploiting Memory-Level Parallelism to Mitigate the Impact of Critical Loads on Performance and Energy Efficiency. Masters thesis, Georgia Institute of Technology, 2024. https://hdl.handle.net/1853/76973