Georgia Institute of Technology
Exploiting Memory-Level Parallelism to Mitigate the Impact of Critical Loads on Performance and Energy Efficiency
Abstract
dc:description.abstractLong latency loads that stall the Re-Order Buffer (ROB) while holding execution resources are critical to processor performance. In the scenario of multi-threaded coroutines, the context switching cost is only a few nanoseconds which is the same order of magnitude as the stalled cycles. This opens up an opportunity to hide the latency of these critical loads behind useful work from other threads (thread-level parallelism – TLP) by context switching after encountering critical loads and paying the context-switching cost. The context switch can be triggered after a load stalls the ROB (reactive approach), but this limits gains from TLP. Instead, we propose predicting critical loads early in the pipeline and triggering a switch (proactive approach). We find that the gains from TLP alone are not always sufficient to balance the context-switching cost. In addition to TLP, there can be multiple independent loads within each thread (memory-level parallelism – MLP), which are free of cost. After predicting a critical load, the optimal number of instructions to expose intra-thread MLP is predicted using an MLP predictor. These optimal instructions are then fetched in the runahead mode before triggering a context switch so that they don’t stall the ROB. In this way, our proposed design maximizes the gain per context switch decision, by exposing both TLP as well as MLP. Further, our design takes the frequency of the processor into consideration to optimize for energy efficiency. The criticality predictor and MLP predictor are crucial building blocks for our design. Existing MLP predictors proposed in prior literature do not consider load criticality and end up overestimating available MLP. We propose a new MLP metric – “MLP Density,” which better represents available MLP. Existing criticality predictors only identify critical loads but do not quantify how critical they are. We propose a new criticality metric – “Criticality Impact,” which captures the degree of criticality of loads and can directly be compared against the context-switching cost for making optimal context-switching decisions. Our design achieves a 10-15% average speedup over the single-thread baseline as well as an 8% speedup compared to MLP-agnostic policies. Our frequency-aware context-switching policy achieves a 9% average improvement in EDP.
Degree
thesis:*- Level thesis:degree_level
- Masters
- Department dc:contributor.department
- Electrical and Computer Engineering
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Jabade, Shreyan Siddharth
- Advisor dc:contributor.advisor
-
- Daglis, Alexandros
- Committee members dc:contributor.committeemember
-
- Conte, Tom
- Kim, Hyesoon
Subjects
dc:subject × 3Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/76973