Hierarchical Means Single Number Benchmarking with …分级意味着单一数字基准….pptVIP

Hierarchical Means: Single Number Benchmarking with Workload Cluster Analysis Richard M. Yoo Hsien-Hsin S. Lee Han Lee Kingsum Chow Agenda Identify a new type of workload redundancy specific to benchmark suite merger Discuss a framework to detect workload redundancy Propose a new set of scoring methods to workaround workload redundancy Case study Benchmark Suite Merger Creating a new benchmark suite by adopting workloads from pre-existing benchmark suites Examples MineBench will incorporate workloads from ClusBench Next release of SPECjvm would include workloads from SciMark2 It is good Create a new benchmark suite in a relatively short amount of time Overcome the lack of domain knowledge Inherit the proven credibility of existing benchmark suites It is bad Significantly increases workload redundancy Categorizing Workload Redundancy Natural Redundancy Occurs when sampling the user workload space Ex) Scientific applications are usually floating-point intensive = Scientific benchmark suite contains many floating-point workloads Reflects the user workload spectrum Traditional definition of workload redundancy in a benchmark suite Artificial Redundancy Specific to benchmark suite merger Artificial Redundancy Explained Newly added workloads fail to ‘mix-in’ with the rest of the workloads All the workloads in the adoption set become redundant to each other Artificial Redundancy Considered Harmful Artificial redundancy biases the score calculation methods Current scoring methods (arithmetic mean, geometric mean, etc.) = Do not differentiate redundant workloads from ‘critical’ workloads Giving the same ‘vote’ to all the workloads regardless of their importance Redundant workloads misleadingly amplify their aggregated effect on the overall score Compiler or hardware enhancement techniques will be misleadingly targeted for redundant workloads Ill minded optimizations could break the robustness of the scoring metric by specifically focusing on the redundant