[PATCH 1/1] benchtest: Run benchmark only once for each input

[Sekhar, Ashwin]

On Thu, 2017-07-06 at 14:28 +0530, Siddhesh Poyarekar wrote:
> On Thursday 06 July 2017 02:21 PM, Ashwin Sekhar T K wrote:
> > 
> > Currently for each input in the input file, the benchtest framework
> > runs the microbenchmark in a loop a specific number of times before
> > it moves onto the next input. This has the drawback that it might
> > eliminate the cache effects and branch effects that we see in real
> > world programs.
> > 
> > This patch changes the framework in such a way that the
> > microbenchmark
> > is run only once for each input before moving onto the next input.
> This is solved with Wilco's workload addition patch.  That is, for
> workload tests it makes sense to have cache effects in place since
> the
> call sequence kinda resembles a real workload - note that there is
> still
> a non-trivial effect due to other code in the program that cannot be
> emulated without actually running the program itself.
> 
Apologies didn't notice that this patch was already made.

> These inputs on the other hand simply exercise branches of the
> program
> to measure the variance in performance across branches irrespective
> of
> caching.  I'm open to changing the constitution of the inputs to a
> different model, but running them exactly once in that sequence does
> not
> really emulate anything.
> 
> Siddhesh