Re: UndoDB's performance

[Sean Chen <sean dot chen1234 at gmail dot com>]

On Tue, Dec 15, 2009 at 9:50 PM, Marc Khouzam <marc.khouzam@ericsson.com> wrote:
>> -----Original Message-----
>> From: gdb-owner@sourceware.org
>> [mailto:gdb-owner@sourceware.org] On Behalf Of Greg Law
>> Sent: Tuesday, December 15, 2009 4:12 AM
>> To: Sean Chen
>> Cc: Hui Zhu; gdb@sourceware.org
>> Subject: Re: UndoDB's performance
>>
>> Sean Chen wrote:
>> > On Tue, Dec 15, 2009 at 11:56 AM, Hui Zhu
>> <teawater@gmail.com> wrote:
>> >> I did some test but its speed close to prec. ?Some others
>> was better.
>> >> Maybe it have some special performance technology for some code.
>> >>
>> >> Hui
>> >>
>> >> On Tue, Dec 15, 2009 at 10:40, Sean Chen
>> <sean.chen1234@gmail.com> wrote:
>> >>> On Mon, Dec 14, 2009 at 11:07 PM, Hui Zhu
>> <teawater@gmail.com> wrote:
>> >>>> Try undodb.
>> >>>>
>> >>>> On Mon, Dec 14, 2009 at 21:18, Sean Chen
>> <sean.chen1234@gmail.com> wrote:
>> >>>>> On Mon, Dec 14, 2009 at 7:14 PM, Hui Zhu
>> <teawater@gmail.com> wrote:
>> >>>>>> gdb-7 reverse debugging accelerator.
>> >>>>>> Regular gdb-7 reverse runs apps ? ? ? ? ?40,000x
>> ? ? ? ? ?slower
>> >>>>>> UndoDB+gdb-7 reverse runs apps ? ? ? ? ? 1.7x
>> ? slower!
>> >>>>>>
>> >>>>>>
>> >>>>>> Are you really do some try?
>> >>>>>> I suggest you do some test on it. ?:)
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> Hui
>> >>>>>>
>> >>>>>> On Fri, Dec 4, 2009 at 23:34, Sean Chen
>> <sean.chen1234@gmail.com> wrote:
>> >>>>>>> On UndoDb's website, I saw the following ad.
>> >>>>>>>
>> >>>>>>> Regular gdb-7 reverse runs apps 40,000x slower
>> >>>>>>> UndoDB+gdb-7 reverse runs apps 1.7x slower!
>> >>>>>>>
>> >>>>>>> In my experiment, gdb-7 reverse does run apps more
>> than 20,000x
>> >>>>>>> slower. How does UndoDB improve the performance so
>> much without a
>> >>>>>>> simulator and without record? Below is its
>> self-introduction on the
>> >>>>>>> website. UndoDB's "snapshot-and-replay" technique
>> stores periodic
>> >>>>>>> copy-on-write snapshots of the application and only
>> non-deterministic
>> >>>>>>> inputs (system calls, thread-switches, shared memory
>> reads, etc).
>> >>>>>>>
>> >>>>>>> Are there any obvious disadvantages in UndoDB? I
>> tried to search
>> >>>>>>> UndoDB in the archive of the mailing list, however,
>> little discussion
>> >>>>>>> is found.
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Best Regards,
>> >>>>>>> Sean Chen
>> >>>>>>>
>> >>>>> That's UndoDB's ad on its website.
>> >>>>>
>> >>>>> In my experiment, gdb-7 reverse runs apps about 22,000x slower.
>> >>>>>
>> >>>>> --
>> >>>>> Best Regards,
>> >>>>> Sean Chen
>> >>>>>
>> >>> I also tried UndoDB, and it is really fast. So I am sure they are
>> >>> using different strategy.
>> >>>
>> >>> --
>> >>> Best Regards,
>> >>> Sean Chen
>> >>>
>> >
>> > It might not use process record technique like your design.
>> >
>> > Below is its self-introduction on its website.
>> > UndoDB's "snapshot-and-replay" technique stores periodic
>> copy-on-write
>> > snapshots of the application and only non-deterministic
>> inputs (system
>> > calls, thread-switches, shared memory reads, etc).
>> >
>> > Any comments?
>>
>> Yes, I can confirm that's what we do. ?We did it that way
>> (and generally
>> tried very hard) to get the best record-time performance. ?To
>> answer the
>> OP, the downside is replay mode performance, namely stepping
>> backwards,
>> will be worse because you have a lot more work to do. ?You
>> also need to
>> store the snapshots, although as noted, copy-on-write can
>> mitigate this
>> cost.
>
> In February 09, I had run a couple of tests. ?Here are the results,
> but keep in mind they are almost a year old and that things might
> have changed since then.
>
> I had a very simple program with a recursive call that contained a printf.
> I ran the program with and without recording with different depth
> of recursion. ?No replaying. ?Two results below.
>
> GDB with *no* recording: 1.35 seconds
> GDB with recording (PRecord): 223 seconds (165 times slower)
> UndoDB: 2.17 seconds (1.6 times slower)
>
> GDB with *no* recording: 0.63 seconds
> GDB with recording (PRecord): 107 seconds (169 times slower)
> UndoDB: 0.74 seconds (1.17 times slower)
>
> As Greg explains, I did notice that in some cases, when actually
> doing back stepping, UndoDB would take a long time to
> perform an operation. ?Also, going forward again was sometimes
> very slow with UndoDb.
>
> PRecord was better at replaying and had the extremely nice
> feature of allowing you to change memory and resume
> execution.
>
> Marc
>

Hi Marc,

I am quite surprise that the performance you logged is 100 times
faster than mine with recording (PRecord).

I am using the following simple code with gdb 7.0.

  gettimeofday(&tv1, NULL);

  for (i=0;i<1000000;i++) {
  }

  gettimeofday(&tv2, NULL);


In my test, the result is quite different from yours.

GDB with *no* recording: 2.63 ms
GDB with recording (PRecord): 58554 ms (20000 times slower)

(gdb) info record insn-number
Record instruction number is 3001052. (Including gettimeofday())

I also tried the following recursive function, and the result was
about 10000 times slower.

void recursive_print(long n)
{
  if (n == 0) {
    printf("recursive_print end\n");
  } else {
    recursive_print(n-1);
  }
  return;
}

Did you use any optimization? Please do correct me if I make any
mistake. Thanks.

-- 
Best Regards,
Sean Chen