This is the mail archive of the
kawa@sources.redhat.com
mailing list for the Kawa project.
FW: gabriel benchmarks
- From: "Mamading Ceesay" <mceesay at evangineer dot force9 dot co dot uk>
- To: <kawa at sources dot redhat dot com>
- Date: Sun, 10 Mar 2002 17:37:55 -0000
- Subject: FW: gabriel benchmarks
- Reply-to: <mceesay at evangineer dot force9 dot co dot uk>
Got this from the SISC mailing list. I thought it might be
of interest, particularly in the light of recent postings
regarding Kawa performance.
Regards,
Mamading Ceesay
"Don't worry about what anybody else is going to do.
The best way to predict the future is to invent it."
-- Alan Kay
-----Original Message-----
From: sisc-devel-admin@lists.sourceforge.net
[mailto:sisc-devel-admin@lists.sourceforge.net] On Behalf Of
sisc-devel-request@lists.sourceforge.net
Sent: 09 March 2002 20:31
To: sisc-devel@lists.sourceforge.net
Subject: SISC-devel digest, Vol 1 #42 - 1 msg
Today's Topics:
1. benchmarks (Matthias Radestock)
--__--__--
Message: 1
Date: Sat, 09 Mar 2002 15:43:50 +0000
From: Matthias Radestock <matthias@sorted.org>
To: sisc-devel@lists.sourceforge.net
Subject: [SISC-devel] benchmarks
test setup:
700MHz PIII
RedHat6.2
JDK1.4
debug off
optimization on
Test: gabriel benchmarks
I benchmarked SISC, Kawa(interpreted), Kawa(interpreted with tailcalls),
Kawa(compiled), Kawa(compiled with tailcalls), JScheme(interpreted),
JScheme(compiled). I also tried PS3I but couldn't get it to work.
The most recent packaged versions of the software was used in all tests
except for SISC which uses a recent CVS snapshot that does however not
differ substantially from the most recent packaged release as far as
performance is concerned.
Here are the results:
SISC:
(CPSTAK 379 avg ms)
(Puzzle 170 avg ms)
(FFT 839 avg ms)
(Tprint 677 avg ms)
(Fprint 14 avg ms)
(Fread 105 avg ms)
(Destructive 1592 avg ms)
(Div-rec 682 avg ms)
(Div-iter 657 avg ms)
(Dderiv 1314 avg ms)
(Deriv 1309 avg ms)
(CTAK 1297 avg ms)
(TAKR 333 avg ms)
(TAKL 3539 avg ms)
(TAK 334 avg ms)
Kawa(interpreted):
(Puzzle 143 avg ms)
(FFT 430 avg ms)
(Tprint 706 avg ms)
(Fprint 57 avg ms)
(Fread 127 avg ms)
(Destructive 1050 avg ms)
(Div-rec 105 avg ms)
(Div-iter 91 avg ms)
(Dderiv 382 avg ms)
(Deriv 340 avg ms)
(CTAK 10562 avg ms)
(TAKR 579 avg ms)
(TAKL 601 avg ms)
(TAK 164 avg ms)
Kawa(interpreted with tailcalls):
(CPSTAK 492 avg ms)
(Fread 91 avg ms)
(Destructive 1126 avg ms)
(Div-rec 418 avg ms)
(Div-iter 96 avg ms)
(Dderiv 963 avg ms)
(Deriv 804 avg ms)
(CTAK 18639 avg ms)
(TAKR 1074 avg ms)
(TAKL 2285 avg ms)
(TAK 377 avg ms)
Kawa(compiled):
(Puzzle 75 avg ms)
(FFT 268 avg ms)
(Tprint 525 avg ms)
(Fprint 56 avg ms)
(Fread 98 avg ms)
(Destructive 279 avg ms)
(Div-rec 43 avg ms)
(Div-iter 41 avg ms)
(Dderiv 245 avg ms)
(Deriv 172 avg ms)
(CTAK 9701 avg ms)
(TAKR 213 avg ms)
(TAKL 36 avg ms)
(TAK 27 avg ms)
Kawa(compiled with tailcalls):
(CPSTAK 191 avg ms)
(Fread 109 avg ms)
(Destructive 309 avg ms)
(Div-rec 216 avg ms)
(Div-iter 53 avg ms)
(Dderiv 701 avg ms)
(CTAK 17848 avg ms)
(TAKR 522 avg ms)
(TAKL 380 avg ms)
(TAK 109 avg ms)
JScheme(interpreted):
(CPSTAK 1268 avg ms)
(Puzzle 470 avg ms)
(Tprint 551 avg ms)
(Fprint 24 avg ms)
(Fread 275 avg ms)
(Destructive 3750 avg ms)
(Div-rec 1491 avg ms)
(Div-iter 1558 avg ms)
(Dderiv 2048 avg ms)
(Deriv 1335 avg ms)
(CTAK 21841 avg ms)
(TAKR 782 avg ms)
(TAKL 7328 avg ms)
(TAK 854 avg ms)
JScheme(compiled):
(CPSTAK 928 avg ms)
(Puzzle 419 avg ms)
(Tprint 771 avg ms)
(Fprint 27 avg ms)
(Fread 52 avg ms)
(Destructive 4394 avg ms)
(Div-rec 656 avg ms)
(Div-iter 1471 avg ms)
(Dderiv 1043 avg ms)
(Deriv 602 avg ms)
(CTAK 12544 avg ms)
(TAKR 1379 avg ms)
(TAKL 2080 avg ms)
(TAK 332 avg ms)
I recommend ignoring the Tprint results since Tprint spends most of its
time writing to the console and hence the results are heavily influenced
by the environment the program is run in (e.g. I run it in an emacs
buffer, which is a lot slower than running it in an xterm).
Here are the same figures translated into percentages with SISC as the
baseline (i.e. 50 means that the system ran the test 50% faster than
SISC and -50 means it ran 50% slower):
((cpstak
(sisc . 0)
(kawa . n/a)
(kawa-tc . -30)
(kawa-c . n/a)
(kawa-ctc . 50)
(jscheme . -235)
(jscheme-c . -145))
(puzzle
(sisc . 0)
(kawa . 16)
(kawa-tc . n/a)
(kawa-c . 56)
(kawa-ctc . n/a)
(jscheme . -176)
(jscheme-c . -146))
(fft
(sisc . 0)
(kawa . 49)
(kawa-tc . n/a)
(kawa-c . 68)
(kawa-ctc . n/a)
(jscheme . n/a)
(jscheme-c . n/a))
(tprint
(sisc . 0)
(kawa . -4)
(kawa-tc . n/a)
(kawa-c . 22)
(kawa-ctc . n/a)
(jscheme . 19)
(jscheme-c . -14))
(fprint
(sisc . 0)
(kawa . -307)
(kawa-tc . n/a)
(kawa-c . -300)
(kawa-ctc . n/a)
(jscheme . -71)
(jscheme-c . -93))
(fread
(sisc . 0)
(kawa . -21)
(kawa-tc . 13)
(kawa-c . 7)
(kawa-ctc . -4)
(jscheme . -162)
(jscheme-c . 50))
(destructive
(sisc . 0)
(kawa . 34)
(kawa-tc . 29)
(kawa-c . 82)
(kawa-ctc . 81)
(jscheme . -136)
(jscheme-c . -176))
(div-rec
(sisc . 0)
(kawa . 85)
(kawa-tc . 39)
(kawa-c . 94)
(kawa-ctc . 68)
(jscheme . -119)
(jscheme-c . 4))
(div-iter
(sisc . 0)
(kawa . 86)
(kawa-tc . 85)
(kawa-c . 94)
(kawa-ctc . 92)
(jscheme . -137)
(jscheme-c . -124))
(dderiv
(sisc . 0)
(kawa . 71)
(kawa-tc . 27)
(kawa-c . 81)
(kawa-ctc . 47)
(jscheme . -56)
(jscheme-c . 21))
(deriv
(sisc . 0)
(kawa . 74)
(kawa-tc . 39)
(kawa-c . 87)
(kawa-ctc . n/a)
(jscheme . -2)
(jscheme-c . 54))
(ctak
(sisc . 0)
(kawa . -714)
(kawa-tc . -1337)
(kawa-c . -648)
(kawa-ctc . -1276)
(jscheme . -1584)
(jscheme-c . -867))
(takr
(sisc . 0)
(kawa . -74)
(kawa-tc . -223)
(kawa-c . 36)
(kawa-ctc . -57)
(jscheme . -135)
(jscheme-c . -314))
(takl
(sisc . 0)
(kawa . 83)
(kawa-tc . 35)
(kawa-c . 99)
(kawa-ctc . 89)
(jscheme . -107)
(jscheme-c . 41))
(tak
(sisc . 0)
(kawa . 51)
(kawa-tc . -13)
(kawa-c . 92)
(kawa-ctc . 67)
(jscheme . -156)
(jscheme-c . 1)))
Compiled Kawa comes top in most tests. Interpreted Kawa beats SISC in
quite a few cases too - "interpreted" isn't quite the right word, Kawa
actually compiles most code on the fly when run in interpreted mode.
However, SISC is *never* more than twice as slow than the fastest system
and outperforms interpreted JScheme in all tests except tprint (but see
above comment), often by a substantial margin, and outperforms compiled
JScheme in most tests.
The continuation-intensive ctak test is where SISC really excels - the
nearest competitor is a 650% slower.
It's probably also worth mentioning that we are not comparing apples
with apples since SISC's performance is affected by:
* Strict standard adherence. This, amongst other things, results in SISC
passing all the tests whereas Kawa and JScheme require modifications to
the tests and even then still fail some.
* SISC always performs tail call optimisation
* SISC implements the full numeric tower, including arbitrary precision
math. (I'm not sure where Kawa and JScheme stand on this)
* SISC has full support for continuations, not just escaping
continuations (I'm not sure where Kawa and JScheme stand on this)
Matthias
--__--__--
_______________________________________________
SISC-devel mailing list
SISC-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sisc-devel
End of SISC-devel Digest