This is the mail archive of the xsl-list@mulberrytech.com mailing list .


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]

Re: Latest XSLTMark benchmark



Mike Kay wrote:
> A new set of results for the XSLTMark benchmark has been published at
>  http://www.datapower.com/XSLTMark/res_2001_03_28.html#ChartOverall
>
> The benchmarking methodology is to compile the stylesheet, and then run
it a
> large number of times over the same source document, averaging the
results.
>
> I have taken a look at some of the drivers for different XSLT processors.
>
> As far as I can see the drivers for Oracle, MSXML, and Sun XSLTC parse
and
> load the source document once, and then transform it many times from the
> same in-memory tree, while the drivers for xt, Saxon, and Xalan parse the
> source document and build the tree each time round the loop.
>
> Since parsing and building the tree account for a significant proportion
of
> the transformation time, this would significantly skew the results.
>
> Are my suspicions correct?

That was my conclusion as well.  I also examined their Xalan-C++ driver,
which is poorly written, leaks memory, and uses a suboptimal source tree
for the transformation.  It took 15 minutes for me to re-write their
driver, and the results are significantly improved.  I only wish someone
had bothered to post a request to the Xalan list for a review of their
driver, or at least some notice that they were doing this.

> (Even if this is corrected, the methodology is flawed, because a
processor
> could legitimately cache the results of parsing the document in memory,
> unknown to the driver. Ideally the source file should be trivially
modified
> between each iteration to prevent this).

Companies which post benchmarks ought to be more responsible.

Dave Bertoni
IBM Corporation (Xalan-C++ committer)


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]