Quantcast

Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
It seems I just came across a similar performance difference as someone already noticed 5 years ago in http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/net-components-performance-question-td8223.html

Running a transformation using transform.exe takes about 5 minutes for a reasonably complex stylesheet (about 500kb of source, split into roughly 35 included modules) and a 5 MB xml file.
The same thing with Saxon.Api takes about 30 Minutes for no appearent reason.

As this is Saxon-HE, the old suggestion of using allNodesUntyped does not apply (it seemed to have been removed with the move from Saxon-B to Saxon-HE somewhere around 9.2).
Anything else I could do, short of using the Java-style classes directly (like transform.exe does)?

I'm using Saxon-HE 9.4.0.6N on .NET 3.5SP1, in case it makes a difference. Ok, latest is 9.4.0.9, but I doubt it makes much of a difference.

Regards, Emanuel

---------
//relevant source, in case it matters:
private readonly Processor _processor = new Processor(false);
public void Transform(string sourcePath, string stylesheetPath, string outputPath)
{
        using (var source = File.OpenRead(sourcePath))
        using (var stylesheet = File.OpenRead(stylesheetPath))
        {
                Uri stylesheetUri, sourceUri;
                if (!Uri.TryCreate(sourcePath, UriKind.RelativeOrAbsolute, out sourceUri))
                        throw new ArgumentException(string.Format("Could not create an URI from source path '{0}'.", sourcePath), "sourcePath");
                if (!Uri.TryCreate(stylesheetPath, UriKind.RelativeOrAbsolute, out stylesheetUri))
                        throw new ArgumentException(string.Format("Could not create an URI from stylesheet path '{0}'.", stylesheetPath), "stylesheetPath");

                var compiler = _processor.NewXsltCompiler();

                compiler.BaseUri = stylesheetUri; //required to resolve stylesheet includes
                compiler.ErrorList = new ArrayList();
                var executable = compiler.Compile(stylesheet);
                var transformer = executable.Load();
                var builder = _processor.NewDocumentBuilder();
                builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver); //required to ignore DTDs in System IDs
                builder.BaseUri = sourceUri;
                transformer.InitialContextNode = builder.Build(source);
                var destination = new Serializer();
                using (var output = File.Open(outputPath, FileMode.Create))
                {
                        destination.SetOutputStream(output);
                        transformer.Run(destination);
                }
        }
}
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay

On 25 Sep 2013, at 13:48, Emanuel Wlaschitz wrote:

> It seems I just came across a similar performance difference as someone
> already noticed 5 years ago in
> http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/net-components-performance-question-td8223.html
>
> Running a transformation using transform.exe takes about 5 minutes for a
> reasonably complex stylesheet (about 500kb of source, split into roughly 35
> included modules) and a 5 MB xml file.
> The same thing with Saxon.Api takes about 30 Minutes for no appearent
> reason.

Thanks for reporting this. We've been investigating a performance anomaly with Saxon on .NET over the last three weeks (and making rather little headway with it, I have to admit). Perhaps this example will provide another line of attack and fresh information. I suspect some kind of configuration issue but it's proving very difficult to pin down. If you can send us the workload in question in a form that enables us to run it, then the first thing we can see is whether we are able to reproduce the anomaly.

It could be a completely different problem of course; but either way, we will need to reproduce it to make any headway.

There's certainly no intrinsic reason why there should be such a performance difference, and there's nothing obviously wrong with your invoking code.

One thing to check with slow-running code is whether it's attempting to fetch DTDs or schema files from the W3C server. W3C throttle such requests artificially, leading to long elapsed times. Saxon attempts to resolve such requests to local copies, but I note you have your own XmlResolver here, so we would first need to see what URIs you are fetching and how they are being resolved.

Michael Kay
Saxonica

>
> As this is Saxon-HE, the old suggestion of using allNodesUntyped does not
> apply (it seemed to have been removed with the move from Saxon-B to Saxon-HE
> somewhere around 9.2).
> Anything else I could do, short of using the Java-style classes directly
> (like transform.exe does)?
>
> I'm using Saxon-HE 9.4.0.6N on .NET 3.5SP1, in case it makes a difference.
> Ok, latest is 9.4.0.9, but I doubt it makes much of a difference.
>
> Regards, Emanuel
>
> ---------
> //relevant source, in case it matters:
> private readonly Processor _processor = new Processor(false);
> public void Transform(string sourcePath, string stylesheetPath, string
> outputPath)
> {
> using (var source = File.OpenRead(sourcePath))
> using (var stylesheet = File.OpenRead(stylesheetPath))
> {
> Uri stylesheetUri, sourceUri;
> if (!Uri.TryCreate(sourcePath, UriKind.RelativeOrAbsolute, out sourceUri))
> throw new ArgumentException(string.Format("Could not create an URI from
> source path '{0}'.", sourcePath), "sourcePath");
> if (!Uri.TryCreate(stylesheetPath, UriKind.RelativeOrAbsolute, out
> stylesheetUri))
> throw new ArgumentException(string.Format("Could not create an URI from
> stylesheet path '{0}'.", stylesheetPath), "stylesheetPath");
>
> var compiler = _processor.NewXsltCompiler();
>
> compiler.BaseUri = stylesheetUri; //required to resolve stylesheet
> includes
> compiler.ErrorList = new ArrayList();
> var executable = compiler.Compile(stylesheet);
> var transformer = executable.Load();
> var builder = _processor.NewDocumentBuilder();
> builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver);
> //required to ignore DTDs in System IDs
> builder.BaseUri = sourceUri;
> transformer.InitialContextNode = builder.Build(source);
> var destination = new Serializer();
> using (var output = File.Open(outputPath, FileMode.Create))
> {
> destination.SetOutputStream(output);
> transformer.Run(destination);
> }
> }
> }
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
Thats interresting to know, thanks!
Michael Kay wrote
One thing to check with slow-running code is whether it's attempting to fetch DTDs or schema files from the W3C server. W3C throttle such requests artificially, leading to long elapsed times. Saxon attempts to resolve such requests to local copies, but I note you have your own XmlResolver here, so we would first need to see what URIs you are fetching and how they are being resolved.
The XmlResolver (as the name implies) ignores all requests for *.dtd and *.xsd, because in some cases we couldn't resolve them, even if we wanted to. And from practise we know that just passing the source as well-formed document works good enough, so we ignore them right away. Hence I don't think that DTD-fetching is an issue. Infact, I just ran Wireshark while the transformation was running, and no requests going anywhere.

I tried profiling the code, but as compiling Saxon from source is a tedious task, the results aren't really useful without symbols and per-line data.

As for sample data, I'll have to try and create a package first, as the testdata I used isn't something I'm supposed to give away.
I'll get back to you once I got something that shows the issue.
In the meantime, is there anything else I could try? Like, for example, dump the internal Configuration object to some sort of text/xml/readable format and compare the two?

Regards, Emanuel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay
If you can run profiles for the two cases and send them both to us, then they might tell us something - even if it's only to eliminate some possible causes.

Michael Kay
Saxonica

On 26 Sep 2013, at 07:13, Emanuel Wlaschitz wrote:

> Thats interresting to know, thanks!
>
> Michael Kay wrote
>> One thing to check with slow-running code is whether it's attempting to
>> fetch DTDs or schema files from the W3C server. W3C throttle such requests
>> artificially, leading to long elapsed times. Saxon attempts to resolve
>> such requests to local copies, but I note you have your own XmlResolver
>> here, so we would first need to see what URIs you are fetching and how
>> they are being resolved.
>
> The XmlResolver (as the name implies) ignores all requests for *.dtd and
> *.xsd, because in some cases we couldn't resolve them, even if we wanted to.
> And from practise we know that just passing the source as well-formed
> document works good enough, so we ignore them right away. Hence I don't
> think that DTD-fetching is an issue. Infact, I just ran Wireshark while the
> transformation was running, and no requests going anywhere.
>
> I tried profiling the code, but as compiling Saxon from source is a tedious
> task, the results aren't really useful without symbols and per-line data.
>
> As for sample data, I'll have to try and create a package first, as the
> testdata I used isn't something I'm supposed to give away.
> I'll get back to you once I got something that shows the issue.
> In the meantime, is there anything else I could try? Like, for example, dump
> the internal Configuration object to some sort of text/xml/readable format
> and compare the two?
>
> Regards, Emanuel
>
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12294.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
I'm not exactly sure if this helps, but I've ran some tests using Docbook 5: The definitive Guide and the official XSLT 2.0 stylesheets.

transform.exe* with about 2 seconds vs. the Api approach with about 8 seconds.
In the profiler, the time required to run the two is somewhere around 12 vs. 20 seconds (due to profiler overhead). I hope the results come thru, I tried to attach them as zip (which are the results exported from RedGate ANTS Performance Profiler 8.2 as both XML and HTML, for transform.exe and Saxon.Api respectively), but I can't tell if they really end up on the list.
The time spent in Controller.transform differs by about a second between the two, or even 4 seconds looking at wall-clock time (including the time waiting for synchronization).
I don't know what that could mean, or if it would mean anything, as the stacktrace simply looks like XSL processing (duh).

My testcase is still running with the profiler, and I don't expect it to finish within the next hour, but I'll keep you posted on anything I might see in there.

*) Actually, I didn't profile transform.exe, but I used two methods in my test program. Transform is the one I posted earlier (using Saxon.Api), while TransformDirect uses Transform.doTransform just like transform.exe does (and it really gets the same performance there).

profiling.zip
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

O'Neil Delpratt
As Mike has mentioned in an earlier message we are currently investigating an  performance anomaly on SAxon .NET.

What is interesting is the time spent 'Waiting for synchronisation'

I came across this in my own profiling and it is still puzzling me what this exactly means because it seems most of the time is spent on this synchronisation thread.

Kind regards,

O'Neil



On 26/09/13 13:52, Emanuel Wlaschitz wrote:
I'm not exactly sure if this helps, but I've ran some tests using Docbook 5:
The definitive Guide and the official XSLT 2.0 stylesheets.

transform.exe* with about 2 seconds vs. the Api approach with about 8
seconds.
In the profiler, the time required to run the two is somewhere around 12 vs.
20 seconds (due to profiler overhead). I hope the results come thru, I tried
to attach them as zip (which are the results exported from RedGate ANTS
Performance Profiler 8.2 as both XML and HTML, for transform.exe and
Saxon.Api respectively), but I can't tell if they really end up on the list.
The time spent in Controller.transform differs by about a second between the
two, or even 4 seconds looking at wall-clock time (including the time
waiting for synchronization).
I don't know what that could mean, or if it would mean anything, as the
stacktrace simply looks like XSL processing (duh).

My testcase is still running with the profiler, and I don't expect it to
finish within the next hour, but I'll keep you posted on anything I might
see in there.

*) Actually, I didn't profile transform.exe, but I used two methods in my
test program. Transform is the one I posted earlier (using Saxon.Api), while
TransformDirect uses Transform.doTransform just like transform.exe does (and
it really gets the same performance there).

profiling.zip
<http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/file/n12296/profiling.zip>  



--
View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12296.html
Sent from the saxon-help mailing list archive at Nabble.com.

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 



--
O'Neil Delpratt
Software Developer, Saxonica Limited
Email: [hidden email]
Tel: +44 118 946 5894
Web: http://www.saxonica.com
Saxonica Community Site: http://dev.saxonica.com Saxonica Bug tracking System: https://saxonica.plan.io/

------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
Dr O'Neil Delpratt wrote
What is interesting is the time spent 'Waiting for synchronisation'

I came across this in my own profiling and it is still puzzling me what
this exactly means because it seems most of the time is spent on this
synchronisation thread.
Synchronization is usually:
- dispatching stuff to the UI thread (which shouldn't be the case in a console application nor in a library)
- locking/blocking/signaling/Monitor.Wait things, also Thread.Join I believe.
I don't think any of those should be happening in there, multithread etc. is off by default.

To get back to my profiling sample; the unprofiled 5 minutes became 45 minutes under the profiler; and the result is far too big to upload here.
Percentage-wise, 84% is "Waiting for synchronization" while the next percentage is 1% in java.lang.CharSequence.length(); with the rest <1%.
But thats for transform.exe; and I believe the synchronization is a side-effect of the profiling itself.
I'll run the profiler with api over night, as I expect it to run for several hours; and look at it tomorrow.

Regards, Emanuel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
The results are in. 30 minutes became 2 hours 40 minutes in the profiler, and showed...nothing really useful?
One thing that looked odd was a fairly linear time distribution until one Mode.applyTemplates which split up into TextCopyOnlyRuleSet.process and ApplyTemplates+ApplyTemplatesPackage.processLeavingTail (with a weighting of about 65:35). The transform trace didn't show such results until very deep down where it came to a bunch of java methods from IKVM. Not sure if that means anything.
The main difference is the distribution of "Waiting for synchronization" - 95% with the next trace being TinyTree.getNode at 0.7%.

The question is still where this waiting comes from. I'll try the visual studio profiler next, as it has options and stuff for concurrency profiling.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay
We also saw heavy incidence of "Waiting for synchronization" when we used this profiler on .NET, and we came to the conclusion (which might be wrong!) that it was a Heisenberg effect - something caused by the act of profiling.

It's hard to see any logical reason why the timing from the command line should be so different from the timing via the API, and so far we haven't been able to observe such an effect. But there are observations we can't currently explain, including a severe worsening of performance when we rebuild the code with IKVM 7; so all data points are useful.

Michael Kay
Saxonica


On 27 Sep 2013, at 07:43, Emanuel Wlaschitz wrote:

> The results are in. 30 minutes became 2 hours 40 minutes in the profiler, and
> showed...nothing really useful?
> One thing that looked odd was a fairly linear time distribution until one
> Mode.applyTemplates which split up into TextCopyOnlyRuleSet.process and
> ApplyTemplates+ApplyTemplatesPackage.processLeavingTail (with a weighting of
> about 65:35). The transform trace didn't show such results until very deep
> down where it came to a bunch of java methods from IKVM. Not sure if that
> means anything.
> The main difference is the distribution of "Waiting for synchronization" -
> 95% with the next trace being TinyTree.getNode at 0.7%.
>
> The question is still where this waiting comes from. I'll try the visual
> studio profiler next, as it has options and stuff for concurrency profiling.
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12299.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

O'Neil Delpratt
In reply to this post by Emanuel Wlaschitz
Hi emanuel,

I would like to reproduce the timing results you are getting. Please may
you detail me how you did the timing using transform.exe and the
Saxon.Api, respectively? Maybe you could send me the files privately if
they are sensitive.

Thanks and kind regards,

O'Neil

On 27/09/2013 07:43, Emanuel Wlaschitz wrote:

> The results are in. 30 minutes became 2 hours 40 minutes in the profiler, and
> showed...nothing really useful?
> One thing that looked odd was a fairly linear time distribution until one
> Mode.applyTemplates which split up into TextCopyOnlyRuleSet.process and
> ApplyTemplates+ApplyTemplatesPackage.processLeavingTail (with a weighting of
> about 65:35). The transform trace didn't show such results until very deep
> down where it came to a bunch of java methods from IKVM. Not sure if that
> means anything.
> The main difference is the distribution of "Waiting for synchronization" -
> 95% with the next trace being TinyTree.getNode at 0.7%.
>
> The question is still where this waiting comes from. I'll try the visual
> studio profiler next, as it has options and stuff for concurrency profiling.
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12299.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help
>
>
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2014.0.4117 / Virus Database: 3604/6699 - Release Date: 09/25/13
>
>


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
Hi O'Neil,
the Api snippet is attached to my first post, and the transform.exe results basically just mimic what transform.exe does (as cmd/Transform.cs shows):

----------------
public static void TransformDirect(string sourcePath, string stylesheetPath, string outputPath)
{
    new net.sf.saxon.Transform.doTransform(new[] { "-s:" + sourcePath, "-xsl:" + stylesheetPath, "-o:" + outputPath }, "transform");
}
----------------

Both of them wrapped up in a simple command line application that doesn't do anything else than calling either Transform or TransformDirect with sample file paths taken as command line options. It prints the starting and ending time, along with a stopwatch to show the total duration:

----------------
static void Main(string[] args)
{
    if (args.Length < 2 || args.Length > 3)
    {
        Console.WriteLine("Usage: SaxonTransform input stylesheet [output]");
        return;
    }
    try
    {
        string sourcePath = Path.GetFullPath(args[0]);
        string stylesheetPath = Path.GetFullPath(args[1]);
        string outputPath = args.Length == 3 ? Path.GetFullPath(args[2]) : Path.ChangeExtension(sourcePath, ".out.xml");
        Console.WriteLine("Starting transform at {0}", DateTime.Now);
        var sw = Stopwatch.StartNew();
        //TransformDirect(sourcePath, stylesheetPath, outputPath);
        Transform(sourcePath, stylesheetPath, outputPath);
        sw.Stop();
        Console.WriteLine("Finished transform at {0} (after {1})", DateTime.Now, sw.Elapsed);
    }
    catch (Exception ex)
    {
        Console.WriteLine("Unhandled exception: {0}", ex);
    }
}
----------------

For the results attached earlier as XML/HTML, I used the DocBook 5 definitive guide (which is available at http://sourceforge.net/p/docbook/code/HEAD/tree/trunk/defguide5/en/ - the only thing I did was resolve the XIncludes and flatten the file to have a single XML module to transform) with the official XSLT 2.0 stylesheets for XSL:Fo output (available at https://github.com/docbook/xslt20-stylesheets/tree/master/xslt/base/fo).
The plus is that it runs fairly quick (<10 seconds) and shows the issue at the same time; while my other case takes hours under a profiler (even without one it takes around 30 minutes with the Api case).

I'll see what I can do about the other case and come back to you.

Regards, Emanuel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
I just found something new in my tests.
The main difference between the transform.exe code and my code from the first post here is the way the input file is used. transform.exe just passes the filename and uses it (more or less) directly. My Saxon.Api code uses a DocumentBuilder to create an InitialContextNode.

Depending on input document size, I can get noticable improvements in performance by directly using the input stream as follows:
----------------------------------------
transformer.InputXmlResolver = new DtdIgnoringResolver(transformer.InputXmlResolver); //required to ignore DTDs in System IDs
transformer.SetInputStream(source, sourceUri);
----------------------------------------
instead of
----------------------------------------
var builder = _processor.NewDocumentBuilder();
builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver); //required to ignore DTDs in System IDs
builder.BaseUri = sourceUri;
transformer.InitialContextNode = builder.Build(source);
----------------------------------------

Not sure if the XdmNode one is supposed to be way slower, but it at least provides a way to speed things up if the InitialContextNode is the root node anyways.
I can't exactly remember why I choose to use the DocumentBuilder instead, but I guess I just didn't notice that the transformer also has a way to set the XmlResolver for the input document.

Regards, Emanuel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay

Thanks for this. I was looking at this with O'Neil on Friday and suggested he explore the question of how the source document was built. My first thought was that the call on buildDocument might be generating a DOM rather than a TinyTree, but that does not appear to be the case. If it's not that, then I can't currently see why there's such a big difference, but we'll investigate this avenue further.

Michael Kay
Saxonica

On 30 Sep 2013, at 10:24, Emanuel Wlaschitz wrote:

> I just found something new in my tests.
> The main difference between the transform.exe code and my code from the
> first post here is the way the input file is used. transform.exe just passes
> the filename and uses it (more or less) directly. My Saxon.Api code uses a
> DocumentBuilder to create an InitialContextNode.
>
> Depending on input document size, I can get noticable improvements in
> performance by directly using the input stream as follows:
> ----------------------------------------
> transformer.InputXmlResolver = new
> DtdIgnoringResolver(transformer.InputXmlResolver); //required to ignore DTDs
> in System IDs
> transformer.SetInputStream(source, sourceUri);
> ----------------------------------------
> instead of
> ----------------------------------------
> var builder = _processor.NewDocumentBuilder();
> builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver);
> //required to ignore DTDs in System IDs
> builder.BaseUri = sourceUri;
> transformer.InitialContextNode = builder.Build(source);
> ----------------------------------------
>
> Not sure if the XdmNode one is supposed to be way slower, but it at least
> provides a way to speed things up if the InitialContextNode is the root node
> anyways.
> I can't exactly remember why I choose to use the DocumentBuilder instead,
> but I guess I just didn't notice that the transformer also has a way to set
> the XmlResolver for the input document.
>
> Regards, Emanuel
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12305.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay
In reply to this post by Emanuel Wlaschitz
Just to give you an update on this, we've run your docbook example, and we are getting anomalous results too - but they are different from yours. In our first attempt, it was the API version that ran fast and the command line version that ran slow. We got different results when we made some configuration changes, but we currently have no idea what is causing these effects. The investigation continues.

Michael Kay
Saxonica

On 30 Sep 2013, at 10:24, Emanuel Wlaschitz wrote:

> I just found something new in my tests.
> The main difference between the transform.exe code and my code from the
> first post here is the way the input file is used. transform.exe just passes
> the filename and uses it (more or less) directly. My Saxon.Api code uses a
> DocumentBuilder to create an InitialContextNode.
>
> Depending on input document size, I can get noticable improvements in
> performance by directly using the input stream as follows:
> ----------------------------------------
> transformer.InputXmlResolver = new
> DtdIgnoringResolver(transformer.InputXmlResolver); //required to ignore DTDs
> in System IDs
> transformer.SetInputStream(source, sourceUri);
> ----------------------------------------
> instead of
> ----------------------------------------
> var builder = _processor.NewDocumentBuilder();
> builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver);
> //required to ignore DTDs in System IDs
> builder.BaseUri = sourceUri;
> transformer.InitialContextNode = builder.Build(source);
> ----------------------------------------
>
> Not sure if the XdmNode one is supposed to be way slower, but it at least
> provides a way to speed things up if the InitialContextNode is the root node
> anyways.
> I can't exactly remember why I choose to use the DocumentBuilder instead,
> but I guess I just didn't notice that the transformer also has a way to set
> the XmlResolver for the input document.
>
> Regards, Emanuel
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12305.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
Thats really interresting. I mean, I've asked some colleagues to run the same samples and got different results aswell, but in all cases the Api was slower.

To be precise (and as stated earlier), using InitialContextNode was slower in all cases compared to SetInputStream. Some were seeing a difference of 1 vs. 12 seconds, others 5 vs. 8 and another one saw 2 vs. 5 seconds - all in favor of the Stream and/or command line version. All on different machines and hardware configurations (one of them even in a virtual machine running on top of OSX; surprisingly that was the 2/5sec result).

Maybe just a shot into the blue, but from the profiling results I noticed a difference in how often certain methods of the stylesheet processor were called (such as NameTest.matches(TinyTree, int); ~1.1G times in the fast profile vs. only 460M times in the slow one). Other methods (such as AnyNodeTest.matches(TinyTree, int)) heavily appear in the slow profile (about 3G times), but rarely (about 550k times) in the fast one (which uses the same input and stylesheet!).
Could it be that the input source causes a different kind of optimization, or rather the fact that the Api sets certain flags (or not) compared to the command line?

Regards, Emanuel

Michael Kay wrote
Just to give you an update on this, we've run your docbook example, and we are getting anomalous results too - but they are different from yours. In our first attempt, it was the API version that ran fast and the command line version that ran slow. We got different results when we made some configuration changes, but we currently have no idea what is causing these effects. The investigation continues.

Michael Kay
Saxonica
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay
O'Neil has now run this again with a Saxon-HE build, rather than (Saxon-EE with no license file) (which we would expect to behave the same way), and this reproduces your results. He's also confirmed the difference if a stream is used as input rather than building the tree. More data points, but no resolution as yet.

Michael Kay
Saxonica

On 3 Oct 2013, at 07:19, Emanuel Wlaschitz wrote:

> Thats really interresting. I mean, I've asked some colleagues to run the same
> samples and got different results aswell, but in all cases the Api was
> slower.
>
> To be precise (and as stated earlier), using InitialContextNode was slower
> in all cases compared to SetInputStream. Some were seeing a difference of 1
> vs. 12 seconds, others 5 vs. 8 and another one saw 2 vs. 5 seconds - all in
> favor of the Stream and/or command line version. All on different machines
> and hardware configurations (one of them even in a virtual machine running
> on top of OSX; surprisingly that was the 2/5sec result).
>
> Maybe just a shot into the blue, but from the profiling results I noticed a
> difference in how often certain methods of the stylesheet processor were
> called (such as NameTest.matches(TinyTree, int); ~1.1G times in the fast
> profile vs. only 460M times in the slow one). Other methods (such as
> AnyNodeTest.matches(TinyTree, int)) heavily appear in the slow profile
> (about 3G times), but rarely (about 550k times) in the fast one (which uses
> the same input and stylesheet!).
> Could it be that the input source causes a different kind of optimization,
> or rather the fact that the Api sets certain flags (or not) compared to the
> command line?
>
> Regards, Emanuel
>
>
> Michael Kay wrote
>> Just to give you an update on this, we've run your docbook example, and we
>> are getting anomalous results too - but they are different from yours. In
>> our first attempt, it was the API version that ran fast and the command
>> line version that ran slow. We got different results when we made some
>> configuration changes, but we currently have no idea what is causing these
>> effects. The investigation continues.
>>
>> Michael Kay
>> Saxonica
>
>
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12313.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay
In reply to this post by Emanuel Wlaschitz
We've established why the DocumentBuilder approach is slower: the stylesheet uses xsl;strip-space, and it's a lot more efficient to strip unwanted whitespace while building the tree than to "ignore" it dynamically while navigating the tree.

As it happens, my most recent blog posting is on this subject:

http://dev.saxonica.com/blog/mike/2013/09/

We're still taking measurements to establish whether this accounts for the whole of the observed effects; I'm not convinced it does.

I have to say I was a little disappointed by  the profiling tools we've been using (Redgate). It gives you masses of data and it's very easy to explore it, but at the end of the day it didn't lead us to the solution; in fact, it took us down a lot of unproductive paths.

Michael Kay
Saxonica

On 30 Sep 2013, at 10:24, Emanuel Wlaschitz wrote:

> I just found something new in my tests.
> The main difference between the transform.exe code and my code from the
> first post here is the way the input file is used. transform.exe just passes
> the filename and uses it (more or less) directly. My Saxon.Api code uses a
> DocumentBuilder to create an InitialContextNode.
>
> Depending on input document size, I can get noticable improvements in
> performance by directly using the input stream as follows:
> ----------------------------------------
> transformer.InputXmlResolver = new
> DtdIgnoringResolver(transformer.InputXmlResolver); //required to ignore DTDs
> in System IDs
> transformer.SetInputStream(source, sourceUri);
> ----------------------------------------
> instead of
> ----------------------------------------
> var builder = _processor.NewDocumentBuilder();
> builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver);
> //required to ignore DTDs in System IDs
> builder.BaseUri = sourceUri;
> transformer.InitialContextNode = builder.Build(source);
> ----------------------------------------
>
> Not sure if the XdmNode one is supposed to be way slower, but it at least
> provides a way to speed things up if the InitialContextNode is the root node
> anyways.
> I can't exactly remember why I choose to use the DocumentBuilder instead,
> but I guess I just didn't notice that the transformer also has a way to set
> the XmlResolver for the input document.
>
> Regards, Emanuel
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287p12305.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Michael Kay
In reply to this post by Emanuel Wlaschitz
We've now satisfied ourselves that the performance anomalies were entirely due to the whitespace-stripping differences: that is, in the API case you were constructing a tree containing all the whitespace, whereas in the command line case the whitespace is stripped during the process of tree construction.

We were being seriously misled for a while by the fact that the profiler results seemed to be heavily distorted by calls to system date/time functions. We eventually got to the bottom of the issue when we inserted our own counters to see what methods were being executed how often.

We're seeing execution times on .NET that are typically anything from 25% to 50% (but in one case 100%) higher than on Java; we think that's probably reasonable.

The overhead of "dynamic space stripping" on .NET was significantly higher than on Java (this is the case where we had the 100% factor). My instinct would be that this has something to do with the cost of allocating and garbage-collecting large numbers of short-life objects.

Michael Kay
Saxonica


On 25 Sep 2013, at 13:48, Emanuel Wlaschitz wrote:

> It seems I just came across a similar performance difference as someone
> already noticed 5 years ago in
> http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/net-components-performance-question-td8223.html
>
> Running a transformation using transform.exe takes about 5 minutes for a
> reasonably complex stylesheet (about 500kb of source, split into roughly 35
> included modules) and a 5 MB xml file.
> The same thing with Saxon.Api takes about 30 Minutes for no appearent
> reason.
>
> As this is Saxon-HE, the old suggestion of using allNodesUntyped does not
> apply (it seemed to have been removed with the move from Saxon-B to Saxon-HE
> somewhere around 9.2).
> Anything else I could do, short of using the Java-style classes directly
> (like transform.exe does)?
>
> I'm using Saxon-HE 9.4.0.6N on .NET 3.5SP1, in case it makes a difference.
> Ok, latest is 9.4.0.9, but I doubt it makes much of a difference.
>
> Regards, Emanuel
>
> ---------
> //relevant source, in case it matters:
> private readonly Processor _processor = new Processor(false);
> public void Transform(string sourcePath, string stylesheetPath, string
> outputPath)
> {
> using (var source = File.OpenRead(sourcePath))
> using (var stylesheet = File.OpenRead(stylesheetPath))
> {
> Uri stylesheetUri, sourceUri;
> if (!Uri.TryCreate(sourcePath, UriKind.RelativeOrAbsolute, out sourceUri))
> throw new ArgumentException(string.Format("Could not create an URI from
> source path '{0}'.", sourcePath), "sourcePath");
> if (!Uri.TryCreate(stylesheetPath, UriKind.RelativeOrAbsolute, out
> stylesheetUri))
> throw new ArgumentException(string.Format("Could not create an URI from
> stylesheet path '{0}'.", stylesheetPath), "stylesheetPath");
>
> var compiler = _processor.NewXsltCompiler();
>
> compiler.BaseUri = stylesheetUri; //required to resolve stylesheet
> includes
> compiler.ErrorList = new ArrayList();
> var executable = compiler.Compile(stylesheet);
> var transformer = executable.Load();
> var builder = _processor.NewDocumentBuilder();
> builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver);
> //required to ignore DTDs in System IDs
> builder.BaseUri = sourceUri;
> transformer.InitialContextNode = builder.Build(source);
> var destination = new Serializer();
> using (var output = File.Open(outputPath, FileMode.Create))
> {
> destination.SetOutputStream(output);
> transformer.Run(destination);
> }
> }
> }
>
>
>
> --
> View this message in context: http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help 


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

Emanuel Wlaschitz
Wow, you really got you crystal ball at work here; the stylesheet we use does infact use <xsl:strip-space elements="*"/> (as we had some interresting issues with insignificant whitespace being treated as significant ones after rendering once).
I did see the blob-post recently, but now I see how it applies to my problem. The second-last paragraph applies pretty much exactly to my case; I simply didn't know that it would make that much of a difference.

Maybe one thing about RedGate: I've been using their Performance- and Memory-Profiler for quite a few years, and apart from this case, they always found me a place to optimize.
And, infact, I also saw your last assumption; namely short-lived objects and gc-overhead - but I didn't know where to put it as I don't know the internals of Saxon that well.

And one last question: strip-space * probably isn't the best solution, would it make a difference if we specified the element names instead? I'd have to talk to my colleague about specifics, but the strip-space is defined in a module called tables.xsl, and the issue I remember was about table cells (<entry/>) with mixed PCDATA and paragraphs (<para/>) inside. It probably wouldn't make much sense to strip whitespace from  or <row/> as they can never directly contain character data (then again, this would explain why O'Neil saw different results with Saxon-EE; he probably had a DTD or Schema available that told Saxon exactly that).

Either way, thanks for your (and O'Neils) efforts on analyzing this! I'll brief my colleague about those findings and see if we can do something about the strip-space stuff.

Regards, Emanuel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api

alan.painter-2
In reply to this post by Michael Kay
Hi Michael,

You wrote:

>> We're seeing execution times on .NET that are typically anything from 25% to 50% (but in one case 100%) higher than on Java; we think that's probably reasonable.


I'm curious if this is a general statement, that saxonica execution under .NET give better performance than under Java?

thanks for any clarification

Alan PAINTER
Team Colossus | Strategic Components
15, rue Vernet, 75419 Paris, France
_____________________________________________

Phone  +33 (0) 1 40 70 31 19
Mobile +33 (0) 6 75 13 29 29
Email alan.painter@...


_____________________________________________





De :        Michael Kay <[hidden email]>
A :        Mailing list for the SAXON XSLT and XQuery processor <[hidden email]>
Date :        10/04/2013 07:00 PM
Objet :        Re: [saxon] Saxon-HE 9.4N performance - transform.exe vs. Saxon.Api




We've now satisfied ourselves that the performance anomalies were entirely due to the whitespace-stripping differences: that is, in the API case you were constructing a tree containing all the whitespace, whereas in the command line case the whitespace is stripped during the process of tree construction.

We were being seriously misled for a while by the fact that the profiler results seemed to be heavily distorted by calls to system date/time functions. We eventually got to the bottom of the issue when we inserted our own counters to see what methods were being executed how often.

We're seeing execution times on .NET that are typically anything from 25% to 50% (but in one case 100%) higher than on Java; we think that's probably reasonable.

The overhead of "dynamic space stripping" on .NET was significantly higher than on Java (this is the case where we had the 100% factor). My instinct would be that this has something to do with the cost of allocating and garbage-collecting large numbers of short-life objects.

Michael Kay
Saxonica


On 25 Sep 2013, at 13:48, Emanuel Wlaschitz wrote:

> It seems I just came across a similar performance difference as someone
> already noticed 5 years ago in
>
http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/net-components-performance-question-td8223.html
>
> Running a transformation using transform.exe takes about 5 minutes for a
> reasonably complex stylesheet (about 500kb of source, split into roughly 35
> included modules) and a 5 MB xml file.
> The same thing with Saxon.Api takes about 30 Minutes for no appearent
> reason.
>
> As this is Saxon-HE, the old suggestion of using allNodesUntyped does not
> apply (it seemed to have been removed with the move from Saxon-B to Saxon-HE
> somewhere around 9.2).
> Anything else I could do, short of using the Java-style classes directly
> (like transform.exe does)?
>
> I'm using Saxon-HE 9.4.0.6N on .NET 3.5SP1, in case it makes a difference.
> Ok, latest is 9.4.0.9, but I doubt it makes much of a difference.
>
> Regards, Emanuel
>
> ---------
> //relevant source, in case it matters:
> private readonly Processor _processor = new Processor(false);
> public void Transform(string sourcePath, string stylesheetPath, string
> outputPath)
> {
>                  using (var source = File.OpenRead(sourcePath))
>                  using (var stylesheet = File.OpenRead(stylesheetPath))
>                  {
>                                   Uri stylesheetUri, sourceUri;
>                                   if (!Uri.TryCreate(sourcePath, UriKind.RelativeOrAbsolute, out sourceUri))
>                                                    throw new ArgumentException(string.Format("Could not create an URI from
> source path '{0}'.", sourcePath), "sourcePath");
>                                   if (!Uri.TryCreate(stylesheetPath, UriKind.RelativeOrAbsolute, out
> stylesheetUri))
>                                                    throw new ArgumentException(string.Format("Could not create an URI from
> stylesheet path '{0}'.", stylesheetPath), "stylesheetPath");
>
>                                   var compiler = _processor.NewXsltCompiler();
>
>                                   compiler.BaseUri = stylesheetUri; //required to resolve stylesheet
> includes
>                                   compiler.ErrorList = new ArrayList();
>                                   var executable = compiler.Compile(stylesheet);
>                                   var transformer = executable.Load();
>                                   var builder = _processor.NewDocumentBuilder();
>                                   builder.XmlResolver = new DtdIgnoringResolver(builder.XmlResolver);
> //required to ignore DTDs in System IDs
>                                   builder.BaseUri = sourceUri;
>                                   transformer.InitialContextNode = builder.Build(source);
>                                   var destination = new Serializer();
>                                   using (var output = File.Open(outputPath, FileMode.Create))
>                                   {
>                                                    destination.SetOutputStream(output);
>                                                    transformer.Run(destination);
>                                   }
>                  }
> }
>
>
>
> --
> View this message in context:
http://saxon-xslt-and-xquery-processor.13853.n7.nabble.com/Saxon-HE-9-4N-performance-transform-exe-vs-Saxon-Api-tp12287.html
> Sent from the saxon-help mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
>
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> saxon-help mailing list archived at
http://saxon.markmail.org/
> [hidden email]
>
https://lists.sourceforge.net/lists/listinfo/saxon-help


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at
http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help


Ensemble adoptons des gestes responsables : N'imprimez ce mail que si necessaire. Les informations contenues dans ce message et les pieces jointes (ci-apres denomme le message) sont confidentielles et peuvent etre couvertes par le secret professionnel. Si vous n'etes pas le destinataire de ce message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez recu ce message par erreur, nous vous remercions de le supprimer de votre systeme, ainsi que toutes ses copies, et d'en avertir immediatement HSBC France et ses filiales par message de retour. Il est impossible de garantir que les communications par messagerie electronique arrivent en temps utile, sont securisees ou denuees de toute erreur, alteration, falsification ou virus. En consequence, HSBC France et ses filiales declinent toute responsabilite du fait des erreurs, alterations, falsifications ou omissions qui pourraient en resulter. Consider the environment before printing this mail. The information contained in this e-mail is confidential. It may also be legally privileged. If you are not the addressee you may not copy, forward, disclose or use any part of it. If you have received this message by error, please delete it and all copies from your system and notify the sender immediately by return e-mail. E-mail communications cannot be guaranteed to be timely secure, error or virus-free. The sender does not accept liability for any errors or omissions which arise as a result.
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
12
Loading...