Re: Silly issue: NullPointerException when processing

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Silly issue: NullPointerException when processing

Marcin Domański

Thanks Michael,
   I am aware that there is an impact, but was not aware it is even up to 10 times slower. I am using DOM tree as an input, because I have a code that I cannot easily modify (a shared code), that is using STaX (esp. XMLOutputFactory) to generate a document that is(among other sources) a param of xml transformation. I would need to force it to generate Saxon's native tree format while still using generic (JAXP) API, then pass it to transformation logic. Right now the shared code is as follows:

            XMLOutputFactory xmlof = XMLOutputFactory.newInstance();
            XMLStreamWriter xmlw = xmlof.createXMLStreamWriter(domResult);
            xmlw.writeStartDocument();
            ...

My only idea right now is to negotiate to refactor this code to provide XMLStreamWriter by the caller, and then pass the StreamWriterToReciever object to build Saxon tree object, then provide it as a source. We might consider refactoring in next development iteration...

Let me know if you were able to reproduce the problem or find any workaround...

Cheers,
   Marcin

2015-06-21 10:18 GMT+02:00 <[hidden email]>:
Date: Sun, 21 Jun 2015 09:18:01 +0100
From: Michael Kay <[hidden email]>
Subject: Re: [saxon] Silly issue: NullPointerException when processing
        self-closing element.
To: Mailing list for the SAXON XSLT and XQuery processor
        <[hidden email]>
Message-ID: <[hidden email]>
Content-Type: text/plain; charset="utf-8"

Thanks for reporting it. It does seem a little surprising that it should fail on what would appear to be a common path.

The specification here is unclear: Node.getNodeValue() for a text node says it returns ?the same as CharacterData.data?, and CharacterData.getData() gives no indication that null is an acceptable return value. But it?s not unusual that the DOM specification is unclear, that?s something we have to live with. We?ll see if we can reproduce it.

I trust you are aware that using a DOM tree as input to your transformation, rather than using Saxon?s native tree format, slows things down by a factor of 5 to 10.

You can track progress on the issue here: https://saxonica.plan.io/issues/2407

Michael Kay
Saxonica


> On 21 Jun 2015, at 00:17, Marcin Doma?ski <[hidden email]> wrote:
>
> Hi there,
>    I was using Saxon with success for a few years or so, until I stumbled upon a problem that should virtually not exist. I was again using Saxon with a transformation like this:
>
> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform <http://www.w3.org/1999/XSL/Transform>" version="2.0">
>   <xsl:output method="xml"/>
>   <xsl:param name="activity-de0ab587-5f1c-9889-c31e-0697250ef9fb"/>
>   <xsl:template match="/">
>     <documentContent xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance <http://www.w3.org/2001/XMLSchema-instance>">
>       <xsl:for-each select="$activity-de0ab587-5f1c-9889-c31e-0697250ef9fb/output/row">
>         <row>
>           <c>
>             <xsl:value-of select="col[@name='MainCategory']"/>
>           </c>
>           <c>
>             <xsl:value-of select="col[@name='ItemCode']"/>
>           </c>
>           <c>
>             <xsl:value-of select="col[@name='ItemName']"/>
>           </c>
>           <c>
>             <xsl:value-of select="col[@name='LongDescription']"/>
>           </c>
>           <c>
>             <xsl:value-of select="col[@name='Price']"/>
>           </c>
>           <c>
>             <xsl:value-of select="col[@name='CostPrice']"/>
>           </c>
>           <c>
>             <xsl:value-of select="col[@name='UOM']"/>
>           </c>
>           <c>
>             <xsl:value-of select="col[@name='Inventory']"/>
>           </c>
>         </row>
>       </xsl:for-each>
>     </documentContent>
>   </xsl:template>
> </xsl:stylesheet>
>
> I was very surprised when I started getting NullPointerExceptions when elements being selected were empty (self-closing in this case). Example param value:
>
> <output>
> <row>
>   <col name="MainCategory">
>   ABC
> </col>
> <col name="ItemCode">40257
> </col>
> <col name="ItemName">IName
> </col>
> <col name="LongDescription"/>
> <col name="Price">58.12
> </col>
> <col name="CostPrice">-0.0
> </col>
> <col name="UOM">pcs
> </col>
> <col name="Inventory">68.0
> </col>
> </row>
> </output>
>
>
> The stack trace is as follows:
>
> Caused by: java.lang.RuntimeException: Internal error evaluating template  at line 4 in module
>         at net.sf.saxon.expr.instruct.Template.applyLeavingTail(Template.java:349) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:1124) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.Controller.transformDocument(Controller.java:2106) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.Controller.transform(Controller.java:1705) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.s9api.XsltTransformer.transform(XsltTransformer.java:547) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.jaxp.TransformerImpl.transform(TransformerImpl.java:177) ~[Saxon-HE-9.6.0-6.jar:na]
>         ...
>         ... 4 common frames omitted
> Caused by: java.lang.NullPointerException: null
>         at net.sf.saxon.tree.util.FastStringBuffer.append(FastStringBuffer.java:68) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.dom.DOMNodeWrapper.expandStringValue(DOMNodeWrapper.java:337) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.dom.DOMNodeWrapper.getStringValueCS(DOMNodeWrapper.java:283) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.tree.wrapper.AbstractNodeWrapper.atomize(AbstractNodeWrapper.java:104) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.tree.iter.UntypedAtomizingIterator.next(UntypedAtomizingIterator.java:53) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.tree.iter.UntypedAtomizingIterator.next(UntypedAtomizingIterator.java:32) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.ItemMappingIterator.next(ItemMappingIterator.java:90) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.functions.FoldingFunction.evaluateItem(FoldingFunction.java:56) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.Expression.evaluateAsString(Expression.java:524) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.SimpleNodeConstructor.processLeavingTail(SimpleNodeConstructor.java:216) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ValueOf.processLeavingTail(ValueOf.java:280) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:144) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:450) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:389) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.Block.processLeavingTail(Block.java:669) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:144) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:450) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:389) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:144) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ForEach.processLeavingTail(ForEach.java:419) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:144) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:450) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:389) ~[Saxon-HE-9.6.0-6.jar:na]
>         at net.sf.saxon.expr.instruct.Template.applyLeavingTail(Template.java:336) ~[Saxon-HE-9.6.0-6.jar:na]
>         ... 12 common frames omitted
>
> After debug, it seems the null pointer exception is being thrown in the FastStringBuffer because it is being fed with content of #text node that is null in self-closing elements (as parsed by Xerces).
>
> So this seems silly as self-closing elements are popular and it is quite unlikely, that Saxon has a but that prevents processing such elements in xslt.  I am overriding two values:
>
>         System.setProperty("javax.xml.transform.TransformerFactory", "net.sf.saxon.TransformerFactoryImpl");
>         System.setProperty(XPathFactory.DEFAULT_PROPERTY_NAME + ":" + XPathFactory.DEFAULT_OBJECT_MODEL_URI, "net.sf.saxon.xpath.XPathFactoryImpl");
>
> Everything else is generic (I am using java.xml.transform API).
>
> Does anybody know what I am missing? Any help is appreciated...
>
> Marcin
> ------------------------------------------------------------------------------
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help
*******************************************


------------------------------------------------------------------------------

_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|

Re: Silly issue: NullPointerException when processing

Michael Kay

> On 21 Jun 2015, at 19:51, Marcin Domański <[hidden email]> wrote:
>
>
> Thanks Michael,
>    I am aware that there is an impact, but was not aware it is even up to 10 times slower. I am using DOM tree as an input, because I have a code that I cannot easily modify (a shared code), that is using STaX (esp. XMLOutputFactory) to generate a document that is(among other sources) a param of xml transformation. I would need to force it to generate Saxon's native tree format while still using generic (JAXP) API, then pass it to transformation logic. Right now the shared code is as follows:
>
>             XMLOutputFactory xmlof = XMLOutputFactory.newInstance();
>             XMLStreamWriter xmlw = xmlof.createXMLStreamWriter(domResult);
>             xmlw.writeStartDocument();
>             ...
>
> My only idea right now is to negotiate to refactor this code to provide XMLStreamWriter by the caller, and then pass the StreamWriterToReciever object to build Saxon tree object, then provide it as a source.

You could certainly consider using

PipelineConfiguration pipe = getConfiguration().makePipelineConfiguration()
Builder builder = TreeModel.TINY_TREE.newBuilder(pipe);
XMLStreamWriter xmlw = new StreamWriterToReceiver(builder);

though note that there have been a few issues discovered recently about weak conformance of the XMLStreamWriter returned, especially if you try to do clever things with namespace bindings. If you always supply namespace, prefix, and localName on all startElement and attribute calls then it should be OK.

(These are partly though not entirely because XMLStreamWriter, like much of JAXP, is not particularly well specified)

Michael Kay
Saxonica
------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|

Re: Silly issue: NullPointerException when processing

Marcin Domański
In reply to this post by Marcin Domański
Michael,
   I analyzed further the NullPointerException issue. It seems that following code will generate DOM tree that throws NPE in Saxon:

xmlStreamWriter.writeStartElement("ABC");
xmlStreamWriter.writeCharacters(null);
xmlStreamWriter.writeEndElement();

while this will not generate NPE:

xmlStreamWriter.writeStartElement("ABC");
xmlStreamWriter.writeEndElement();

 In our case there is no null checking in the writeCharacters method invocation, therefore if there is null data it generates NPE in Saxon, but if dump both trees to String they look the same.

Marcin


2015-06-22 11:25 GMT+02:00 <[hidden email]>:
From: Michael Kay <[hidden email]>
To: Mailing list for the SAXON XSLT and XQuery processor <[hidden email]>
Cc: 
Date: Mon, 22 Jun 2015 10:25:25 +0100
Subject: Re: [saxon] Silly issue: NullPointerException when processing

> On 21 Jun 2015, at 19:51, Marcin Domański <[hidden email]> wrote:
>
>
> Thanks Michael,
>    I am aware that there is an impact, but was not aware it is even up to 10 times slower. I am using DOM tree as an input, because I have a code that I cannot easily modify (a shared code), that is using STaX (esp. XMLOutputFactory) to generate a document that is(among other sources) a param of xml transformation. I would need to force it to generate Saxon's native tree format while still using generic (JAXP) API, then pass it to transformation logic. Right now the shared code is as follows:
>
>             XMLOutputFactory xmlof = XMLOutputFactory.newInstance();
>             XMLStreamWriter xmlw = xmlof.createXMLStreamWriter(domResult);
>             xmlw.writeStartDocument();
>             ...
>
> My only idea right now is to negotiate to refactor this code to provide XMLStreamWriter by the caller, and then pass the StreamWriterToReciever object to build Saxon tree object, then provide it as a source.

You could certainly consider using

PipelineConfiguration pipe = getConfiguration().makePipelineConfiguration()
Builder builder = TreeModel.TINY_TREE.newBuilder(pipe);
XMLStreamWriter xmlw = new StreamWriterToReceiver(builder);

though note that there have been a few issues discovered recently about weak conformance of the XMLStreamWriter returned, especially if you try to do clever things with namespace bindings. If you always supply namespace, prefix, and localName on all startElement and attribute calls then it should be OK.

(These are partly though not entirely because XMLStreamWriter, like much of JAXP, is not particularly well specified)

Michael Kay
Saxonica


------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
saxon-help mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help
The saxon-help list is archived at http://saxon.markmail.org/


------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help 
Reply | Threaded
Open this post in threaded view
|

Re: Silly issue: NullPointerException when processing

Michael Kay
I suspected something a bit like this. The Javadoc for XMLStreamWriter is hopelessly casual, in that it doesn’t say what the effect of passing a null argument is, but to create a DOM that has a text node whose string value is null seems a pretty crass interpretation; Saxon is then reading a DOM with a text node whose string value is null, and I don’t think that’s a valid state for the DOM.

We get all sorts of mess like this when trying to read a DOM tree that’s been written programmatically; there are very few checks that the DOM satisfies a reasonable set of constraints.

Michael Kay
Saxonica


On 22 Jun 2015, at 10:39, Marcin Domański <[hidden email]> wrote:

Michael,
   I analyzed further the NullPointerException issue. It seems that following code will generate DOM tree that throws NPE in Saxon:

xmlStreamWriter.writeStartElement("ABC");
xmlStreamWriter.writeCharacters(null);
xmlStreamWriter.writeEndElement();

while this will not generate NPE:

xmlStreamWriter.writeStartElement("ABC");
xmlStreamWriter.writeEndElement();

 In our case there is no null checking in the writeCharacters method invocation, therefore if there is null data it generates NPE in Saxon, but if dump both trees to String they look the same.

Marcin


2015-06-22 11:25 GMT+02:00 <[hidden email]>:
From: Michael Kay <[hidden email]>
To: Mailing list for the SAXON XSLT and XQuery processor <[hidden email]>
Cc: 
Date: Mon, 22 Jun 2015 10:25:25 +0100
Subject: Re: [saxon] Silly issue: NullPointerException when processing

> On 21 Jun 2015, at 19:51, Marcin Domański <[hidden email]> wrote:
>
>
> Thanks Michael,
>    I am aware that there is an impact, but was not aware it is even up to 10 times slower. I am using DOM tree as an input, because I have a code that I cannot easily modify (a shared code), that is using STaX (esp. XMLOutputFactory) to generate a document that is(among other sources) a param of xml transformation. I would need to force it to generate Saxon's native tree format while still using generic (JAXP) API, then pass it to transformation logic. Right now the shared code is as follows:
>
>             XMLOutputFactory xmlof = XMLOutputFactory.newInstance();
>             XMLStreamWriter xmlw = xmlof.createXMLStreamWriter(domResult);
>             xmlw.writeStartDocument();
>             ...
>
> My only idea right now is to negotiate to refactor this code to provide XMLStreamWriter by the caller, and then pass the StreamWriterToReciever object to build Saxon tree object, then provide it as a source.

You could certainly consider using

PipelineConfiguration pipe = getConfiguration().makePipelineConfiguration()
Builder builder = TreeModel.TINY_TREE.newBuilder(pipe);
XMLStreamWriter xmlw = new StreamWriterToReceiver(builder);

though note that there have been a few issues discovered recently about weak conformance of the XMLStreamWriter returned, especially if you try to do clever things with namespace bindings. If you always supply namespace, prefix, and localName on all startElement and attribute calls then it should be OK.

(These are partly though not entirely because XMLStreamWriter, like much of JAXP, is not particularly well specified)

Michael Kay
Saxonica


------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
saxon-help mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help
The saxon-help list is archived at http://saxon.markmail.org/

------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help


------------------------------------------------------------------------------
Monitor 25 network devices or servers for free with OpManager!
OpManager is web-based network management software that monitors
network devices and physical & virtual servers, alerts via email & sms
for fault. Monitor 25 devices for free with no restriction. Download now
http://ad.doubleclick.net/ddm/clk/292181274;119417398;o
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help