alternative to using element constructors for extracting info from multiple elements

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

alternative to using element constructors for extracting info from multiple elements

manavaputra
Hi

I am developing some framework using xquery
expressions to typically to extract info from
instances of a particular element, but some times we
also need info from other elements in the document,
typically the elements that happen to be in the path
for the leaf element and some times from elements that
do not fall in this path. So, i have been using FLWOR
exprs and element constructors for this purpose using
multiple for loops and capturing information from
different intermediate elements as attributes or
elements inside the constructed element. This was
pretty intutive and useful, but what i have been
noticing in my memory usage and performance tests is
that, element construction is using a lot of memory.

Are the constructed elements and documents not
represented in the saxon tree format? I paste below
one example of the difference in the memory
consumption. The computed elements are definitely much
smaller in size than the elements in the original
document.

Saxon Doc Load(1):  time=2 sec 884 msec,  mem=8M 782K
712 bytes

4819 Saxon Query run(1):  time=8 sec 422 msec,
mem=49M 122K 40 bytes

Is there an alternative to extract info from multiple
elements than having to construct a new element? I
would definitely need to save memory because for large
documents this kind of memory usage in queries would
not be viable. I have had an out-of-memory problem
when i was running my appication against a series of
large documents.

-Girish

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
saxon-help mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help
Reply | Threaded
Open this post in threaded view
|

RE: alternative to using element constructors for extracting info from multiple elements

Michael Kay
It's true that constructing temporary trees will generally use memory: how
much memory depends on the nature of the trees and the length of time that
they are retained. It's impossible for me to comment on the specifics of
your measurements as you haven't given me any information about the source
documents or the query, or indicated what your numbers actually mean.

I'd suggest you drill down a bit more deeply to the point where you can
relate the memory usage to specific things that your query is doing. This
would then be useful data to work with.

As an experiment, you might see whether the -pull option makes any
difference. This option increases the amount of pipelining/streaming by
reading trees as they are written. (It might have no effect, depending on
the details of the query, or it might increase time but decrease space.)

Michael Kay
http://www.saxonica.com/ 

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of
> [hidden email]
> Sent: 18 February 2006 02:26
> To: [hidden email]
> Subject: [saxon] alternative to using element constructors
> for extracting info from multiple elements
>
> Hi
>
> I am developing some framework using xquery
> expressions to typically to extract info from
> instances of a particular element, but some times we
> also need info from other elements in the document,
> typically the elements that happen to be in the path
> for the leaf element and some times from elements that
> do not fall in this path. So, i have been using FLWOR
> exprs and element constructors for this purpose using
> multiple for loops and capturing information from
> different intermediate elements as attributes or
> elements inside the constructed element. This was
> pretty intutive and useful, but what i have been
> noticing in my memory usage and performance tests is
> that, element construction is using a lot of memory.
>
> Are the constructed elements and documents not
> represented in the saxon tree format? I paste below
> one example of the difference in the memory
> consumption. The computed elements are definitely much
> smaller in size than the elements in the original
> document.
>
> Saxon Doc Load(1):  time=2 sec 884 msec,  mem=8M 782K
> 712 bytes
>
> 4819 Saxon Query run(1):  time=8 sec 422 msec,
> mem=49M 122K 40 bytes
>
> Is there an alternative to extract info from multiple
> elements than having to construct a new element? I
> would definitely need to save memory because for large
> documents this kind of memory usage in queries would
> not be viable. I have had an out-of-memory problem
> when i was running my appication against a series of
> large documents.
>
> -Girish
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com 
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep
> through log files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  
> DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&
dat=121642
> _______________________________________________
> saxon-help mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/saxon-help
>




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
saxon-help mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/saxon-help