8,487,480 members and growing!

Forums

General Programming

Product Lifecycle

Database and SysAdmin

General Discussions

Non-English Language

Bold indicates new messages since 6:42 14 Feb '12

XML / XSL

RSS:

Home > Forums > XML / XSL

You must Sign In to use this message board. (secure sign-in)

Search this forum
	Profile popups Noise Layout Per page Show

How to de-batch very large XMLDocument into bit size chunks

ONeil Tomlinson

8:39 28 Feb '10

I have a very large XMLDocument with 56,000 child records each having 8 child field elements. The application that i have written performs a complicated process on the XML. It can process 10,000 records in 5 minutes and 20,000 in 25 minutes (increases exponentially) but takes maybe 10 hours to process the 56,000 record (which is unacceptable). Whats the best way to de-batch this XML Document into bite size chunks (eg 10,000 and after 5 iteration will process 50,000. which will execute in 25-30 minutes processing time)

Thanks.

Re: How to de-batch very large XMLDocument into bit size chunks

daveyerwin

9:44 28 Feb '10

How many megabytes is your xml file? Does the "complicated process" operate on only one of the 56,000 children independantly, without refferencing the other children ? Show me a childNode and tell me about this "complicated process". How much free memory does your computer have available ?
Sign In·View Thread·Permalink

Re: How to de-batch very large XMLDocument into bit size chunks

ONeil Tomlinson

7:33 1 Mar '10

There is a sample of the child record. And there are 56,000 of them in the xml document. I can realy give you the full details of what the complicated process is doing but part of it is using the Muenchian Method grouping on 'cost_centre' and 'expense_code'.
The app is runningon a dev box 2.50GH, 2GB.
It has 1/2GB free when app is not running

<Transaction> 
<Entry>
    <payroll_id>payroll_id_0</payroll_id>
    <payroll_nm>payroll_nm_0</payroll_nm>
    <payRunDate>payRunDate_0</payRunDate>
    <payRunID>payRunID_0</payRunID>
    <empno>10</empno>
    <surname>surname_0</surname>
    <forename>forename_0</forename>
    <pos_ref>pos_ref_0</pos_ref>
    <pos_name>pos_name_0</pos_name>
    <element_id>element_id_0</element_id>
    <element_name>element_name_0</element_name>
    <cost_centre>c11</cost_centre>
    <expense_code>e11</expense_code>
    <debit_credit>debit_credit_0</debit_credit>
    <amount>10</amount>
    <signed_amount>10</signed_amount>
  </Entry>
<Entry>
....
 </Entry>
</Transaction>

Unclosed tr tag

John Jones II

5:35 26 Feb '10

Why is this invalid? <xsl:choose> <xsl:when test="position() mod 2 = 1"> <tr bgcolor="Silver"> </xsl:when> <xsl:otherwise> <tr> </xsl:otherwise> </xsl:choose> ... </tr> Internet Explorer reports the error "End tag 'xsl:when' does not match the start tag 'tr'"
Sign In·View Thread·Permalink	1.00/5 (1 vote)

Re: Unclosed tr tag

Richard MacCutchan

6:23 26 Feb '10

The line: <tr bgcolor="Silver"> is unclosed, so the next closing tag should be `</tr>` rather than `</xsl:when>`. txtspeak is the realm of 9 year old children, not developers. Christian Graus
Sign In·View Thread·Permalink

Re: Unclosed tr tag

Stuart Dootson

8:45 26 Feb '10

The previous answer told you why the error occurred - here's a solution: <tr> <xsl:if test="position() mod 2 = 1"> <xsl:attribute name="bgcolor">Silver</xsl:attribute> </xsl:if> ... </tr> Java, Basic, who cares - it's all a bunch of tree-hugging hippy crp* CodeProject MVP for 2010 - who'd'a thunk it!
Sign In·View Thread·Permalink

XML schema - Element name used to identify item problem... - need to use the element name to identify value when importing into SQL 2005 tables

VicarH

8:37 25 Feb '10

I am working on a project for a State Agency where the data is received from multiple private entities and loaded into SQL 2005 tables. The XML schema was designed and published before my involvement with the project and now it cannot be changed. One of the elements to receive (Charges) occurs multiple times within a client but each is for a different concept. Instead of using an attribute to identify the reason for the Charge, they used the element name to do it. For example:

<FERTILIZER_CHARGE>2300</FERTILIZER_CHARGE> -- 20 different charge types identified

instead of

<CHARGE TYPE=”FERTILIZER” AMOUNT=”2300” />

I need to move all the charges to a single table and identify each one with its type but cannot find any examples where the element name is used to update a column in a table. PLEASE HELP!!!

Display a image in picturebox

kpuneeth7

21:36 23 Feb '10

i just want to load one image to the picturebox, who's path is stored in xml file. please tell me how to do this. GIve examples.
Sign In·View Thread·Permalink

Re: Display a image in picturebox

Stuart Dootson

21:50 23 Feb '10

kpuneeth7 wrote:
GIve examples

Without knowing a little about the structure of your XML file and what technology you're using, that would be difficult to do...

But it's simply a case of reading the XML file with an XML parser (think XmlReader in .NET), and finding the node containing the image file's path.

The rest of it (displaying the image in the picture box) is nothing to do with XML.

Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p

CodeProject MVP for 2010 - who'd'a thunk it!

Re: Display a image in picturebox [modified]

kpuneeth7

22:01 23 Feb '10

I am using a simple xml file..... <image> <img> C:\Documents and Settings\Imran\Desktop\Picture\apple.jpg </img> </image> And i am using visual studio 2008 C# language. .net framework modified on Wednesday, February 24, 2010 3:20 AM
Sign In·View Thread·Permalink

Re: Display a image in picturebox

Stuart Dootson

22:36 23 Feb '10

Here's a short C# program that will read the bit of XML you want and write it to the console.

using System.Xml.XPath;
using System.IO;
 
namespace ConsoleApplication1
{
   class Program
   {
      static void Main(string[] args)
      {
         // The XML file is specified as the first command line argument
         // imagePath will contain the image path contained in the XML file
         string imagePath = new XPathDocument(new StreamReader(args[0])).CreateNavigator().SelectSingleNode("/image/img").Value;
         System.Console.WriteLine(imagePath);
      }
   }
}

I've used XPath to navigate to the required element in the XML file (that's the .SelectSingleNode("/image/img") bit) and read the text there (the .Value bit). Simples.

Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p

CodeProject MVP for 2010 - who'd'a thunk it!

Re: Display a image in picturebox

kpuneeth7

0:53 24 Feb '10

Thanks for the code.... BUt its giving indexoutofrange exception...... HElp
Sign In·View Thread·Permalink

Re: Display a image in picturebox

Stuart Dootson

1:08 24 Feb '10

You need to give a command line argument that specifies the XML file...the code comments DO say that.... Java, Basic, who cares - it's all a bunch of tree-hugging hippy crp* CodeProject MVP for 2010 - who'd'a thunk it!
Sign In·View Thread·Permalink

Re: Display a image in picturebox

kpuneeth7

2:26 24 Feb '10

Thanks dude..... I got it Thank u very much
Sign In·View Thread·Permalink

Multiple group using Muenchian Method [modified]

ONeil Tomlinson

5:15 23 Feb '10

I have the following input XML. I want to group all unique Code/ID combination.

<List>
 <Section>
  	<Code>AA</Code>
  	<ID>11</ID>
 <Section>
 <Section>
  	<Code>AA</Code>
  	<ID>11</ID>
 <Section>
 <Section>
  	<Code>CC</Code>
  	<ID>11</ID>
 <Section>
 <Section>
  	<Code>AA</Code>
  	<ID>22</ID>
 <Section>
 <Section>
  	<Code>AA</Code>
  	<ID>11</ID>
 <Section>
<List>

So result should be as shown below.

<Code>AA</Code>
<ID>11</ID>
 
<Code>CC</Code>
<ID>11</ID>
 
<Code>AA</Code>
<ID>22</ID>

Im using the Muenchian Method. Currely my XSL (below) is grouping by Code (and not Code/ID combination)

<?xml version="1.0" encoding="UTF-16"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:var="http://schemas.microsoft.com/BizTalk/2003/var" exclude-result-prefixes="msxsl var" version="1.0" xmlns:ns0="http://A4C.Interface.HRP.Payroll.Schemas">
	<xsl:output omit-xml-declaration="yes" method="xml" version="1.0" />
 
	<xsl:key name="CCids" match="/List/Section" use="Code"/>
 
	<xsl:template match="/">
		<xsl:apply-templates select="/ns0:Payroll" />
	</xsl:template>
	<xsl:template match="/ns0:Payroll">
		<ns0:List>
			<xsl:for-each select="/List/Section [generate-id(.)=generate-id(key('CCids',Code))]">
				<Section>
					<Code>
						<xsl:value-of select="code/text()" />
					</Code>
					<ID>
						<xsl:value-of select="id/text()" />
					</ID>
				</Section>
			</xsl:for-each>
		</ns0:List>
	</xsl:template>
</xsl:stylesheet>

How do i get this to use a Code/ID combination? I tried nested for-each loop but no luck. Thanks

modified on Tuesday, February 23, 2010 11:40 AM

Re: Multiple group using Muenchian Method

Stuart Dootson

21:47 23 Feb '10

Firstly, your key needs to have an expression which reflects what you want to retrieve:

   <xsl:key name="CCids" match="/List/Section" use="concat(Code,'|',ID)"/>

And then you need to use that same expression when accessing items referenced by the key.

Oh - and in addition, I would suggest not using a for loop - it's not idiomatic XSL. Here's the file I'd use:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:var="http://schemas.microsoft.com/BizTalk/2003/var" exclude-result-prefixes="msxsl var" version="1.0" xmlns:ns0="http://A4C.Interface.HRP.Payroll.Schemas">
   <xsl:output omit-xml-declaration="yes" method="xml" version="1.0" encoding="utf-8"/>
 
   <xsl:key name="CCids" match="/List/Section" use="concat(Code,'|',ID)"/>
 
   <xsl:template match="/">
      <ns0:List>
         <xsl:apply-templates select="/List/Section [generate-id(.)=generate-id(key('CCids',concat(Code,'|',ID)))]"/>
      </ns0:List>
   </xsl:template>
            
   <xsl:template match="Section">
      <Section>
         <Code>
            <xsl:value-of select="Code/text()" />
         </Code>
         <ID>
            <xsl:value-of select="ID/text()" />
         </ID>
      </Section>
   </xsl:template>
</xsl:stylesheet>

Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p

CodeProject MVP for 2010 - who'd'a thunk it!

Re: Multiple group using Muenchian Method

ONeil Tomlinson

1:05 24 Feb '10

Thanks..Really helpful
Sign In·View Thread·Permalink

How Do I deserialize the following Xml string? [modified]

ashishtango

0:05 23 Feb '10

Hi,
I am getting this error There is an error in XML document (1, 2)
When i Deserialize following Xml.

<EnvironmentConfigurations>
<EnvironmentConfiguration>
<EnvironmentName>ABC</EnvironmentName>
<CMSConfiguration>
<Address>net.tcp://localhost:9000/Manager</Address>
<Binding>netTcpBinding</Binding>
<Contract>Common.Interfaces.Server</Contract>
</CMSConfiguration>
</EnvironmentConfiguration>
<EnvironmentConfiguration>
<EnvironmentName>XYZ</EnvironmentName>
<CMSConfiguration>
<Address>net.tcp://localhost:9000/Manager</Address>
<Binding>netTcpBinding</Binding>
<Contract>Common.Interfaces.Server</Contract>
</CMSConfiguration>
</EnvironmentConfiguration>
</EnvironmentConfigurations>

modified on Tuesday, February 23, 2010 5:11 AM

Re: How Do I deserialize the following Xml string?

Stuart Dootson

0:54 23 Feb '10

Deserialize to what? The way in which it's deserialized depends on the type you think it is. Are you sure this was serialized from .NET? That's not the only way to read and write XML, you know - maybe you should be using an XMLReader and parsing the XML yourself? Java, Basic, who cares - it's all a bunch of tree-hugging hippy crp* CodeProject MVP for 2010 - who'd'a thunk it!
Sign In·View Thread·Permalink

Re: How Do I deserialize the following Xml string?

ashishtango

17:52 23 Feb '10

I have serialized using XmlSerializer. I have removed Xml version and namespace information from the XML. I have changed the root node from <ArrayOfEnvironmentConfigurations> to <EnvironmentConfigurations> I want to desrialize this xml. Thank you.
Sign In·View Thread·Permalink

Re: How Do I deserialize the following Xml string?

Stuart Dootson

21:27 23 Feb '10

The answer to your problem is very, very clear - don't mess with the XML that the serializer produces - all of the things you changed are in there for a reason!!!! Java, Basic, who cares - it's all a bunch of tree-hugging hippy crp* CodeProject MVP for 2010 - who'd'a thunk it!
Sign In·View Thread·Permalink

Re: How Do I deserialize the following Xml string?

ashishtango

19:35 24 Feb '10

The problem was with the root node element which i changed. I kept the root name as ArrayOfEnvironmentConfigurations and the problem was solved. Thanks for replying.
Sign In·View Thread·Permalink

Accessing a image from a xml file and display in visual studio picture box

kpuneeth7

22:15 22 Feb '10

Hi everyone, I came across this prob... I am a beginner in xml. I wanted to create a xml file which stores the location of the images. This xml file must be accessed in visual studio picture box and it must display each image wen i click the next button and back wen i press the previous button... Please help
Sign In·View Thread·Permalink

Re: Accessing a image from a xml file and display in visual studio picture box

Richard MacCutchan

23:31 22 Feb '10

kpuneeth7 wrote:
I am a beginner in xml.

Then you should study some of the tutorials here and on MSDN on the use of XML. From the remainder of your question it appears that you are asking for a complete image viewer solution. If you have no idea how to start on this problem then you have a lot of studying still to complete.

txtspeak is the realm of 9 year old children, not developers. Christian Graus

xsd to entity [modified]

pokiri

18:45 22 Feb '10

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" id="BookEasy">

<xsd:element name="BookEasy">

<xsd:complexType>

<xsd:sequence>

<xsd:element name="Control">

<xsd:complexType>

<xsd:sequence>
<xsd:element name="ControlID" type="xsd:int"/>

<xsd:element name="ReservationRequest">

<xsd:complexType>

<xsd:sequence>
<xsd:element name="InDate" type="xsd:dateTime"/>
<xsd:element name="Period" type="xsd:int"/>
<xsd:element name="Adults" type="xsd:int"/>
<xsd:element name="Children" type="xsd:int"/>
<xsd:element name="Infants" type="xsd:int"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>

<xsd:element name="Operators">

<xsd:complexType>

<xsd:sequence>

<xsd:element name="Operator" minOccurs="0" maxOccurs="unbounded">

<xsd:complexType>

<xsd:sequence>
<xsd:element name="OperatorID" type="xsd:int"/>

<xsd:element name="Rooms">

<xsd:complexType>

<xsd:sequence>

<xsd:element name="Room" minOccurs="0" maxOccurs="unbounded">

<xsd:complexType>

<xsd:sequence>
<xsd:element name="RoomID" type="xsd:int"/>

<xsd:element name="Availability">

<xsd:complexType>

<xsd:sequence>
<xsd:element name="IsAvailable" type="xsd:boolean"/>

<xsd:element name="Days">

<xsd:complexType>

<xsd:sequence>

<xsd:element name="Day" maxOccurs="unbounded">

<xsd:complexType>

<xsd:sequence>
<xsd:element name="Date" type="xsd:dateTime"/>
<xsd:element name="IsAvailable" type="xsd:boolean"/>

<xsd:element name="Rates">

<xsd:complexType>

<xsd:sequence>
<xsd:element name="FullRate" type="xsd:decimal"/>
<xsd:element name="LastMinuteRate" type="xsd:decimal"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>

modified on Tuesday, February 23, 2010 12:29 AM

General News Suggestion Question Bug Answer Joke Rant Admin

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Advertise | Privacy | Mobile
Web01 | 2.5.120210.1 | Last Updated 10 Feb 2012

Layout: fixed | fluid