Click here to Skip to main content
14,383,511 members
Rate this:
Please Sign up or sign in to vote.
See more:
I'm new to .net and need your help .

Please provide me the vb.net code to read the large xml file then it should remove the hexadecimal from code then we should load the xml file in dataset. from dataset we can display that in log files by using datatables.

What I have tried:

I have tried with following.

1. 'TS = FSO.OpenTextFile(sFTPLogXml, IWshRuntimeLibrary.IOMode.ForReading) '-sFTPLog => Original Log file
'Final = TS.ReadAll
'TS.Close()

2.'Final = File.ReadAllText(sFTPLogXml) 'TAS-fails here with out of memory error, trying to read the entire file. The input file gets pretty big
Posted
Comments
AnvilRanger 17-Feb-16 17:33pm
   
This is not a code to order service. You might new to DotNet development, but you can query Google just like anyone else.

But I will give you a piece of advice. It appears you are trying to use the FileSystemObject, FSO. My advice is don't. This is left over from the VB6 days. If you are working with files you need to look at the System.IO namespace.

1 solution

Rate this:
Please Sign up or sign in to vote.

Solution 1

You did it wrong. You need to parse XML as you read, without reading it in intermediate text file. Do you want to have double set of data in your memory, raw XML and XML parsed from a file? This is absurd, especially when your concern is the memory limitation.

Most economic and fast way of parsing XML offered by .NET FCL is the class System.Xml.XmlReader, which should operate directly on a file on disk or a stream, without using any intermediate text:
XmlReader Class (System.Xml)[^].

Now, eventually the size of a file can grow beyond the limit of available system RAM. It's especially easy with x86 processes which are limited by 4G of address space, and really available memory will be considerably less. With WOW64 in a 64-bit system the situation is with a x86 (32-bit) process can be considerably better (you don't need the whole OS to reside in part of this memory), but principle limitation is still the same.

So, with this approach, you can develop the technique of parsing just the part of file, skipping nodes which are no longer needed. One benefit of System.Xml.XmlReader is that you only navigate through XML tree, but actually read and store in memory only what you want.

The alternative implementation of this idea can be achieved with another XML parsing approach. You can use the class System.Xml.Linq.XDocument; this is the most adequate way similar to that of XmlDocument, supporting LINQ to XML programming. Please see:
http://msdn.microsoft.com/en-us/library/system.xml.xmldocument.aspx, http://msdn.microsoft.com/en-us/library/bb387063.aspx,
LINQ to XML Overview[^],
LINQ to XML[^].

—SA
   
v3
Comments
Maciej Los 18-Feb-16 15:36pm
   
Absolutely right, a 5!
Sergey Alexandrovich Kryukov 18-Feb-16 18:52pm
   
Thank you, Maciej.
—SA

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)




CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100