Click here to Skip to main content
12,065,060 members (41,630 online)
Click here to Skip to main content
Add your own
alternative version

Tagged as

Stats

8K views
12 bookmarked
Posted

Easy Method to Split Large XML File Using LINQ to XML

, 23 Jun 2014 CPOL
Rate this:
Please Sign up or sign in to vote.
Use LINQ to XML to split an XML file into a number of smaller files

Introduction

You have a large well formed XML file which you wish to split into smaller manageable files. Each output file is also a well formed XML file. This approach uses Skip and Take LINQ extension methods to intuitively slice and dice the source XML into smaller parts.

Using the Code

Hopefully the code is sufficiently commented so that further explanation is not required.

The source XML file can be downloaded here.

You need to drop the source XML file into your "C:\temp" folder.

On running the code, the source file products.xml containing 504 elements of <Product> are split across three files containing 200, 200, 104 elements of <Product>.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Xml.Linq;
using System.Xml.Schema;

namespace SplitXmlFile
{
    class Program
    {
        static void Main(string[] args)
        {
            string sourceFile = @"C:\Temp\Products.xml";
            string rootElement = "Products";
            string descElement = "Product";
            int take = 200;
            string destFilePrefix = "ProductsPart";
            string destPath = @"C:\temp\";

            SplitXmlFile(sourceFile, rootElement, descElement, take,
                        destFilePrefix, destPath);

            Console.ReadLine();
        }

        private static void SplitXmlFile(string sourceFile
                        , string rootElement
                        , string descendantElement
                        , int takeElements
                        , string destFilePrefix
                        , string destPath)
        {
            XElement xml = XElement.Load(sourceFile);
            // Child elements from source file to split by.
            var childNodes = xml.Descendants(descendantElement);

            // This is the total number of elements to be sliced up into 
            // separate files.
            int cnt = childNodes.Count();

            var skip = 0;
            var take = takeElements;
            var fileno = 0;

            // Split elements into chunks and save to disk.
            while (skip < cnt)
            {
                // Extract portion of the xml elements.
                var c1 = childNodes
                            .Skip(skip)
                            .Take(take);

                // Setup number of elements to skip on next iteration.
                skip += take;
                // File sequence no for split file.
                fileno += 1;
                // Filename for split file.
                var filename = String.Format(destFilePrefix + "_{0}.xml", fileno);
                // Create a partial xml document.
                XElement frag = new XElement(rootElement, c1);
                // Save to disk.
                frag.Save(destPath + filename);
            }
        }
    }
}

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

ravgill66
Software Developer (Senior)
United Kingdom United Kingdom
No Biography provided

You may also be interested in...

Comments and Discussions

 
QuestionNeed to remove namespaces in xml files and then splitting Pin
parimal226710-Apr-15 22:17
memberparimal226710-Apr-15 22:17 
GeneralThank you Pin
naat_8019-Dec-14 12:24
membernaat_8019-Dec-14 12:24 
QuestionLarge file, large memory consumption Pin
wmjordan23-Jun-14 16:36
memberwmjordan23-Jun-14 16:36 
AnswerRe: Large file, large memory consumption Pin
ravgill6624-Jun-14 1:47
memberravgill6624-Jun-14 1:47 
GeneralRe: Large file, large memory consumption Pin
johannesnestler25-Jun-14 4:24
memberjohannesnestler25-Jun-14 4:24 
GeneralRe: Large file, large memory consumption Pin
ravgill6626-Jun-14 4:36
memberravgill6626-Jun-14 4:36 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.160204.4 | Last Updated 23 Jun 2014
Article Copyright 2014 by ravgill66
Everything else Copyright © CodeProject, 1999-2016
Layout: fixed | fluid