Click here to Skip to main content
Click here to Skip to main content

Easy Method to Split Large XML File Using LINQ to XML

, 23 Jun 2014 CPOL
Rate this:
Please Sign up or sign in to vote.
Use LINQ to XML to split an XML file into a number of smaller files

Introduction

You have a large well formed XML file which you wish to split into smaller manageable files. Each output file is also a well formed XML file. This approach uses Skip and Take LINQ extension methods to intuitively slice and dice the source XML into smaller parts.

Using the Code

Hopefully the code is sufficiently commented so that further explanation is not required.

The source XML file can be downloaded here.

You need to drop the source XML file into your "C:\temp" folder.

On running the code, the source file products.xml containing 504 elements of <Product> are split across three files containing 200, 200, 104 elements of <Product>.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Xml.Linq;
using System.Xml.Schema;

namespace SplitXmlFile
{
    class Program
    {
        static void Main(string[] args)
        {
            string sourceFile = @"C:\Temp\Products.xml";
            string rootElement = "Products";
            string descElement = "Product";
            int take = 200;
            string destFilePrefix = "ProductsPart";
            string destPath = @"C:\temp\";

            SplitXmlFile(sourceFile, rootElement, descElement, take,
                        destFilePrefix, destPath);

            Console.ReadLine();
        }

        private static void SplitXmlFile(string sourceFile
                        , string rootElement
                        , string descendantElement
                        , int takeElements
                        , string destFilePrefix
                        , string destPath)
        {
            XElement xml = XElement.Load(sourceFile);
            // Child elements from source file to split by.
            var childNodes = xml.Descendants(descendantElement);

            // This is the total number of elements to be sliced up into 
            // separate files.
            int cnt = childNodes.Count();

            var skip = 0;
            var take = takeElements;
            var fileno = 0;

            // Split elements into chunks and save to disk.
            while (skip < cnt)
            {
                // Extract portion of the xml elements.
                var c1 = childNodes
                            .Skip(skip)
                            .Take(take);

                // Setup number of elements to skip on next iteration.
                skip += take;
                // File sequence no for split file.
                fileno += 1;
                // Filename for split file.
                var filename = String.Format(destFilePrefix + "_{0}.xml", fileno);
                // Create a partial xml document.
                XElement frag = new XElement(rootElement, c1);
                // Save to disk.
                frag.Save(destPath + filename);
            }
        }
    }
}

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

ravgill66
Software Developer (Senior)
United Kingdom United Kingdom
No Biography provided

Comments and Discussions

 
GeneralThank you Pinmembernaat_8019-Dec-14 12:24 
QuestionLarge file, large memory consumption Pinmemberwmjordan23-Jun-14 16:36 
AnswerRe: Large file, large memory consumption Pinmemberravgill6624-Jun-14 1:47 
GeneralRe: Large file, large memory consumption Pinmemberjohannesnestler25-Jun-14 4:24 
GeneralRe: Large file, large memory consumption Pinmemberravgill6626-Jun-14 4:36 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.141223.1 | Last Updated 23 Jun 2014
Article Copyright 2014 by ravgill66
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid