5,276,801 members and growing! (15,935 online)
Email Password   helpLost your password?
Languages » C# » General     Intermediate

SqlBulkCopy in ADO.NET 2.0 , SqlBulkCopy Class in C#, SqlBulkCopy help ,SqlBulkCopy guide,sql Bulk Copy class,SqlBulkCopy tutorial

By idreeskhan

SqlBulkCopy in ADO.NET 2.0 , SqlBulkCopy Class in C#, SqlBulkCopy help ,SqlBulkCopy guide,Transferring Data Using SqlBulkCopy - SqlBulkCopy - Copy Table Data Between SQL Servers at High Speeds ,Perform bulk copies with .NET 2.0's SqlBulkCopy class,SqlBulkCopy tutorial
SQL, C# 1.0, C# 2.0, C# 3.0, C#Windows, .NET, .NET 3.0, .NET 1.0, .NET 1.1, .NET 2.0, WinXP, ASP.NET, ADO.NET, WebForms, SQL 2000, VS.NET2002, VS.NET2003, VS2005, SQL, VS, DBA, Dev

Posted: 28 Sep 2007
Updated: 28 Sep 2007
Views: 10,443
Announcements
Want a new Job?



Search    
Advanced Search
Sitemap
14 votes for this Article.
Popularity: 4.65 Rating: 4.06 out of 5
0 votes, 0.0%
1
1 vote, 7.1%
2
3 votes, 21.4%
3
5 votes, 35.7%
4
5 votes, 35.7%
5
Note: This is an unedited contribution. If this article is inappropriate, needs attention or copies someone else's work without reference then please Report This Article

Introduction

SqlBulkCopy in ADO.NET 2.0

SqlBulkCopy is a new feature in ADO.NET 2.0 that gives you DTS-like speeds when you need to programmatically copy data from one database to another.That lets you to perform copy operation for a large amount of data between a source data store and a destination data table.

Background

I was facing problem in my last application. I have to work on the data migration. It was from Fox Pro to SQL server 2005.There are million of records in each table and also other problem some of table contains more 150 Column. There was no concept of normalization in the fox pro database I also have to manage that. I come across SqlBulkCopy Class in Stem.Data.sqlcleint.

This class helps me solve my problem lots. So I have make mind to share the basic ideas of this class with u people.Copying a large amount of data from a source data store to a destination table in SQL
database by using a traditional approach has performance effect because you need to call database for several times. There were some ways to solve this issue but now by having ADO.NET 2.0 in hand, you can perform a bulk copy and reduce the number of database accesses to improve performance and speed. SqlBulkCopy is the heart of bulk copy in ADO.NET 2.0 and SqlBulkCopyColumnMapping and SqlBulkCopyColumnMappingCollection objects assist it in this way. Later I'll discuss these objects in more details.

Using the code

SqlBulkCopy

SqlBulkCopy is the object that helps you to perform a bulk copy. You can use a DataReader or DataTable as source data store (you can load your data from SQL database, Access database, XML or ... into these objects easily) and copy them to a destination table in database.

To accomplish this task, SqlBulkCopy uses a collection of SqlBulkCopyColumnMapping objects which will be saved as its SqlBulkCopyColumnMappingCollection property. SqlBulkCopyColumnMapping maps a column in data source to a table in destination table via their name or index.

SqlBulkCopy has some important properties that you should be aware of them to be able to use it:

BatchSize: This integer value specifies the number of rows that should be copied in each attempt to copy to database. This value has direct effect on the number of accesses to database.
BulkCopyTimeOut: The number of seconds that system should wait to let SqlBulkCopy to copy rows.
ColumnMappings: A ReadOnly SqlBulkCopyColumnMappingCollection. You need to use its Add() method to add a new SqlBulkCopyColumnMapping object to its collection.

DestinationTableName: String value of destination table's name.
NotifyAfter: SqlRowsCopied event handler will be called when the number of rows specified in this property has been copied.

This object also has four overloads. You can pass a SqlConnection (or a connection string) plus an optional SqlBulkCopyOptions and SqlTransaction to its constructor. Latest two parameters can change the behavior of SqlBulkCopy object. Using SqlBulkCopyOptions enumerator you can specify that for example SqlBulkCopy keeps identities or check constraints and some other options. Using SqlTransaction you can pass an external SqlTransaction and your SqlBulkCopy uses this transaction in all parts of the process.

SqlBulkCopy also has a SqlRowsCopied event handler that triggers when the specific number of DataRows that have been copies. You specified this value via NotifyAfter property. This handler is helpful when you want to be aware of your process (for instance showing it via a ProgressBar to end user).
Useful Method

The last thing that should be mentioned about SqlBulkCopy object is its WriteToServer() method. This method can get an array of DataRows, a DataTable or a DataReader and copies their content to destination table in database.

SqlBulkCopyColumnMapping

SqlBulkCopyColumnMapping is the object that maps your source columns to destination columns in a bulk copy. A SqlBulkCopyColumnMapping can get the source and destination column names or ordinals via its properties or its constructor. It has these properties:
SourceColumn: String value of source column's name.
SourceOrdinal: Integer value of source column's index.
DestinationColumn: String value of destination column's name.
DestinationOrdinal: Integer value of destination column's index.

One of SourceColumn and SourceOrdinal and one of DestinationColumn and DestinationOrdinal should be set. Also you can set these properties via constructor which is an easier way.

Note that if your source and destination columns have same names, it's not required to use SqlBulkCopyColumnMapping objects because SqlBulkCopy can do its job automatically.

Write a Sample Application

Let's have look on example. I think the sample application you'll see in a moment can cover all you need to use SqlBulkCopy.

NOTE: If u have same table column on both sides mean source and Destination. Then I will suggest that u shout not mention SqlBulkCopyColumnMapping properties. Because it perform all tasks automatically.

PerformBulkCopy() method is used because both side column are same.

"MsoNormal"> Code

public class mySqlBulkCopy
{
private static void PerformBulkCopy()
{
    string connectionString =
            @"Server=localhost;Database=Northwind;Trusted_Connection=true";
    // get the source data

    using (SqlConnection sourceConnection = 
            new SqlConnection(connectionString))
    {
        SqlCommand myCommand =
            new SqlCommand("SELECT * FROM Customer", sourceConnection);
        sourceConnection.Open();
        SqlDataReader reader = myCommand.ExecuteReader();

        // open the destination data

        using (SqlConnection destinationConnection =
                    new SqlConnection(connectionString))
        {
            // open the connection

            destinationConnection.Open();

            using (SqlBulkCopy bulkCopy =
            new SqlBulkCopy(destinationConnection.ConnectionString))
            {
                bulkCopy.BatchSize = 500;
                bulkCopy.NotifyAfter = 1000;
                bulkCopy.SqlRowsCopied +=
                    new SqlRowsCopiedEventHandler(bulkCopy_SqlRowsCopied);
                bulkCopy.DestinationTableName = " Customer_Backup";
                bulkCopy.WriteToServer(reader);
            }
        }
        reader.Close();
    }
}
// this method is used if u want to mape column with different names

// u can also use the index of column by using SourceOrdinal

private static void PerformBulkCopyMyDifferentSchema()
{
    string connectionString = @"Server=
        localhost;Database=Northwind;Trusted_Connection=true";
    DataTable sourceData = new DataTable();
    // get the source data

    using (SqlConnection sourceConnection =
                    new SqlConnection(connectionString))
    {
        SqlCommand myCommand =
            new SqlCommand("SELECT * 
            FROM Customer", sourceConnection);
        sourceConnection.Open();
        SqlDataReader reader = myCommand.ExecuteReader();
        // open the destination data

        using (SqlConnection destinationConnection =
                    new SqlConnection(connectionString))
        {
            // open the connection

            destinationConnection.Open();
            using (SqlBulkCopy bulkCopy =
                new SqlBulkCopy(destinationConnection.ConnectionString))
            {
                bulkCopy.ColumnMappings.Add("CustomerID","ID");
                bulkCopy.ColumnMappings.Add("CustomerName", "Name");
                bulkCopy.ColumnMappings.Add("CustomerPoint", "Points");
                bulkCopy.DestinationTableName = " CustomerPoints";
                bulkCopy.WriteToServer(reader);
            }
        }
        reader.Close();
    }
}

}

}

"MsoNormal">

Summary If u are familiar with SQLHELPER class than u can improve ur performance more efficiently In this article I talked about newly added feature in ADO.NET 2.0, SqlBulkCopy, which helps you to bulk copy large amounts of data between data source and data table and improve your performance

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

idreeskhan


I'm from Pakistan and have one year of experience in C# and Asp.net in visual stdio 2005 and 2008 and SQL Server 2000/2005
idrees_bit@hotmail.com
Occupation: Web Developer
Location: Pakistan Pakistan

Other popular C# articles:

Article Top
Sign Up to vote for this article
You must Sign In to use this message board.
FAQ FAQ Noise ToleranceSearch Search Messages 
 Layout  Per page   
 Msgs 1 to 11 of 11 (Total in Forum: 11) (Refresh)FirstPrevNext
Subject  Author Date 
GeneralSQLBulkCopymemberHemant.Kamalakar2:25 20 Oct '07  
GeneralRe: SQLBulkCopymemberidreeskhan20:21 21 Oct '07  
QuestionRe: SQLBulkCopymemberHemant.Kamalakar23:11 21 Oct '07  
GeneralCan SqlBulkCopy copy data to the local temporary table which is dynamicly generated in a store procedure?memberdddd2185:57 1 Oct '07  
Generalthat is to say, how can I use SqlBulkCopy to copy data to the local temporary table which is dynamicly generated in a store procedure?memberdddd2186:14 1 Oct '07  
GeneralRe: that is to say, how can I use SqlBulkCopy to copy data to the local temporary table which is dynamicly generated in a store procedure?memberidreeskhan0:29 2 Oct '07  
GeneralRe: Can SqlBulkCopy copy data to the local temporary table which is dynamicly generated in a store procedure?memberidreeskhan0:27 2 Oct '07  
GeneralRe: Can SqlBulkCopy copy data to the local temporary table which is dynamicly generated in a store procedure?memberdddd2180:59 2 Oct '07  
GeneralNice Articlememberrilov4:43 1 Oct '07  
GeneralRe: Nice Articlememberidreeskhan20:30 1 Oct '07  
GeneralRe: Nice Articlememberrilov3:40 2 Oct '07  

General General    News News    Question Question    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

PermaLink | Privacy | Terms of Use
Last Updated: 28 Sep 2007
Editor:
Copyright 2007 by idreeskhan
Everything else Copyright © CodeProject, 1999-2008
Web13 | Advertise on the Code Project