Batch inserts into SQL server table

Question

4.50/5 (2 votes)

See more:

Hello,
i am having a performance problem when trying to insert multiple rows into a table in SQL server. I am parsing lines in a txt file and inserting the lines each in a table row. The problem is that on average i have 7k plus lines in each file this is making my program very slow and not adequate. Any help ?

Posted 11-Oct-10 4:40am

SuperAdministrator

Add a Solution

3 solutions

Solution 1

One way could be to build all your queries together in a string and then make a single call to the database. That will at least reduce the performance drag of connecting to your server every single time and then performing an insert / update.

Posted 11-Oct-10 4:44am

Abhinav S

Updated 11-Oct-10 7:00am

Comments

SuperAdministrator 12-Oct-10 2:32am

My queries are SP's and building them in a string inside the code will cost me compile time .. since they need to be compiled.

Solution 3

you can use bcp to insert datat from file into your DB

http://msdn.microsoft.com/en-us/library/aa174646(SQL.80).aspx[^]

Posted 12-Oct-10 22:06pm

Mouadh TRABELSI

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Karthik. A · Accepted Answer · 2010-10-12T15:05:00

Build a DataTable in-memory and then do a bulk insert using SqlBulkCopy. Check out this example http://blogs.msdn.com/b/nikhilsi/archive/2008/06/11/bulk-insert-into-sql-from-c-app.aspx[^]

Edit:

Sorry, I read your comment, you have said that you are using a stored proc. Do you mean that you call a stored proc for every line you read? That's always going to be costly. See if you could move to the approach I gave above. Else make sure you are not opening / closing the database connection for every line read/ every stored proc call...