inserting large volume of records

Question

0.00/5 (No votes)

See more:

Hi,
I have a scenario that inserting bulk records into sqlserver table.
Here i am validating each record-column value based on some condition.

So i have one temporary table, from that table each and every record i am moving to original table after validation.

My problem is,
when the first time i am inserting records it is ge tting loaded veryfast.It takes around 3-5 mins for 100000 records.if i insert 200000 then 10 mins .But when i am trying to insert next time ie,1000 records for the next time(means after an hour or two)it is getting very slow.

If i truncate all values from original table and started the process again ,then its loading very fast.

Why this happens?

Can anybody tell the reason?
And also help me to solve this issue.
please.....

Posted 14-Nov-12 4:30am

Arun.Mohandas

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Christian Graus · Answer 1 · 2012-11-14T04:33:00

Solution 1

Your database gets slower as it has to insert more entries. I'd guess the reason has to do with indexes you have set up on the table. Does it have a clustered index ? That makes lookups fast, but inserts get slower as the table size grows.

Posted 14-Nov-12 4:33am

Christian Graus

Comments

Arun.Mohandas 14-Nov-12 10:43am

thanks for your reply,
yes, initially i was having index.but i removed the index and tried.but no change.
My doubt is ,if i insert 300000 records also first time it is inserting fast.but for second time its getting slow even if i try for 1000 records.Why?

Christian Graus 14-Nov-12 10:44am

The answer is obvious. Once you have a lot of records, it is more expensive to add more. The reason has to do with your indexing, I am sure.