Export file data to SQL

Question

0.00/5 (No votes)

See more:

Hi,

I have records in file with some delimitor. My front end is in PHP. Presently i have build logic in front end which reads data from file row wise, column wise and then insert the same in SQL database.

Till some extent the above logic is working, but when my data is increasing, my process is getting slow.

I wanted to do all processing at SQL level.

How can i achieve this task as i am bit new to PHP.

Thanks in advance.

Cheers

Posted 8-Jul-14 23:15pm

Magic Wonder

Add a Solution

Comments

Mohibur Rashid 9-Jul-14 5:35am

Excel is slow, if it get bigger it will get slower too. I have to process lots of excel file. I just convert the file to csv. I use Open Office application to convert to csv. Because it allows me to choose encoding type,

Magic Wonder 9-Jul-14 6:32am

I have flat file .

CHill60 9-Jul-14 8:39am

"lets talk like a computer programmer" ... :laugh: "flat file" is a programming term that has been around for many, many years! :)

Magic Wonder 9-Jul-14 9:11am

ha ha ha....let me re - put my words with correction. I have a file in CSV format.

CHill60 9-Jul-14 8:44am

A little confusing as you say you "wanted to do all processing at SQL level" - does that mean SSIS is not an option for you?
Do you have any indexes on the table(s) you are importing to?

Magic Wonder 9-Jul-14 9:14am

Not a big task i wanted to do at SQL level. In front end i am inserting data row by row which is taking time as data is increasing, instead of this i want a bulk insert kind of functionality to dump my data.

Yes i have indexes on tables.

CHill60 9-Jul-14 9:24am

The index on the table will slow things down during an insert. Try dropping the indexes (on the table(s) you are inserting to) and recreate it/them after you have completed the import. See this article also Handling BULK Data insert from CSV to SQL Server[^]

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Gauri Chodanker · Answer 1 · 2014-07-09T04:25:00

Solution 1

Writing a front end logic to upload the file will take time as the data in the file increases. Becuase it is talking between 2 different languages SQL and PHP.
I would advise to script a SSIS package in SQL to run upload faster.

Posted 9-Jul-14 4:25am

Gauri Chodanker