Click here to Skip to main content
       

Database

 
QuestionImporting of Huge fixed width text file in sql server table Pinmemberibrahim sharief27-Dec-12 10:24 
AnswerRe: Importing of Huge fixed width text file in sql server table PinmemberPIEBALDconsult27-Dec-12 15:08 
AnswerRe: Importing of Huge fixed width text file in sql server table PinmemberDeepak Kr130-Dec-12 21:29 
AnswerRe: Importing of Huge fixed width text file in sql server table Pinmemberjschell31-Dec-12 10:14 
Questionshow Sum(Balance) as ob on Rows Header and add next rows balance in ob Pinmemberhadeed147227-Dec-12 7:52 
AnswerRe: show Sum(Balance) as ob on Rows Header and add next rows balance in ob Pinmemberdjj5527-Dec-12 8:53 
GeneralRe: show Sum(Balance) as ob on Rows Header and add next rows balance in ob Pinmemberhadeed147227-Dec-12 9:03 
QuestionSQL BulkCopy using IDataReader - how can you cast the fields? PinmemberAllan Watkins26-Dec-12 20:12 
I'm looking for a method/example/pseudo-code of importing a CSV file into a SQL Server table with several requirements:
1) I do not know the quantity of columns, column header nor data type of each column prior to the program running. Along with the header fields, I have another file which describes the type and size of each column (like integer, decimal or string) and I can already create the destination table based on this input.
2) Because I don't know the format of the input table before running the program (i.e. the input table changes every time), I can't create a class specifically designed to describe the contents of each record from the CSV file.
3) I already have a method working where a DataTable gets fully populated and the SQLBulkCopy variable uses the .WriteToServer() method to insert all of the records. HOWEVER, there's a problem when the source file gets too large (over 500mb or so) and the result is an OutOfMemory exception.
4) The AdHocGeek has a partial solution which works with strings only but doesn't address integers, decimals, dates, et.al.
 
The problem is that I can't see a way to use an IDataReader into a complex (i.e. many field types) table on the fly. I've added the .ColumnMappings() method to determine the name of the columns with which to map the input columns into the destination SQL Server table. When the application is run however, it gets an error indicating "The given value of type String from the data source cannot be converted to type decimal of the specified target column".
 
I can't determine why the DataTable works and the DataReader doesn't and can't find any examples to prove otherwise.
 
Any ideas or examples would be appreciated.
AnswerRe: SQL BulkCopy using IDataReader - how can you cast the fields? PinmemberMycroft Holmes26-Dec-12 21:32 
GeneralRe: SQL BulkCopy using IDataReader - how can you cast the fields? PinmemberAllanW1111127-Dec-12 7:02 
AnswerRe: SQL BulkCopy using IDataReader - how can you cast the fields? PinmemberPIEBALDconsult27-Dec-12 9:48 
QuestionSubquery returns more than 1 row PinmemberJassim Rahma24-Dec-12 2:02 
AnswerRe: Subquery returns more than 1 row PinmemberEddy Vluggen24-Dec-12 2:29 
AnswerRe: Subquery returns more than 1 row PinmemberMaulikDusara24-Dec-12 22:44 
GeneralRe: Subquery returns more than 1 row PinmemberEddy Vluggen25-Dec-12 3:41 
GeneralRe: Subquery returns more than 1 row PinmemberMycroft Holmes26-Dec-12 12:17 
GeneralRe: Subquery returns more than 1 row PinmemberMaulikDusara26-Dec-12 19:45 
GeneralRe: Subquery returns more than 1 row PinmemberEddy Vluggen27-Dec-12 5:24 
QuestionOn the Naming of Columns for Lookup tables. PinmemberBrady Kelly17-Dec-12 23:44 
AnswerRe: On the Naming of Columns for Lookup tables. PinmemberMycroft Holmes18-Dec-12 0:19 
GeneralRe: On the Naming of Columns for Lookup tables. PinmemberBrady Kelly18-Dec-12 0:26 
GeneralRe: On the Naming of Columns for Lookup tables. PinmemberMycroft Holmes18-Dec-12 2:11 
GeneralRe: On the Naming of Columns for Lookup tables. PinmemberBrady Kelly18-Dec-12 2:15 
AnswerRe: On the Naming of Columns for Lookup tables. PinmemberPIEBALDconsult18-Dec-12 5:05 
GeneralRe: On the Naming of Columns for Lookup tables. PinmemberBrady Kelly18-Dec-12 5:12 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.


Advertise | Privacy | Mobile
Web04 | 2.8.141220.1 | Last Updated 19 Dec 2014
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid