Click here to Skip to main content
15,909,332 members
Home / Discussions / Database
   

Database

 
GeneralRe: DataTable.Select() Pin
Guillermo Rivero5-Oct-04 6:15
Guillermo Rivero5-Oct-04 6:15 
GeneralRe: DataTable.Select() Pin
EdbertP5-Oct-04 16:59
EdbertP5-Oct-04 16:59 
GeneralRe: DataTable.Select() Pin
amatyasik5-Oct-04 21:28
amatyasik5-Oct-04 21:28 
Generalerror when calling stored procedure in ole db Pin
sudheer_ars4-Oct-04 20:28
sudheer_ars4-Oct-04 20:28 
GeneralRe: error when calling stored procedure in ole db Pin
sreejith ss nair7-Oct-04 1:11
sreejith ss nair7-Oct-04 1:11 
GeneralRe: error when calling stored procedure in ole db Pin
Colin Angus Mackay7-Oct-04 1:42
Colin Angus Mackay7-Oct-04 1:42 
GeneralSaving files into image field on sql server2000 Pin
firad3-Oct-04 20:18
sussfirad3-Oct-04 20:18 
QuestionOpinions on loading large file into database? Pin
ThomasH13-Oct-04 12:30
ThomasH13-Oct-04 12:30 
Hello everyone, I'd like your opinions! I'm rewriting a program that I did back in 1996; it takes a large "external-system" text file, and compares it against our database. If there's a difference between the file entry and database entry, we'll take the entry in the file and put it into the DB, and make note of it on a report. We don't run this program often, and we always take the file as the "correct" data. I'd like to load the file into an ADO.NET dataset, but I'm a little hesitant because of the average file size.

The size of the flat file is usually between 2 and 4 MB large. Is this an okay size for a dataset in memory? I'd want to load up the entire file at one time into a dataset. Then I'd set up a DataAdapter and DataSet and etc for my database, and compare both datasets together. I could just call DataAdapter.Update on the database's dataset to make the changes. So not only would I have the 2-4 MB flatfile in memory, but I'd also potentially have the same size (2-4MB) from the database! The benefit is that I'd really be "using" ADO.NET... comparisions would be easier, and the update would be cleaner.

My alternative is to open the file, and read it one record at a time. I'd then do an executescalar to check our database, and another executescalar if I had to update our database. This method seems memory-friendly, but database-intensive and network-intensive because of all the individual transactional updates... I'd have to update the database one-record-at-a-time!

Do any of you load up huge 2-4 MB text files into a dataset? If so, do you have any problems with this approach? I really want to use datasets- but this program will be running on workstations, on an as-required basis- it won't be running on a server, so I don't want to write a program that takes forever to run. The workstations are semi-powerful; but this "might" have to run on an XP Pro box with only 256MB of memory. (Most of the XP Pros have 512 mb ram.)

Oh, I'll be using C#.NET and ADO.NET under .NET Framework version 1.1. The file isn't in XML, and I think it would be pointless to convert it to XML first, and then process it. Correct me if I'm wrong, though!

Thanks for any ideas!


-Thomas
GeneralDatetime problem on sql statement Pin
Gamoss3-Oct-04 3:54
Gamoss3-Oct-04 3:54 
GeneralRe: Datetime problem on sql statement Pin
Jeremy Oldham3-Oct-04 3:57
Jeremy Oldham3-Oct-04 3:57 
GeneralRe: Datetime problem on sql statement Pin
Gamoss3-Oct-04 4:06
Gamoss3-Oct-04 4:06 
GeneralShutdown and restart SQL Server programatically Pin
Xiangyang Liu 刘向阳2-Oct-04 16:39
Xiangyang Liu 刘向阳2-Oct-04 16:39 
GeneralRe: Shutdown and restart SQL Server programatically Pin
Kurt _B7-Oct-04 13:09
Kurt _B7-Oct-04 13:09 
GeneralRe: Shutdown and restart SQL Server programatically Pin
Xiangyang Liu 刘向阳14-Oct-04 8:05
Xiangyang Liu 刘向阳14-Oct-04 8:05 
GeneralRe: Shutdown and restart SQL Server programatically Pin
Xiangyang Liu 刘向阳14-Oct-04 15:04
Xiangyang Liu 刘向阳14-Oct-04 15:04 
GeneralNeed help do design the table for this requirement Pin
percyvimal2-Oct-04 6:44
percyvimal2-Oct-04 6:44 
GeneralRe: Need help do design the table for this requirement Pin
Colin Angus Mackay2-Oct-04 7:00
Colin Angus Mackay2-Oct-04 7:00 
GeneralRe: Need help do design the table for this requirement Pin
percyvimal2-Oct-04 7:14
percyvimal2-Oct-04 7:14 
GeneralRe: Need help do design the table for this requirement Pin
Colin Angus Mackay2-Oct-04 7:52
Colin Angus Mackay2-Oct-04 7:52 
GeneralRe: Need help do design the table for this requirement Pin
percyvimal2-Oct-04 8:25
percyvimal2-Oct-04 8:25 
GeneralRe: Need help do design the table for this requirement Pin
Colin Angus Mackay2-Oct-04 8:59
Colin Angus Mackay2-Oct-04 8:59 
QuestionMark DataTable column so changes do not affect RowState? Pin
DrGerry1-Oct-04 12:09
DrGerry1-Oct-04 12:09 
GeneralCheck for syntax Pin
macupryk30-Sep-04 21:16
macupryk30-Sep-04 21:16 
GeneralRe: Check for syntax Pin
Colin Angus Mackay1-Oct-04 1:24
Colin Angus Mackay1-Oct-04 1:24 
GeneralI need to Create a User Login and Registration Webform in Oracle and ODBC Pin
macupryk30-Sep-04 21:15
macupryk30-Sep-04 21:15 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.