C# Discussion Boards - CodeProject

C# Windows application and data store

1-Aug-03 4:12

Have you read this: CustomAttribute : 0x0C[^]

-Nick Parker

ryc31-Jul-03 20:20

31-Jul-03 20:20

I am creating a windows application using c# and have the need for a robust data store. My application is similar to that of Outlook (different data, similar size + display) where the listbox that contains your mail items can have upwards of 10k entries and 100mb+ of data. I am researching different methods of storing this data so that the application can have easy access to it (not only for displaying but also random access and searching). Storing the data in an external database is not an option. The data store must be internal to the application (atleast from the user's point of view).

I remember something about Outlook using a striped down version of sql server on the inside... I have looked a little into using embeded mysql server and I have seen a few people talking about doing the same. Can anyone provide feedback weather or not this is a good idea? Are there better alternatives for a data store that can handle this quantity of data effeciently?

Thanks.

Kannan Kalyanaraman31-Jul-03 21:09

Kannan Kalyanaraman

31-Jul-03 21:09

Have you considered using MS Access.
You dont have to install anything spl(drivers et al) for accessing them also you can password protect the db, I know CityDesk[^](a content mgmt. system) uses them extensively.

Regards,
Kannan

ryc1-Aug-03 9:22

1-Aug-03 9:22

I have considered using an external database but realized it just isnt an option. I dont want the user to be required to run an external service just to use my application. They should be able to run it, and it alone, and have it display their data. Think Outlook Express and large quantities of email.

Thanks.

Rampas Tomas31-Jul-03 21:55

Rampas Tomas

31-Jul-03 21:55

You can use classes from System.Data namespace and build the database only on the client. You would have DataSet, DataTable, DataView, constraints etc. Moreover you could serialize the data and read them when App start. Advantage: you work with database without any DB engine Smile | :)

.

Tomas Rampas
------------------------------
gedas CR s.r.o.
System analyst, MCP
TGM 840,
293 01 Mlada Boleslav,
Czech Republic
Telefon/phone +420(326)711411
Telefax/fax +420(326)711420
rampas@gedas.cz
http://www.gedas.com/
------------------------------

To be or not to be is true...
George Bool

Rein Hillmann31-Jul-03 22:23

Rein Hillmann

31-Jul-03 22:23

Why not use MSDE? I'm not sure if this qualifies as an "external database". If you're looking at using mysql you're using an external database anyway.

I really wouldn't go and reinvent the wheel though.

ryc1-Aug-03 9:15

1-Aug-03 9:15

Sorry I didnt make this clearer in my original post. The particular flavor of mysql I was looking at is an embeded version that runs inside your application. You use their dll to interact with the database files. There is no 'external process'.

What I am looking for is a solution that the user wont have to muck with. I want them to run my application, and that is it. I dont want them to have to start a database server first. Think Outlook Express.

Thanks for the feedback, I will look into MSDE (not all that familiar with MS technologies).

StealthyMark1-Aug-03 0:04

StealthyMark

1-Aug-03 0:04

Internally, I'd work with DataSets and DataTables, no question. Because both MSDE and Access databases have quite expensive disk costs, I'd use the XML serialization features in conjunction with compression (SharpZipLib perhaps?) to persist these structures to disk.

Here comes the clue: You can use the DPAPI (Data Protection API; look for CryptProtectData in MSDN) to encrypt them without worrying about key/password management.

Nathan Blomquist1-Aug-03 2:12

Nathan Blomquist

1-Aug-03 2:12

For compression you could also check out the J# libraries. They have built in Zip compression.

-Nathan

---------------------------
Hmmm... what's a signature?

Kannan Kalyanaraman1-Aug-03 3:26

Kannan Kalyanaraman

1-Aug-03 3:26

But do you need to ship a separate j# redistributable Unsure | :~

Kannan

Nathan Blomquist1-Aug-03 5:51

Nathan Blomquist

1-Aug-03 5:51

yeah, that is a down side... Frown | :(

But with the app I am working on, it is a small thing compared to the rest (DirectX 9, MSDE, Framework 1.1)... so hell why not Smile | :)

---------------------------
Hmmm... what's a signature?

ryc1-Aug-03 9:20

1-Aug-03 9:20

Forgive me if I am saying something that doesn't make sensen as I am not familiar with DataSets and tables... The amount of data I will be working with will typically be around 100mb (maybe 10k rows). If DataSets and DataTables keep everything in memory without using disk this would give my application took big of a memory footprint (although the access speed would be nice =) ). Also, I would be concerned with the amount of time it would take to serialize and compress the XML, then on load bring it all back in.

A similar application to what I would like is Outlook Express. Potentially the user could have 3 years of email in several folders. In order to access their data all they do is run Outlook, they dont need to start an external database server of any kind... and the memory footprint is reasonable even if you have 10k email messages.

Thanks for the incite, I will definitly look into DataSets and tables more to see what they are all about. It should definitly come in handy some day.

Explain this to me

Ista31-Jul-03 17:33

31-Jul-03 17:33

Customer singleCustomer = new Customer();
PropertyManager pm = this.BindingContext[singleCustomer] as PropertyManager;

What does the as keyword do here is it the same as type casting with the parentheses?

nick

I'm not an expert yet, but I play one at work. Yeah and here too.

Nick Parker31-Jul-03 17:42

31-Jul-03 17:42

Ista wrote:
What does the as keyword do here is it the same as type casting with the parentheses?

The basic idea is the "as" keyword does the same thing as casting except if the object can't be cast it won't throw an error, it will return null so you might want to do something like:

Customer singleCustomer = new Customer();
PropertyManager pm = this.BindingContext[singleCustomer] as PropertyManager;
if(pm != null)
  //... continue on with pm

-Nick Parker

Ista31-Jul-03 17:44

31-Jul-03 17:44

So is that bad programming practice and should only be used in strictly sure cases or it doesnt really matter?

I'm not an expert yet, but I play one at work. Yeah and here too.

Nick Parker31-Jul-03 17:53

31-Jul-03 17:53

Ista wrote:
So is that bad programming practice and should only be used in strictly sure cases or it doesnt really matter?

There isn't a problem with it as long as you are confirming the cast was performed without returning a null value. If you don't check for a null value and the cast returns a null, accessing your object will then throw a NullReferenceException.

-Nick Parker

Rocky Moore31-Jul-03 17:48

Rocky Moore

31-Jul-03 17:48

There are somethings we are bound to miss, but I have a hard time beleiving, I missed this one. Did not pay attention to the usefulness of "AS"...

Tanx!

Rocky Moore <><

Nick Parker31-Jul-03 17:54

31-Jul-03 17:54

-Nick Parker

J. Dunlap31-Jul-03 17:57

J. Dunlap

31-Jul-03 17:57

Well, if your research patterns are anything like mine, you mainly concentrated on the things things that you needed to get the job done, and therefore skipped some of the extras, like this one. Wink | ;)

"Blessed are the peacemakers, for they shall be called sons of God." - Jesus

"You must be the change you wish to see in the world." - Mahatma Gandhi

Nick Parker31-Jul-03 18:00

31-Jul-03 18:00

jdunlap wrote:
Well, if your research patterns are anything like mine, you mainly concentrated on the things things that you needed to get the job done, and therefore skipped some of the extras, like this one.

Just remember, it all maybe useful sometime down the road, mental bookmarks do it for me. Smile | :)

-Nick Parker

J. Dunlap31-Jul-03 18:04

J. Dunlap

31-Jul-03 18:04

Yes.

Speaking of research patterns:
What I find best is to investigate the technologies you come across just enough to get a feel for what they do and what they would be good for. Then learn in detail the technologies relative to the projects you work on. If you happen to need a certain functionality, your "shallow research" will have given you the knowledge you need to narrow down your search to the technologies that will fit the new need.

"Blessed are the peacemakers, for they shall be called sons of God." - Jesus

"You must be the change you wish to see in the world." - Mahatma Gandhi

Ista31-Jul-03 18:17

31-Jul-03 18:17

Well I kind of like the as even though it resembles vb code. I'm used to type casting in c++ where I constantly check for null values but I would much rather throw an exception.

But then again exceptions in .NET are very costly and therefore as would work better.

But then again if your not using reflection you should know what the object was and it would be easier to test for null.

I guess would say the () is better in that it throws an exception unless it is a popular function in that case I would use as to keep it running fast.

My opinion, critices and comments welcome

nick

I'm not an expert yet, but I play one at work. Yeah and here too.

Nick Parker31-Jul-03 18:31

31-Jul-03 18:31

Ista wrote:
I guess would say the () is better in that it throws an exception unless it is a popular function in that case I would use as to keep it running fast.

Umm, the last part of your sentence does not make any sense, care to elaborate? Casting using "as" will not allow your code to run faster than standard casting mechanisms between types, if standard casting fails it will throw a InvalidCastException. If an "as" cast fails returning null and you access a method or property to your null object you will get a NullReferenceException.

-Nick Parker

Ista1-Aug-03 4:28