Click here to Skip to main content
15,879,348 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I created .Net C# software using SQLite3.0 database. In the software, data is inserted almost per minute, every day. But very rare times (about once in a month), it loses its data like it never was.

My all tables have identity columns. To check the loss, I writes inserted time of new row in every table. When the loss occurred, I observed rows of an hour was lost. But the identity values were not skipped and continued finely. I checked my transactions but they were fine.

There is one connection created and opened when the program started. And that connection is used whole runtime without close and reopen. At runtime, there are many db actions such as insert, update, delete, select.

Can it be a reason for the loss?
Should I open and close a connection for every db actions?
Posted
Comments
Mehdi Gholam 16-May-15 0:55am    
Add logging to your application to see what is happening and any exceptions.

1 solution

If your Identity values don't show gaps, then rows aren't being inserted and deleted - which leaves us three options:

1) Your application is not receiving the data during the "missing" period, so there is nothing to insert.
2) Your application is not trying to insert the data.
or
3) The data for that hour is wrong in some way and SQLite is rejecting it.

If there is nothing that is common to the date time combination when the errors occur (and I assume you've looked at that) then you need to start working out at what stage the data is getting lost. Start by logging what is occurring: you need data (and lots of it) to find intermittent problems.

I'd start by adding something to record the "raw" input data - a text or binary file that gets appended should be fine (and a mechanism so that you can discard the data for days which don't contain missing periods to prevent the log getting too full)
Then a log of the data you are trying to output to SQL.
And finally a good check to make sure you aren't swallowing any exceptions that might be giving clues if you logged that as well (and I'd log every exception you catch to be on the safe side)

Adding a timestamp to each log entry is a damn good idea!

Once you have the logs covering at least one "missing" period, you can use them to identify at least at what stage the data is going missing - which should help you "target" the problem. It may take a couple of runs to get the right information, focussing in each time to work out where it's being lost - intermittent problems are never easy to find!

Good luck!
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900