|
Hello,
I want to store the recent measuring results as well as control samples of older measurements persitently on disk so that the last state can be restored even in the case of a power loss. To make loading the data on startup not unnecessary slow, I want to remove the old, unneeded data from the list.
This means that the measuring results will be continuously appended to the list, but the time when it will be removed is not known in advance.
Would you recommend using a database (ideally some compact system like SQLite so that the user does not have to install a databse server) for this, or are there alternatives?
My idea would be to store the measuring results in one table, using the measuring time as key. In another table, the keys for the last n measuring results would be stored and in a third table the keys of the control samples. If the entries of the first table are not referenced any more from the two other tables, the measuring results should be automatically removed. Is there some concept to acchieve this?
Alex
|
|
|
|
|
This decision should be driven by volume and usage. If you are only processing a few hundred records and do not need to massage the data then an XML or csv file store would be acceptable. Otherwise go for a database.
Why make it complex with multiple tables if you are using text/xml, simply load the entire file into memory and manipulate the collection. Write the collection to the file at regular intervals.
A lot will depend on the criticallity of the data and the frequency of the reads!
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Thank you for your answer.
In a previous version, we wrote the complete XML file - however, due to the relative large amount of data (and the limited resources, as it is an embedded system) - this will not be possible in the final release. So I think the database is the most promising.
Which leaves me with the other questions, how to implement this as a weak reference pattern.
Alex
|
|
|
|
|
LionAM wrote: how to implement this as a weak reference pattern.
Databases generally do not directly support the equivalent of weak reference pattern.
One solution is to append the data as needed in a database table and then run a program in background periodically and check for "old" data and purge them from the database. What "old" means and the frequency of purging depends on the volume of your data.
|
|
|
|
|
As a database kind of guy, I strongly recommend a database.
LionAM wrote: To make loading the data on startup not unnecessary slow
I asssume you do this to show a graph or similar of the recent trend, otherwise I don't see why you would load old data. A database makes this easy by allowing you to query only the recent data. You could sort the data descending by date and take only the top n records or you could query the records where the timestamp is greater than the current time minus soome number of minutes or hours. Or you could purge data out of the table after some period (perhaps after summarizing it into another for historical reasons).
Also, as I don't know what sorts of "measuring results" you are dealing with I'll pass along a concept that you may find useful. It's sort of a data compression technique whereby if the measurements don't change much you don't store every individual measurement. For instance, the temperature of a commercial oven for baking bread or cookies needs to be monitored, it can probably vary a little (let's say a tenth of a degree) from the target temperature without causing an alarm, but if it changes a whole degree (for instance) it should be noted. Furthermore, if the temperature stays close to the target temperature for hours on end (as it should) we do want to log some readings (perhaps at least every fifteen minutes). For these reasons, you may want to develop a system that will only store a reading if the difference from the previous stored reading is outside some specified range or if some specified time span has elapsed.
|
|
|
|
|
In principle, this is exactly what I want - store a live view of a certain time span as well as the status of several control charts. As the statistical spread of the measurement results is of great interest, it makes no sense to store only large fluctuations (then it would perhaps be too late to countersteer ...).
As the data is only used to restore the last program state and therefore stored locally (there is a separate export feature), I would prefer some compact database (without a database server). Which would you recommend? SQLite?
Alex
|
|
|
|
|
LionAM wrote: it makes no sense to store only large fluctuations
You can set the thresholds as appropriate.
LionAM wrote: some compact database
I've dabbled in Sql Server Compact Edition recently, but I can't really give an informed recommendation.
You could also consider serializing the current state to a file periodically.
Maybe my LimitedQueue[^] could be of use to you.
|
|
|
|
|
LionAM wrote: SQLite?
I'd recommend programming against the IDbInterfaces, and actually time/test the various databases that you would want to consider.
- Sqlite is good in terms of performance, once you take in account that everything is in a transaction.
- SqlCe is more integrated into .NET, and would be the preferred option if you plan to synchronize to Sql Server.
- Microsoft Access would be a sweet idea if your client has Office, and if the primary goal of the data is to show reports.
- Excel would work too, almost as fast as Access, and can be accessed over ADO.NET; Not an obvious choice as a datastore, but still very nice for charts.
Bastard Programmer from Hell 
if you can't read my code, try converting it here[^]
|
|
|
|
|
Eddy Vluggen wrote: xcel would work too, almost as fast as Access, and can be accessed over ADO.NET;
Not an obvious choice as a datastore, but still very nice for charts.
And if the primary goal of the data is to be accessed by accountants. 
|
|
|
|
|
Thank you for your answer.
The data is used only to restore the last program state. There is an additional export feature - so I think SQLite ore SQL CE will achieve what I want.
Alex
|
|
|
|
|
Hallo,
I cannot connect to MySQL database from PC outside home network.
I created client aplication in c++ to editing data in MySQL database.
We have 2 PC at home. Both PC has the same TCP/IP address like xxx.xx.xx.xx.
PC with MySQL server has IPv4 address 10.10.11.93. I tried to type xxx.xx.xx.xx/10.10.11.93 but
error is occurred: 2005: unknown MySQL server host "xxx.xx.xx.xx/10.10.11.93" ( 11004).
I can connect to the database from second PC at home wrighting the name of PC with MySQL server.
What I have to write to connect from outside?
Thanks.
|
|
|
|
|
1.fcb1913 wrote: What I have to write to connect from outside?
In a default-installation, it's not possible to make a connection to the server from the outside world. The idea behind that being that you can install a server, without having to worry about me poking at the password over the internet.
You'll have to check the local firewall, and, if it's a home-PC, your router. If that last is the case, you can go to this site[^], and click on "More about you" on the left; it'll load a new page, hopefully with a hostname. If not, you can still connect using the IP-adress, given that the server is configured and reachable.
Bastard Programmer from Hell
if you can't read my code, try converting it here[^]
|
|
|
|
|
On MySQL server is enabled TCP/IP networking, firewall exception for port and enabled root access from remoute machines. I know my TCP/IP - is static.
If I'm trying to connect from outside home network, a firewall don't notice anything.
I can connect from another PC at home network. What I have to do to connect from outside?
|
|
|
|
|
1.fcb1913 wrote: What I have to do to connect from outside?
I suggest you run a little webserver, and use that to verify whether you can connect. If you can't serve a page from the machine, then you won't be able to use a database remotely.
The logical steps would be something like
- verify connectivity
- verify db-server is running
- verify local connectivity
- verify remote connectivity
If your PC is "at home", chances are your router has it's own firewall. Check the manual to verify - the router-setup varies per product. I changed something called "NAT", pointing from the outside to the static IP on the inside-network.
(The numbers inside the network should be different from those outside, and you'd be using the outside-number to connect)
Bastard Programmer from Hell
if you can't read my code, try converting it here[^]
|
|
|
|
|
1.fcb1913 wrote: server has IPv4 address 10.10.11.93. I
That is a private IP address.
It is NOT possible to address a private IP address on the internet. There is absolutely no way to do it.
Your options are
- Figure out your public IP address, which your ISP has given you. There is NO guarantee that you have a public IP address. However if you have one then your cable modem would be the one that has it. You would need to set up your modem to redirect requests to your box.
- Buy a public IP from your ISP. Then do the above.
|
|
|
|
|
Thanks.
I have public IP address.
On a chimney we have white cubic transmitter-receiver and one cable ended with a switch from witch the computers are connected.
I don't know if it is router or modem.
I tried typed my public IP address, IPv4 address, gate IPv4 and server DNS IPv4 address into web browser to set up connection,
but always connect error is occured. I have contacted my internet provider, I hope he'll resolve this.
|
|
|
|
|
Have you tried using port forwarding as Bernhard has suggested?
"Any sort of work in VB6 is bound to provide several WTF moments." - Christian Graus
|
|
|
|
|
Depending on the hardware for connecting your local intranet to the internet, it may be possible to set-up a "port forwarding": that means, you send the request to your public ip address with a port specified, and the router then forwards the request to an internal computer and that port. To get your public ip address, you can use tools like dyndns.
|
|
|
|
|
Yes, port forwarding and using a dyndns tool should be able to work.
"Any sort of work in VB6 is bound to provide several WTF moments." - Christian Graus
|
|
|
|
|
Hi,
I got the following error on test server when I tried to connect to IBM.DB2
Method not found: 'Void IBM.Data.DB2.DB2Connection.set_ParameterStreaming(Boolean)
It worked fine on my desktop with Visual Studio 2010.
I'm appreciate if someone can help.
Thanks,
|
|
|
|
|
Does the test server have everything installed that your desktop has? The same versions? Have you installed your application correctly? With everything it needs?
"If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." Red Adair.
nils illegitimus carborundum
me, me, me
|
|
|
|
|
Hi Friends,
Good noon,
Can we have upload data from excel into SSRS in SQl server,
if any one help me
Regards
Arul.R
|
|
|
|
|
|
Problem: I developed a website with ASP.Net and SQL Server.All data I am accessing from SQL Server.
This site will hosted in public domain as internet website and it is a kind of CRM portal.My Sql server datbase has 11 tables.
My company uses Oracle ERP.Now I want to update data from oracle erp to SQL Server.I have created views in Oracle as data needed in SQL Server.
Means when a customer created/information updated ,it should be updated in SQL Server database.
Is there any solution?
I tried google but there is no suitable answers. Please reply me.
|
|
|
|
|
Uma Shankar Patel wrote: I tried google but there is no suitable answers. Please reply me.
Google for "linked server Sql"; it's a few Sql-statements, allow you to access the Oracle-tables as if they were on Sql Server.
Bastard Programmer from Hell
if you can't read my code, try converting it here[^]
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
