Click here to Skip to main content
14,661,075 members
Home » Discussions » C / C++ / MFC
   

C / C++ / MFC

 
GeneralRe: Exercise Pin
Member 79891229-Jun-20 22:08
MemberMember 79891229-Jun-20 22:08 
GeneralRe: Exercise Pin
Richard MacCutchan9-Jun-20 23:33
mveRichard MacCutchan9-Jun-20 23:33 
QuestionIs there any opensource version of telnet client library(encrypted) available for windows Pin
Member 147721178-Jun-20 2:25
MemberMember 147721178-Jun-20 2:25 
AnswerRe: Is there any opensource version of telnet client library(encrypted) available for windows Pin
Richard MacCutchan8-Jun-20 5:31
mveRichard MacCutchan8-Jun-20 5:31 
QuestionEfficient way to read/write file Pin
manoharbalu7-Jun-20 22:32
Membermanoharbalu7-Jun-20 22:32 
AnswerRe: Efficient way to read/write file Pin
CPallini8-Jun-20 1:28
mveCPallini8-Jun-20 1:28 
AnswerRe: Efficient way to read/write file Pin
Greg Utas8-Jun-20 2:23
professionalGreg Utas8-Jun-20 2:23 
AnswerRe: Efficient way to read/write file Pin
Member 79891228-Jun-20 4:32
MemberMember 79891228-Jun-20 4:32 
manoharbalu wrote:
Please note that I am going to write the structure to the file for 4 days as a maximum period.
4 days is 345,600 seconds. Are you saying that you are going to write 311 gigabytes of data to disk in 4 days? (That is if you don't find a good way to compress it.)

I certainly would not want to manage a single 311 GByte file. Nor would I want to manage 345,600 files ...

Writing a megabyte per second to a modern disk is nothing to worry about. A modern processor could easily handle compression of a megabyte a second, too. A disk to handle a third of a terabyte is no problem. You don't need any super-technology to get this to work. I would be more worried about either handling a 300 GByte file, or a third of a million tiny files (yes, by modern standards, files < 1Mbyte are "tiny").

Compressing to save space is just a technical detail - at application level, you realate to it at its uncompressed size. That comes natural if data are, say, a video stream, but then you would never consider to spread it over a third of a million files. So I am really curious to hear about the nature of the data!

I am not shaing CPallini's sceptisism to general compression algorithms. They work by the principle of recognizing recurring patterns, representing them by a shorter codeword. In totally unprocessed data there is often a large share of repeated patterns, especially in logging data. And in all sorts of readable text. Either, you'll come quite some way using a general method (such as pkzip), or it takes lots of detail knowledge of the information structures to do it much better.

Obviously, if you data is a known media type, such as video, photo or sound, then the mehtods have already been developed; you can get hold of a library to do it - I am thinking of when you to handle you own datatypes for which no dedicated compression method is yet known. Try out general methods first. Only if the results are completely unsatifactory, the files still way too large, may you consider making something of your own. Maybe you can reduce the data volume before you send it to compression, factoring out information elements, omitting data that can be reconstructed from other data etc.

Also consider whether you need losslss compression. If these 900 kB chunks are individual log entries, will they ever later be addressed as individual entries? Maybe you can reduce the data to statistics - averages, sums, whatever. (Like, when you swipe your card at the grocery store, the 49 individual entires on your checkout slip is reduced to a single total sum.)

Giving specific advice would be a lot easier with some details about the nature of the data you want to store.
AnswerRe: Efficient way to read/write file Pin
Joe Woodbury8-Jun-20 14:24
professionalJoe Woodbury8-Jun-20 14:24 
GeneralRe: Efficient way to read/write file Pin
Member 79891228-Jun-20 20:04
MemberMember 79891228-Jun-20 20:04 
GeneralRe: Efficient way to read/write file Pin
Joe Woodbury8-Jun-20 23:25
professionalJoe Woodbury8-Jun-20 23:25 
GeneralRe: Efficient way to read/write file Pin
Member 79891229-Jun-20 3:54
MemberMember 79891229-Jun-20 3:54 
GeneralRe: Efficient way to read/write file Pin
Joe Woodbury9-Jun-20 7:20
professionalJoe Woodbury9-Jun-20 7:20 
GeneralRe: Efficient way to read/write file Pin
charlieg11-Jun-20 11:53
Membercharlieg11-Jun-20 11:53 
GeneralRe: Efficient way to read/write file Pin
Member 798912211-Jun-20 19:39
MemberMember 798912211-Jun-20 19:39 
GeneralRe: Efficient way to read/write file Pin
charlieg12-Jun-20 8:30
Membercharlieg12-Jun-20 8:30 
QuestionDoes WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
PrafullaVedante7-Jun-20 9:02
MemberPrafullaVedante7-Jun-20 9:02 
AnswerRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
Greg Utas7-Jun-20 11:00
professionalGreg Utas7-Jun-20 11:00 
AnswerRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
RedDk7-Jun-20 12:17
MemberRedDk7-Jun-20 12:17 
AnswerRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
CPallini7-Jun-20 20:40
mveCPallini7-Jun-20 20:40 
GeneralRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
PrafullaVedante8-Jun-20 0:09
MemberPrafullaVedante8-Jun-20 0:09 
GeneralRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
CPallini8-Jun-20 1:21
mveCPallini8-Jun-20 1:21 
GeneralRe: Does WaitForSingleObject timeout after 49 days if INFINITE is passed? Pin
Joe Woodbury8-Jun-20 15:04
professionalJoe Woodbury8-Jun-20 15:04 
QuestionTranspose of a matrix Pin
Member 148492464-Jun-20 7:08
MemberMember 148492464-Jun-20 7:08 
AnswerRe: Transpose of a matrix Pin
Member 79891224-Jun-20 7:58
MemberMember 79891224-Jun-20 7:58 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.