|
Member 7989122 wrote: I will spend Christmas vacation completing my self-made deduplicating backup program. Deduplication is at the file level, not disk page level, so it will save no space for those extensive license headings. But once I started looking around, I realized that at least half of my disk space is taken up by file duplicates. At work, we have some huge file servers running deduplicating at page level; those responsible for it estimates that it reduces the requrements for real disk space to less than a third.
Is this something you intend to share here in CP? 
I'm doing rather well myself when it comes to creating file duplicates, but I have a neighbor who's notoriously bad at it, and he's constantly asking for my help to clean up his file system. I'd love to find a fast utility that can locate duplicate files across drives, and give a number of options as to which ones to delete.
|
|
|
|
|
dandy72 wrote: Is this something you intend to share here in CP?
I might, but I have high respect for the "eating your own dog food" paradigm I need to try it out on my own mess of unstructured backups to see if it helps me as much as I hope.
Also, my backup system is explicitly for user files. I will happily backup your installers -.msi files and the like - but I have no intention of a complete backup of the OS, installed system, registry and similar files. If you experienece a total system crash: Reinstall the system. After that, I come with my user file backups. Nothing more.
Many users may not be satisfied by that. They want a total solution for everything. But for those who are most worried about their own stuff: Maybe my lightweitght backup system could be what makes them start making regular backups!
I take your suggestion to publish it on CP as something I should seriously consider. Thanks for the kick in the a**!
|
|
|
|
|
Member 7989122 wrote: my backup system is explicitly for user files. I will happily backup your installers -.msi files and the like - but I have no intention of a complete backup of the OS, installed system, registry and similar files. If you experienece a total system crash: Reinstall the system. After that, I come with my user file backups. Nothing more.
Same here.
I tried the route of taking images, but soon found that it was an enormous waste of time and disc space -- added to the fact that it restores all the cr@p you should have got rid of, anyway -- so now I just have back-ups of user files and settings.
E.g. the "My Documents" (which I don't use at all for user files) and the "ProgramData" directories contain user settings for a large number of programs, which can simply be copied into a new/restored system after the programs have been installed.
And whenever there's a portable version of an application, or an option to store settings in a file, rather than the registry, that's what I go for.
Incremental back-ups of things like that consume precious little time and resources, and while a re-image may be quicker at restoring a system than this way, how often do you have to do it, and how can you use images to set up new machines?
[edit] It's also dead handy when you're setting up VMs. [/edit]
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
Frankly, all I want is the deduplicating part -- I just want something that'll locate identical files, and let me choose what to do with them in bulk.
|
|
|
|
|
File data duplication is necessary up to a certain point, but no further. The 3-2-1 rule for data persistence - 3 copies, 2 different storage types, 1 off sight - gives us a probability of failure that is arguably less than the mean time between failures of hard disks. But copying today occurs quite naturally as has been outlined in this thread and has led to great and ever-increasing redundancy. The real fear is not being able to find the original copy again, not that it doesn't get stored permanently somewhere.
I would like to propose a different approach to solving the redundancy problem: create a system where there is never more than one virtual copy of each of your files and folders. That is the way Hiveware for MyFiles will work.
|
|
|
|
|
There's a nice list here that goes up to yottabyte (2^80 bytes).
I'd like to think that future measurements might include the elephantyourheadabyte and the yeravingalaughinchabyte.
Whenever you find yourself on the side of the majority, it is time to pause and reflect. - Mark Twain
|
|
|
|
|
Too bad healthcare quality isn't as impressive!
I've often wondered if the reason there hasn't been a cure for cancer developed because it's more profitable to treat it then cure it?
I may not be that good looking, or athletic, or funny, or talented, or smart
I forgot where I was going with this but I do know I love bacon!
|
|
|
|
|
I keep reading those conspiracy theories, but OTOH, think of the money to be made from the patents.
|
|
|
|
|
You are a cynical old bugger!
Never underestimate the power of human stupidity -
RAH
I'm old. I know stuff - JSOP
|
|
|
|
|
Why yes I am!
I may not be that good looking, or athletic, or funny, or talented, or smart
I forgot where I was going with this but I do know I love bacon!
|
|
|
|
|
Cannot count how many blu-ray discs needed to copy the data.
TOMZ_KV
|
|
|
|
|
"...has 33 different vice presidents, each earning over 200 thousand dollars a year. Now, I have spent the last two months analyzing what all these guys do, and I still can't figure it out. One thing I do know is that our paper company lost 110 million dollars last year, and I'll bet that half of that was spent in all the paperwork going back and forth between all these vice presidents."
--Gordon Gekko, Wall Street
|
|
|
|
|
I'd be curious to know the nature of the data. Images take a lot more space than say, your chart history does.
|
|
|
|
|
There is so much important scientific data in the healthcare data (globally) the challenge nowadays is to be able to use that data for research purposes and still keep the patient's privacy. It is scattered all over and not one repository for the info to be used for research purposes.
2,314 billion gigabytes? That a lot.
|
|
|
|
|
How much of that will be in unsecured databases or places where they didn’t change the default password or install security fixes, etc.
|
|
|
|
|
Methinks a good chunk would be comprised of genetic data.
-Each of us has ~3 Billion base pairs (A unique sequence of A,C,G,or T)
-The trend (as costs go down) is moving toward patients having their entire genome sequenced instead of just the "we often see mutations in this specific area".
-For some diseases it may be prudent to get the DNA sequence multiple times (as the person's genetics change), although this would not often be the whole genome in these situations.
-Tumors (often have dramatically different gene sequences) are also now sequenced to better plan treatments for patient A vs. patient B.
-But 3 billion characters to store is the beginning of the story (although it is only known later on in the analysis) because before the sequence is known the DNA sequencers generate, yes as mentioned, images, which are used and analyzed to tell what the DNA sequence is, i.e., these are raw files that must be analyzed, compared to a reference set, etc.
-So, lots of data is generated, per genome. Lots.
Separate from the genetics aspect, hospitals I'm guessing will continue to digitize their patients' health care records.
|
|
|
|
|
I ordered laser labels yesterday for my new printer - company I've never used before. Arrived today, and ... the package feels odd, like there are lumps in it ...
Open it up and find the labels, a "10% off your next order voucher, a nice letter thanking me for using the new website - and a bag of chocolate money as a little Christmas present!
Win win! 
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
modified 19-Dec-18 7:18am.
|
|
|
|
|
|
Nope: I may - or may not - order from them again, depends on price and availability.
But I will eat the chocolate! 
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
QED!
It really is your fault that their long-term customers will be overcharged!
Shame on you!
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
Of course they hope that you will smudge the labels with the chocolate so that you have to order new ones 
Bib on !
|
|
|
|
|
I'm suspicious. They somehow knew exactly how to bribe you. 
|
|
|
|
|
Damn! I knew I should have covered the webcam!
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
That might be worth a free plug man, may as well say who it was :p
"Never attribute to malice that which can be explained by stupidity."
- Hanlon's Razor
|
|
|
|
|
OriginalGriff wrote: and a bag of chocolate money as a little Christmas present!
Probably left overs from Chanukah[^] . . . chocolate coins for children is a tradition (in US, at least).
More importantly: dark chocolate or milk chocolate? That's how you determine if you ever reorder.
| Ravings en masse^ |
|---|
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "If you are searching for perfection in others, then you seek disappointment. If you are seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
