The Lounge - CodeProject

There is something else that could affect our electronic civilization that has a much higher probability, in fact more than most people think, and we will likely be ill prepared for it just like the last pandemic: Coronal Mass Ejections (CME). Put your backups in a Faraday cage! Which institutions and systems are prepared enough?

None.

It would knock out the power supply. That alone would kill our culture within 24 hours.

Our homes would turn to ice (right now, my outdoors temperature is 12.4 Celsius below freezing, 10 F).

We couldn't charge our cars.

Our smartphones would go dead within a day or two.

We couldn't charge our portables, and even with a diesel generator to power our desktop PC, the concentrator for our fiber internet connection, or maybe the local ISP itself, is dead from lack of power.

The food in your freezer and fridge would rot - sooner in summertime than in winter, so what would you prefer: Freeze to death, or starve to death?

What about your 'social' contacts, the socalled 'social' media?`
What kind of entertainment would you have available?
Could you read a book in the winter evening? (And: Do you have a book? Smile | :)

)

Your local plant for filtering your drinking water probably depends on a huge amount of electricity. Expect it to close down.
Same with your waste water / sour processing plant. Expect it to close down.

If you want to look at your own electricity dependencies alone, disregarding things such as water supply and wastewater processing: Why don't you tonight, this Friday night, abruptly turn OFF your main electric switch (maybe you don't have a switch but you can unscrew the main fuses) without making any preparations at all. Leave the main switch off (or main fuses unscrewed) until Sunday night 8 days from now, and then tell how well you and your family fared. And how you did it, in which ways you compensated for the lack of electricity.

I don't think very many families would dare to take up this challenge. And I believe that at least half of those who dared to, would ask for at least one day to prepare for it - have all their batteries charged, their gas tanks filled up etc.

Very few of us are anywhere close to prepared for such an event.

They might be related but it's just a snafu, not a conspiracy.

Paul Sanders.
If I had more time, I would have written a shorter letter - Blaise Pascal.
Some of my best work is in the undo buffer.

You're right, I attribute this to incompetence rather than malice.

But the main point remains...if something's important to you, you can't have the only instance of it existing only on a cloud service.

Absolutely

Paul Sanders.
If I had more time, I would have written a shorter letter - Blaise Pascal.
Some of my best work is in the undo buffer.

I do.

An external CD drive on a usb cable to boot Acronis 2014 and a 2tb usb spinner disk to backup images to. I do it while were sleeping, It will be done in the morning. Yes it takes effort, but thinking BIG data has you back is just complacent and lazy.

I have an external Sabrent drive that reads SDD and SATA drives, treating them like really big flash drives. Since I replace my primary HD every 2 years, I have a stack of old drives that are great for backups.

Critical files are backed up onto DVD and finalized. As I've stated in the past, a finalized CD or DVD is ransomware proof, and they're dirt cheap so making multiple copies is cost effective.

I was surprised when my friends and coworkers one by one revealed that they no longer had any player for CD/DVD. I wanted to introduce them to some music or movies I had in my archive, but they couldn't make use of the disk. If I couldn't provide a URL for an online version, they shrugged and started talking about music/movies that are available online.

This started at least 5-6 years ago. Today, I don't know of anyone who has bought a PC with an optical reader for five years. One of my friends still have an old PC with one, but he boots up that machine only when he needs to run some old software that doesn't run on W11.

In earlier years, visiting friends with a disc in your hand was a social thing. We saw the movie together, or listened to the music. In those cases where I could dig up a URL for a friend, we never saw the movie or enjoyed the music together; he went home and watched / listened alone. In the very best case, he reported some reactions next time we met. Usually not.

So if my house burns down, and my computer media is melted, even if I had been keeping off-site optical backup disks, I would not know of anyone who could help me retrieve my files. I would have to go to some commercial and probably expensive service provider to have it done.

There is another problem with finalized DVD disks: The Tao of Backup[^], the Second Head:

The novice asked the backup master: "How often should I backup my files? It has been a month since my last backup."
The master replied: "Just as night follows day, and Autumn follows Summer, so should backups follow work. As you work, so should you backup that work."
The novice said: "I work each day".
The master replied: "Then you should backup each day".
The novice replied: "I agree, but right now I haven't got time to make a backup, as I have too much work to do."
Upon hearing this, the master fell silent.

Backing up to a finalized optical disc and bring the disc offsite every day, is beyond my working habits. Smile | :)

(For those unfamiliar with The Tao of Backup: 26 years old and still 100% true! Read and enjoy it!)

Yep, the days of CD/DVD writers are numbered. My last laptop went even further - no Ethernet port, either! And my most recent PC doesn't even have a knockout in the front panel to install a writer if I wanted to do so.

Will Rogers never met me.

Do you have offsite copies?

Thanks for mentioning it.

That is the only good argument in favor of online backup: Your backup is still there even if your house burns down. Many of my friends make backups of private files, but I know of noone who regularly brings a copy offsite.

Never trust someone else with your stuff.

Plus one for that.
Truenas is my cloud. Turn on, do stuff, turn off.

>64
Some days the dragon wins. Suck it up.

I hardly trust myself with my stuff.

Although fire/flood can spoil that.

Even theft. Some guys (plural) stole cases a talcum powder off a truck so who knows they might run off with something critical which would not otherwise seem to have value.

IMHO, it's important to also back up the top three or five things in the mind, so that whatever's lost can be recovered with just self effort, even if it takes time. Am just talking of code backups here.

modified 28-Nov-23 10:31am.

If anyone actually expects ANY cloud service to respect their stuff then they deserve the rude awakening.

The Cloud is nothing but another tool/option. Treat it as the be-all-end-all thing the Cloud Sales people market it as then you're just waiting for a rude awakening. Treat for what it really is and you'll avoid these headaches.

Keep in mind of course that one should consider carefully what 'backup' actually means.

Specifically even though it is seems to be working, is it being verified on a regular basis? So can one actually get to that old data?

jschell wrote:
Specifically even though it is seems to be working, is it being verified on a regular basis? So can one actually get to that old data?

Even though that's common knowledge, I have little reason to believe even someone as big as Google actually does verify its backups.

jschell wrote:
Specifically even though it is seems to be working, is it being verified on a regular basis? So can one actually get to that old data?

A former employer had a rigorous backup schedule for the office file server. Incremental backup Monday through Thursday, Friday was a complete backup that was saved for 12 weeks (tape cartridges were recycled every 12 weeks) and the last full backup of the month was saved for 12 months. We had a collection of 30-40 tapes that were all well-labeled and cycled through the process consistently.

This system had been in place for 5 years and ran flawlessly.

Then my team had to recover a file from a backup. We spent 8 hours trying to pull that file. Then any file. Then from other backups.

We had a box of tapes and not one of 'em was readable. Completely worthless. We reported this to management.

They replaced the system a year later ...

It gets better ... this was in the time when 386 was the main PC architecture. The backup system was a board that only ran in a 286, so the office used an ancient (well, ancient in computer years) PC that we could not update because the software that came with the board didn't run on newer OS. The office was on Win 3.11, can't remember what the 286 box required.

I'm still reading that those who need to perform massive backups still prefer tapes. I've always been a skeptic.

Isn't everything about a tape backup solution proprietary?

If you have a specific type of tape, you have to have the matching drive. The reverse is also true, if you have a specific drive, you can't just use any tape in it. I suppose there might be compatible makes/models (of both drives and tapes), but things are still not as interchangeable as "hard drive A vs hard drive B".

And then the software tends to be proprietary, and not every OS can just read any random file from a given tape. If you've done a backup with software XYZ, you have to do software XYZ to do the restore. Imagine finding a bug in the restore portion of the software, and the manufacturer has gone out of business years prior.

That would worry me, unless I had an unlimited budget and could purchase all the redundancy I'd need to be comfortable. How many tape drives is that? I have no idea. Imagine having stocked up with 5 spare tapes drives...with a SCSI interface.

Obviously big business has different needs than I do. I'm perfectly happy doing "xcopy backups" to multiple hard disks. No proprietary software, you can access any file in any folder instantaneously, no matter what OS you mount the drive with (within reason), hardware interfaces will remain standard for years if not decades before being deprecated, you can buy them in various capacities from different manufacturers, and they're relatively cheap.

I tend to buy them in sets of 3 - one live, a disconnected backup, and an off-site backup. When I outgrow a set, it becomes more or less a permanent archive of "things as they existed at that point in time".

When SATA became the standard, I had plenty of time to migrate the data from IDE drives to SATA drives. Whenever SATA gets replaced, I'll just do the same.

The non-starter is when you have massive amounts of data. I try to keep my entire backup set on a single drive, and I currently have a trio of 16TB drives. If my archive suddenly grew to double that, I wouldn't have a "single-drive solution", as 32TB drives are still a long way off. I'd hate to have to maintain a RAID, and multiple backups of it...

You are pointing out something that is very essential. As I said in a different post, I guess there are millions of old backups that are useless, for the reasons you mention. In addition comes what you are not mentioning: Having the right software (that includes the OS) to run the application to interpret your files.

But I'd also like to mention that there were standard even in the old days. Open reel 1/2" tape formats followed international standards from the mid 1960s. The standards passed then was essentially to accept the IBM 'proprietary' format as the standard - which a lot of other manufacturers already had done. You certainly could move a tape from one machine to a different one.

QIC (Quarter Inch Cassette) tape was standardized in the early 1980s. The good thing is that there are so many QIC standards to choose from Smile | :)

, but the standards were vendor independent.

We are all familiar with memory sticks: The USB standard defines a mass storage virtual drive. SCSI did, too, in 1982. You could plug a disk or a tape station into any SCSI socket; it would identify itself as a disk drive honoring the virtual disk specification, a tape station honoring the virtual tape drive specification, and so on. There were virtual scanners and printers, too, in the specification.

So it wasn't all proprietary. If you have an old QIC tape, you must find a physical QIC reader; it can be of any make, as it handles the one format on your tape, which is by an international, vendor-independent standard. There are quite a few of them, all international standards. Make sure that your QIC reader is a SCSI one (most of them were). Then you can plug it into any PC with a SCSI interface, which can be found almost anywhere, right? Smile | :)

It can be of any make. Windows, *nix and other OS-es see the same files on the tape.

But cross your fingers for that SCSI interface: I was actively in favor of SCSI for a few years: One great standard for everything! It was daisy chained, not a tree, but the chain could have all sorts of devices in it, much like USB. Except that when I had to buy an adapter cable for the eight plug 'standard' for SCSI, and the salesman said I was lucky: There are fourteen different plugs used with SCSI! (this is 25+ years ago; I guess there are more now), then I called a halt. No more SCSI for me.

USB was sliding down that same slope: I've got A, A 3.0, B, B 3.0, Mini B, Micro B, Micro B 3.0. I have equipment requiring all of these, then there are some I never used. When C arrived, I said: OK, this is your last chance. If you come with anything new to replace C, you won't. Not in my house. Then I am through with USB, like I was through with SCSI.

I wish there was a (non-proprietary) standard. There are loads of (non-proprietary) standards - that's the problem.

Of course we cannot stop progress. We must accept new standards. If you apply for a patent, your new invention must have a minimum 'invention height'. It must add something 'significant' (as far as I understand, this is not as strongly enforced in the US of A as it is in Europe). We should follow the same principle in standards. B micro did not add anything 'significant' to B mini, and should have been rejected. Smartphone SIM card come in (at least) 3 different sizes; these are certainly not distinguished by significant 'invention height'. In software: We should accept a new programming languages only if it adds significantly to the ones we have. Similar with Linux file systems (although there is a natural limit of 42 alternatives that probably won't be exceeded).

I do not see 'proprietary' as a big problem. A much bigger problem is that various vendors who could have used an existing standard rather makes a new one - and we accept it, because it it non-proprietary / 'open'. Even if there is no need for it.

dandy72 wrote:
Isn't everything about a tape backup solution proprietary?

Up until about 20 years ago tape was the only solution. So that really wasn't a consideration.

Not to mention that automation systems even with tape were very expensive. So that meant a IT guy needed to swap tapes in and out.

dandy72 wrote:
The non-starter is when you have massive amounts of data

There are other potential problems. For example a database with encrypted columns. Where is the key?

Or an encrypted file. Where is the key or passphrase for that.

Those are not necessarily relevant just to disaster recovery either. If you have 7 years of backups, which is often considered a norm for financial data, and you get a subpoena you will need for legal reasons to show that you tried to recover that data.

At one company I considered disaster recovery just not possible. The DBA stated it would take more than a week to restore the database from the back up. But in addition there was no back up for the proprietary machine that ran the database. I figured best case scenario was that it would take 6 weeks to recover if the hardware failed. They originally budgeted an onsite back up machine but that was nixed due to the cost. Nixed by those above operations/development.

jschell wrote:
Up until about 20 years ago tape was the only solution.

I would raise that to 30 years, if you talk about the only solution. 21 years ago, in 2002, my employer at the time - the National Library of Norway - threw out their tape robot, replacing it with a big wall of RAIDed hard disks. Tape was considered an outdated technology; certainly not 'the only solution'.

So that meant a IT guy needed to swap tapes in and out.

This tape robot, thrown out in 2002, was fully automated. No manual handling of tapes. I am not sure how old the robot was; it had been there a few years when I entered the job. Their policy was to migrate all archived data to new media every 5 to 10 years, and the move to RAID disks were considered overdue. So I guess the tape robot was, at the very latest, installed around 1995.

For example a database with encrypted columns. Where is the key?

Certainly an essential question. But it is the same regardless of which backup technology you choose.

With one slight modification: When I am in physical control of the backup media, I consider encryption for the purpose of the backup itself to be less important. If I were to send the files to some other backup service, there is a lot more that I would encrypt. But I take your scenario to mean a database encryption that is done in any case, not just for backup purposes.

trønderen wrote:
Tape was considered an outdated technology; certainly not 'the only solution'

In terms of cost and process it was about 20 years ago that a transition became possible.

For the example you gave they were already spending a lot on the existing system and the new system would have also have been expensive. Smaller organizations could not have afforded either solution. And there are many more smaller organizations than larger. So in effect manual tape was the only solution. Even more so since at that point even small companies started increasing the amount of data that was stored digitally (versus paper.)

trønderen wrote:
But it is the same regardless of which backup technology you choose.

Yes but my point was more about the reliance on older back ups. So even if the technology was till accessible the information to use it might no longer exist.

Welcome to the Lounge