Click here to Skip to main content
14,546,262 members

Welcome to the Lounge

   

For discussing anything related to a software developer's life but is not for programming questions. Got a programming question?

The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.
 
GeneralRe: high server config Pin
Nand3211-Mar-20 4:09
MemberNand3211-Mar-20 4:09 
GeneralRe: high server config Pin
lopatir11-Mar-20 4:41
Memberlopatir11-Mar-20 4:41 
JokeRe: high server config Pin
Nand3211-Mar-20 5:05
MemberNand3211-Mar-20 5:05 
GeneralRe: high server config Pin
musefan11-Mar-20 5:48
Membermusefan11-Mar-20 5:48 
GeneralRe: high server config Pin
Nelek11-Mar-20 10:20
protectorNelek11-Mar-20 10:20 
GeneralRe: high server config Pin
musefan11-Mar-20 21:14
Membermusefan11-Mar-20 21:14 
GeneralRe: high server config Pin
Nelek11-Mar-20 21:25
protectorNelek11-Mar-20 21:25 
GeneralRe: high server config Pin
AndyChisholm12-Mar-20 5:41
MemberAndyChisholm12-Mar-20 5:41 
I was Solution Architect a couple of years ago for a system that stored 2PB per week of video and telemetry data from self driving car development.

You are correct that that is a large amount of data, roughly 3.6 GB/s if you think of it as a constant stream. In addition to the challenge of simply storing it, it also has to be simultaneously backed up and analysed.

Data arrive at the data center from the many cars in the field not via cables but on SSD based cartridges that have to be read in via reader stations. The cartridges are then returned to the field to collect more data.

Storage for production and "backup" is provided by hundreds of Dell-EMC Isilon NAS nodes. Analysis of the data is done in more than 300 servers each with 32 cores and 0.5 TB of RAM.

It sounds mind blowing but there really are companies collecting, storing and processing data on that scale.

Andy
GeneralRe: high server config Pin
OriginalGriff11-Mar-20 6:08
mveOriginalGriff11-Mar-20 6:08 
GeneralRe: high server config Pin
  Forogar  11-Mar-20 4:27
professional  Forogar  11-Mar-20 4:27 
AnswerRe: high server config Pin
musefan11-Mar-20 2:22
Membermusefan11-Mar-20 2:22 
GeneralRe: high server config Pin
Nand3211-Mar-20 4:15
MemberNand3211-Mar-20 4:15 
AnswerRe: high server config Pin
W Balboos, GHB11-Mar-20 2:24
mveW Balboos, GHB11-Mar-20 2:24 
GeneralRe: high server config Pin
musefan11-Mar-20 2:30
Membermusefan11-Mar-20 2:30 
AnswerRe: high server config Pin
Dan Neely11-Mar-20 2:57
MemberDan Neely11-Mar-20 2:57 
AnswerRe: high server config Pin
Daniel Pfeffer11-Mar-20 5:04
professionalDaniel Pfeffer11-Mar-20 5:04 
GeneralRe: high server config Pin
musefan11-Mar-20 6:08
Membermusefan11-Mar-20 6:08 
GeneralRe: high server config Pin
Daniel Pfeffer11-Mar-20 6:39
professionalDaniel Pfeffer11-Mar-20 6:39 
AnswerRe: high server config Pin
lw@zi 11-Mar-20 5:33
professional lw@zi 11-Mar-20 5:33 
AnswerRe: high server config Pin
Jörgen Andersson11-Mar-20 6:08
professionalJörgen Andersson11-Mar-20 6:08 
GeneralRe: high server config Pin
musefan11-Mar-20 6:21
Membermusefan11-Mar-20 6:21 
AnswerRe: high server config Pin
Gerry Schmitz11-Mar-20 7:01
mveGerry Schmitz11-Mar-20 7:01 
AnswerRe: high server config Pin
AndyChisholm12-Mar-20 0:43
MemberAndyChisholm12-Mar-20 0:43 
GeneralRe: high server config Pin
Nand3212-Mar-20 3:00
MemberNand3212-Mar-20 3:00 
GeneralCCC Pin
pkfox10-Mar-20 23:38
professionalpkfox10-Mar-20 23:38 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.