Click here to Skip to main content
15,887,746 members
Home / Discussions / C#
   

C#

 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Eddy Vluggen19-Aug-15 4:54
professionalEddy Vluggen19-Aug-15 4:54 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Rob Philpott19-Aug-15 5:09
Rob Philpott19-Aug-15 5:09 
Variable length byte arrays, somewhere between 50 bytes and 5KB, numbered 0, 1, 2, 3 etc. up to 500 million of them. No more than 1TB in total.

Requirement is quite simple really, retrieve byte array by index as fast as possible (random access), provide sequential access as fast as possible (all of them, one by one - obviously this will take a while).

My current thinking is a format of 4 byte length + byte array repeated, and a separate index file to provide the necessary indirection to cope with the variable length nature.

If there were 100 of them you'd stick them in an array, so this really is just a problem of scale. Quite an interesting one.
Regards,
Rob Philpott.

GeneralRe: Performance of FileStream, scale is stalking me Pin
Eddy Vluggen19-Aug-15 7:09
professionalEddy Vluggen19-Aug-15 7:09 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Rob Philpott19-Aug-15 7:18
Rob Philpott19-Aug-15 7:18 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Eddy Vluggen19-Aug-15 8:11
professionalEddy Vluggen19-Aug-15 8:11 
AnswerRe: Performance of FileStream, scale is stalking me Pin
Afzaal Ahmad Zeeshan19-Aug-15 5:41
professionalAfzaal Ahmad Zeeshan19-Aug-15 5:41 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Rob Philpott19-Aug-15 5:52
Rob Philpott19-Aug-15 5:52 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Afzaal Ahmad Zeeshan19-Aug-15 5:58
professionalAfzaal Ahmad Zeeshan19-Aug-15 5:58 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Rob Philpott19-Aug-15 6:10
Rob Philpott19-Aug-15 6:10 
GeneralRe: Performance of FileStream, scale is stalking me Pin
Pete O'Hanlon19-Aug-15 6:18
mvePete O'Hanlon19-Aug-15 6:18 
QuestionMSBuild The type or namespace name 'Class1' could not be found (are you missing a using directive or an assembly reference?) error Pin
Txalaparta18-Aug-15 23:18
professionalTxalaparta18-Aug-15 23:18 
QuestionRe: MSBuild The type or namespace name 'Class1' could not be found (are you missing a using directive or an assembly reference?) error Pin
Eddy Vluggen19-Aug-15 0:44
professionalEddy Vluggen19-Aug-15 0:44 
AnswerRe: MSBuild The type or namespace name 'Class1' could not be found (are you missing a using directive or an assembly reference?) error Pin
Txalaparta19-Aug-15 2:05
professionalTxalaparta19-Aug-15 2:05 
GeneralRe: MSBuild The type or namespace name 'Class1' could not be found (are you missing a using directive or an assembly reference?) error Pin
Eddy Vluggen19-Aug-15 2:39
professionalEddy Vluggen19-Aug-15 2:39 
QuestionSession Ends after any change made in asp.net Pin
Salman Azhar18-Aug-15 22:10
Salman Azhar18-Aug-15 22:10 
QuestionGet all the network activity data in Resouce Monitor Pin
LeHuuTien18-Aug-15 20:35
LeHuuTien18-Aug-15 20:35 
AnswerRe: Get all the network activity data in Resouce Monitor Pin
Richard MacCutchan18-Aug-15 22:00
mveRichard MacCutchan18-Aug-15 22:00 
GeneralRe: Get all the network activity data in Resouce Monitor Pin
LeHuuTien19-Aug-15 16:23
LeHuuTien19-Aug-15 16:23 
GeneralRe: Get all the network activity data in Resouce Monitor Pin
Richard MacCutchan19-Aug-15 20:51
mveRichard MacCutchan19-Aug-15 20:51 
AnswerRe: Get all the network activity data in Resouce Monitor Pin
Nitzan Levi19-Aug-15 9:57
Nitzan Levi19-Aug-15 9:57 
GeneralRe: Get all the network activity data in Resouce Monitor Pin
LeHuuTien19-Aug-15 16:05
LeHuuTien19-Aug-15 16:05 
Questionertan tike dayview calendar control Pin
Member 1189642718-Aug-15 20:01
Member 1189642718-Aug-15 20:01 
AnswerRe: ertan tike dayview calendar control Pin
Pete O'Hanlon18-Aug-15 21:27
mvePete O'Hanlon18-Aug-15 21:27 
AnswerRe: ertan tike dayview calendar control Pin
Nitzan Levi19-Aug-15 10:08
Nitzan Levi19-Aug-15 10:08 
GeneralRe: ertan tike dayview calendar control Pin
Member 1189642720-Aug-15 19:09
Member 1189642720-Aug-15 19:09 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.