 |
|
|
I test on 2TB Virtual Disk,it simulate a 2TB disk,1024,2048,4096 bytes/secotr,this app can not work.I found it on http://www.2tdisk.com
|
|
|
|
|
|
|
|
|
|
Great concept. Why the 3? Because as an article, you left out the meat of this thing. You've got no discussion of how the code works. You've covered the concepts behind the storaget system and how to use the library, but completely left out how the code works and manages the data.
|
|
|
|
|
|
|
I test on 2T Disk Simulator,it simulate a 2TB disk,4096 bytes/secotr,this app can not work.I found it on http://www.2tdisk.com
|
|
|
|
|
|
|
Public Class MSG1
{
My.Vote = 5
string ImSaying = "This just rocks!"
string ImMaking = "A new project called 'VirtualFS'."
string null = "A simple to use" _
interface;
null = "but only basic file functions. MDI user interface also included."
#region Miscanneleus_stuff_I_have_to_say
{
'I credited you, I really like this project. Did you know that I coded VirtualFS with VB.NET but used the C# demo as source?
'I converted it myself, no programs, and it did not find any errors there! Good right? But I am just nine years old, so that does not have to do anything.
}
}
PS: That code was just a joke. I just like codes.
|
|
|
|
|
|
|
Very usefull !! 
|
|
|
|
|
|
|
thank you for this great article
but i have a question pls:
can i expose files to another proccess like sql server and make a database restore from a backup file stored in this File System.
thanx
|
|
|
|
|
|
|
Hi,
First of again, good work on the Virtual Storage, it works perfect.
I do have a question, I would like to put my own application (using your QVFS) on the network. I know of the security issues, that's not the problem.
The app runs fine, but what if more than one user would use it (from a different computers)?
The virtual storage (which I use to store my serialized objects) could be accessed by multiple users, hence given an error that the file is in use.
Do you know of a way to let them access it, as soon as released by the other users?
Or can it be accessed simultaniously in any way?
Inside my application a use synclock or mutex to prevent different threads from accessing the storage at the same time, but how can this be done with two or more completely different instances of the program (on different computers)? Is there a way to control this? Maybe by using a control file or something?
For instance I would like to store some troubleshooting documents, but at the same time my college wants to added or read one, accessing the same virtual storage placed on the network.
I would like to prevent the use of any kind of database system, your virtual storage file system is all I need.
Does this make any sense?
Many thanks in advance.
Kraeven
|
|
|
|
|
|
|
Hi,
I think you can use the following solution:
Add a simple web service or WCF at the remote computer which the virtual storage is placed, use this service to access the virtual storage. And every incoming storage content request is placed in a queue. The service pick up one request in the queue at a time and read the data and return it to client. This can avoid multiple access to the virtual storage. But it may reduce some performance.
I think the above approach can solve your problem.
|
|
|
|
|
|
|
Hi Tony,
Thanks for the quick reply, any hints, examples on how to make such a web service or WCF?
Never done that 
If at all possible I would like to avoid putting anything on the server except the files them self, mainly because if I would install somtehing on the server, I might as well put a Database system on it, like MS SQL or MYSQL...
I was thinking about a sort of access control file so that the clients arrange the access and I don't have to bother the networking group and plead to be allowed to put a service or something on the server 
Kraeven
|
|
|
|
|
|
|
Into function GetEmptyDataLocations(...) I suggest this modification:
................
If ForcedClusterStartIndex <> -1 Then
i = 1
totalLength = 1
intResult(0).StartIndex = ForcedClusterStartIndex
intResult(0).Length = 1
If NeededClusterLength = 1 Then Return intResult
ReDim Preserve intResult(i) // <<<<<<<<< THIS LINE MUST BE ADDED
Else
...............
David Bullmount
aaaa
|
|
|
|
|
|
|
Would there be any difficulty making every 1024th cluster a FAT cluster? That would allow for the FAT system to be grown arbitrarily large, while keeping everything in one file. Having two files would seem to increase the likelihood of things getting out of sync.
Alternatively, how hard would it be to store the FAT at the beginning of the file system, and specify that cluster 'n', where 'n' is less than the number of FAT blocks, will get shoved to cluster 1024*n? This would cause the first file in the system to become fragmented, but would keep the FAT together at the start of the file.
Finally, while it is certainly convenient to have files stored with a chunk size that's a power of two (so finding byte 'n' of a file can be done with shifts and masking) I wonder whether reliability and data recovery might be improved by adding a tag to each cluster identifying that the cluster is block 'n' of file 'f'. Under such a scenario, even if the FAT got completely hosed, all data could still be recovered.
My own file-system experience, btw, is with flash-based systems where the hardware block size is just over a power of two; block tags work quite nicely in such a scenario. I also use a sequenced write scheme to allow for atomic writes. If power is lost while writing data, the system will either commit or roll back the changes. I'm not sure such a thing can be implemented in a nested file system, since there's no guarantee of the sequence in which file operations will be performed in hardware. Nonetheless, it might be nice to have some assurance that if the hardware doesn't perform things out of sequence, unexpected program termination won't corrupt the file system. Unfortunately, a FAT system can easily get corrupted under such a scenario.
|
|
|
|
|
|
|
Thanks for your advice, your experience in file system is very interesting to me. And my purpose for this article is just a to simulate and learn how FAT32 works, as you know the FAT32 is a simple file system, so no advanced functions are extended in my virtual file system. I'll keep studying in this. Thanks again for sharing your experience.
|
|
|
|
|
|
|
what's the differnt to ISO file?
Regards,
unruledboy_at_gmail_dot_com
http://www.xnlab.com
|
|
|
|
|
|
|
While I am not familiar with the details of .ISO files, my impression is that those are intended to hold a nested set of files whose contents are fixed. File systems designed for such purposes often waste less space than those designed for "live" files, but there's no good way to increase the size of a file except by rewriting the whole thing in a new location.
|
|
|
|
|
|
|
|
|
|
This is more than likely a test case for the bigger victim... LINUX.
While I don't think MS would go after Linux under normal circumstances, should Linux get out of line(read too successful), well, stuff happens.
This is not evil, it's just business(at least that's what they said in the Godfather, lol).
|
|
|
|
|
|
|
Hi everybody,
I'm having problems with deleting folders from the Virtual File System (QVFS)
Below you'll see some example code (simplified)
I think I just don't understand what's actually happening, when I do a deletion from the virtual folder. The deletion seems to succeed, but when I would re-create the folder, I get the error below :
Error occured while trying to delete file from the Virtual Storage
No file data detected, the file may be physically deleted
Filename : C:\Documents and Settings\acketk\Desktop\VB - Stuff\VirtualSpace\qvfsKE\QVirtualFileSystemManager.vb
Method : System.IO.MemoryStream GetVFileData(Int32, Int32)
LineN° : 946
Filename : C:\Documents and Settings\acketk\Desktop\VB - Stuff\VirtualSpace\qvfsKE\QVirtualFileSystemManager.vb
Method : System.IO.MemoryStream GetVFileData(QVFS.FileEntry)
LineN° : 903
Filename : C:\Documents and Settings\acketk\Desktop\VB - Stuff\VirtualSpace\qvfsKE\QVirtualFileSystemManager.vb
Method : System.IO.MemoryStream GetVDirEntry(System.String, System.IO.MemoryStream, Int32 ByRef)
LineN° : 1244
Filename : C:\Documents and Settings\acketk\Desktop\VB - Stuff\VirtualSpace\qvfsKE\QVirtualFileSystemManager.vb
Method : System.IO.MemoryStream GetVDirEntry(System.String, Int32 ByRef)
LineN° : 1203
Filename : C:\Documents and Settings\acketk\Desktop\VB - Stuff\VirtualSpace\qvfsKE\QVirtualFileSystemManager.vb
Method : Boolean IsExisted(System.String, Boolean)
LineN° : 356
Filename : C:\Documents and Settings\acketk\Desktop\VB - Stuff\VirtualSpace\qvfsKE\QVirtualFileSystemManager.vb
Method : Void CreateVDirectory(System.String)
LineN° : 295
Filename : C:\Documents and Settings\acketk\Desktop\VB - Stuff\VirtualSpace\VirtualSpace\VirtualStorage.vb
Method : Void btnAddFolder_Click(System.Object, System.EventArgs)
LineN° : 364
No file data detected, the file may be physically deleted ==> I know that's what I wanted
*************************************************************
Source Code :
*************************************************************
Sub DeleteVFile(byval VPath as string)
Dim vfm As QVFS.QVirtualFileSystemManager = Nothing
Try
vfm = New QVirtualFileSystemManager()
vfm.OpenQVFSFile(SettingsFile)
If vfm.IsExisted(VPath) then
vfm.DeleteVFile(VPath, True)
End If
vfm.CloseQVFSFile()
vfm = Nothing
Catch ex As Exception
MsgBox("Error occured while trying to delete file from the Virtual Storage" + vbCrLf + ex.Message + vbCrLf + GetErrorInfo(ex))
Finally
If Not vfm Is Nothing Then
vfm.CloseQVFSFile()
vfm = Nothing
End If
End Try
End Sub
Sub DeleteVFolder(byval VPath as string)
Dim vfm As QVFS.QVirtualFileSystemManager = Nothing
Try
vfm = New QVirtualFileSystemManager()
vfm.OpenQVFSFile(SettingsFile)
vfm.DeleteAllVFiles(VPath)
vfm.DeleteVFile(VPath, True)
vfm.CloseQVFSFile()
vfm = Nothing
Catch ex As Exception
MsgBox("Error occured while trying to delete file from the Virtual Storage" + vbCrLf + ex.Message + vbCrLf + GetErrorInfo(ex))
Finally
If Not vfm Is Nothing Then
vfm.CloseQVFSFile()
vfm = Nothing
End If
End Try
End Sub
Sub AddVFile(byval VPath as string, byval FPath as string)
Dim vfm As QVFS.QVirtualFileSystemManager = Nothing
Try
vfm = New QVirtualFileSystemManager()
vfm.OpenQVFSFile(SettingsFile)
vfm.AddNewVFile(FPath, VPath, True)
vfm.CloseQVFSFile()
vfm = Nothing
Catch ex As Exception
MsgBox("Error occured while trying to delete file from the Virtual Storage" + vbCrLf + ex.Message + vbCrLf + GetErrorInfo(ex))
Finally
If Not vfm Is Nothing Then
vfm.CloseQVFSFile()
vfm = Nothing
End If
End Try
End Sub
Sub AddVFolder(byval VPath as string)
Dim vfm As QVFS.QVirtualFileSystemManager = Nothing
Try
vfm = New QVirtualFileSystemManager()
vfm.OpenQVFSFile(SettingsFile)
vfm.CreateVDirectory(VPath)
vfm.CloseQVFSFile()
vfm = Nothing
Catch ex As Exception
MsgBox("Error occured while trying to delete file from the Virtual Storage" + vbCrLf + ex.Message + vbCrLf + GetErrorInfo(ex))
Finally
If Not vfm Is Nothing Then
vfm.CloseQVFSFile()
vfm = Nothing
End If
End Try
End Sub
******************************************************
I know it has something to do with the vfm.DeleteVFile function
(Can this be used on folders too?)With files this works ok!
If I would say physical delete = False
The folder gets removed from the treeview/listview but it's still in the data file,
I don't want that, it needs to be permanent deleted.
Also if not permanent deleted and I would re-create the folder, it isn't shown in the treeview/listview (probably because the status of the folder is still deleted), but it can be accessed by code.
If I would say physical delete = True, the file/folder is deleted ok, the listview/treeview is updated ok. Al seems fine, like it supposed to look.
But if I would re-create the folder (same name and virtual path) I get an error upon creating the folder. (Error above)
My guess is that there's a check in the code (isExisted) that's not made compatible for detecting the folders. Only files and this results in the error above.
Can someone help me to remove/create folders?
What am I doing wrong? Is there a special function I'm not aware of, that actually handles the deletion of folders?
Many thanks out front,
Kraeven
|
|
|
|
|
|
|
Hi,
The reason is the DeleteVFile() method can't delete a VDirectory.
And I just uploaded a new version, add DeleteVDirectory() method for delete a vdirectory and fixed some other bugs reported by the other people who used it.
Please try the latest version, enjoy.
|
|
|
|
|
|
|
Tony,
You're the man!!!
Works like a charm, thanks a lot and keep up the good work.
BTW i'm using this in a useraccount based windows form application, perfect for me.
Thanks a lot...
Kraeven
|
|
|
|
|
|
|
Hi,
You did an amazing job with this code. Congrats!
I was just wondering are you planning future updates?
Thanks again,
Kraeven
|
|
|
|
|
|
|
good library...
but seems there are little bug...
in method RenameVFile,
intPos after executing GetVFileEntry, contain position of entry within cluster,
(as example 0 or 64 or 128), code that write name use this value for seeking in global data stream...
it is wrong... we need also use position of cluster
something like
strmDataFile.Seek(parentClusterIndex * EachClusterLength + intPos + 1, SeekOrigin.Begin)
(original - strmDataFile.Seek(intPos + 1, SeekOrigin.Begin))
here modification of two methods in QVFS that fix problem
(without fix QVFS will write new name the the beginning off data file at offset 0(or 64 or 128) relative to offset of entry within cluster, and with this fix it will rename file normaly)
Protected Function GetVFileEntry(ByVal VFilePath As String, Optional ByVal IncludeDeletedFile As Boolean = False, Optional ByRef VFileEntryStartPosition As Int32 = 0, Optional ByRef VParentBlockPosition As Int32 = 0) As FileEntry
Dim memDir As MemoryStream = GetVDirEntry(Path.GetDirectoryName(VFilePath), VParentBlockPosition)
Dim binReader As New BinaryReader(memDir)
Dim byteProperty As Byte
Dim FileInfo As FileEntry
memDir.Seek(0, SeekOrigin.Begin)
Dim VFileName As String = Path.GetFileName(VFilePath)
Do While memDir.Position < memDir.Length
byteProperty = binReader.ReadByte()
If byteProperty = &H0 Then Exit Do
If (byteProperty <> Properties.EmptyEntry) And (IncludeDeletedFile Or ((byteProperty And Properties.IsDeleted) <> Properties.IsDeleted)) Then
FileInfo = New FileEntry
FileInfo.Property = byteProperty
FileInfo.Name = Encoding.UTF8.GetString(binReader.ReadBytes(FileNameMaxLengthInBytes)).Replace(Chr(0), "")
If FileInfo.Name = VFileName Then
FileInfo.CreateDateTime = GetDateTimeFromFourByte(binReader.ReadBytes(4))
FileInfo.ModifyDateTime = GetDateTimeFromFourByte(binReader.ReadBytes(4))
FileInfo.ClusterStartIndex = binReader.ReadInt32()
FileInfo.Size = binReader.ReadInt32
VFileEntryStartPosition = memDir.Position - EachDirEntryLength
Return FileInfo
Else
memDir.Seek(EachDirEntryLength - FileNameMaxLengthInBytes - 1, SeekOrigin.Current)
End If
Else
memDir.Seek(EachDirEntryLength - 1, SeekOrigin.Current)
End If
Loop
Return Nothing
End Function
Public Sub RenameVFile(ByVal VFilePath As String, ByVal NewVFileName As String)
Dim parentClusterIndex As Int32
Dim bufFileName() As Byte = Encoding.UTF8.GetBytes(NewVFileName)
If bufFileName.Length > 39 Then
Throw New Exception("The file name of " & NewVFileName & " is too long")
ElseIf bufFileName.Length < 39 Then
ReDim Preserve bufFileName(38)
End If
Dim intPos As Int32
GetVFileEntry(VFilePath, , intPos, parentClusterIndex)//!fix
strmDataFile.Seek(parentClusterIndex * EachClusterLength + intPos + 1, SeekOrigin.Begin)//!fix
binDataWriter.Write(bufFileName)
End Sub
in GetVFileEntry parameter VParentBlockPosition added
and in RenameVFile its used
ps
english is not my native =((
hope this comment is readable )
|
|
|
|
|
|
|
Thank you very much for such a detailed report. I will also check and correct the codes based on your comments. Thanks again.
|
|
|
|
|
|
|
1st off, this is a great article! 
here is my question:
after adding about 3000+ files of about 150MBs in total i noticed the size of virtual disk was around 450MBs. knowing nothing about how FAT32 handles file storing i am wondering if this is normal?
p.s: thanks for this great class!
|
|
|
|
|
|
|
Hi, I think it is normal, because the unit of storage if cluster which is 4KB. This means your files are splitted to many 4KB-length-block. If your files are all small files, it will waste some spaces on the virtual disk. For example, your file is 2KB, it still needs 4KB to store. When your file is 6KB, it needs 8KB spaces and so on.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Rant 
Admin