Click here to Skip to main content
15,887,350 members
Articles / Programming Languages / Visual Basic

QVFS - A FAT32 Based Virtual File System Powered by VB.NET 2005

Rate me:
Please Sign up or sign in to vote.
4.68/5 (33 votes)
8 Jun 2009CPOL5 min read 160.2K   1.7K   100   43
A virtual file system created using VB.NET, an easy way to store multiple folders and files in two real physical files.

Introduction

This is a .NET library implementing a virtual file system (VFS) named QVFS which is based on FAT32 concepts. It uses two files (one information file and one data file) to store multiple folders and files. This is just a virtual file system for storing files, and no other function, such as compress function, is included.

The source codes can be compiled under both .NET Framework 1.1 and .NET Framework 2.0, however, it is not tested under .NET Framework 3.0.

The goal of this library is to use two files to store everything for a program usage. For example, you may write your personal mail management system to store all the mails in VFS instead of creating one file for each mail and creating many many folders for categorizing mails. Your program may have many log files and it is split by different dates, thus one year it will have at least 365 files. Why not use a VFS, only two files, easy to backup, restore and manage.

Background

This virtual file system uses most concepts of FAT32 file system. You can think that this is a virtual compact FAT32 file system. Some concepts will be described below and knowing something about FAT32 will make it easy for you to go through this document.

Concepts & Technique Details About QVFS

File Structure of QVFS

QVFS uses two files to create the "virtual disk" for storing files.

One is an information file of which the extension is ".qfi" and the other is a data file of which the extension is ".qfd" and the file names for these two files will be the same by default.

The information file stores the FAT only and the data file stores the files' data.

The following figure shows the file structure of the information file and the data file.

Screenshot - FileStru.jpg

Main Concepts

NameDescription
Cluster The basic unit to store data, in this VFS it is 4KB for each cluster.
FAT File allocation table, which tells how the file is stored. In another words, which clusters are used to store a file.
Here a sample for the FAT is shown. One cell indicates one FAT entry and FAT entries with the same color indicate one file storage information. The FAT is using a chain table data structure and one FAT entry is mapping to one cluster. The number in the FAT entry tells where to find the next FAT entry (for FAT reader) and cluster (for file reader) of the file and the 'FFFFFFFF' indicates the file has ended in that cluster, the '0' indicates that entry has not been used yet, it is empty.

FATSample.jpg - Click to enlarge

  • Each FAT entry is 4 bytes(32 bit integer)
  • Each cluster is 4KB
  • In principle, the total amount of the bytes that can be stored in QVFS is 232*4K=17TB
Folder (Directory)A folder can contain many files and sub folders. In FAT32, the folder is just a special file, the only the difference is that the file contains the information data about the files and sub folders under itself and a file contains its own data.

How Does the Program Work

When adding a file in QVFS, it will first find the empty FAT entries which are mark as '00000000' and return a list of empty clusters which can be used and then read the source file block by block (the block size is equal to cluster size) from the very beginning and put each block into each cluster and write the cluster index in the FAT till the file's end.

When reading a file in QVFS, it will first read the first cluster index number of the root folder, cluster 0 in the above figure. Then find out the root folder data 'file' and search the root folder for the file you want. If the file is in one sub folder, it will search the sub folders one by one from the root, finally get the file's start cluster index and follow to FAT to read all the data of that file.

Using the Codes

To use the QVFS, you need to create the virtual disk first. The virtual disk is indicated by two files(.qfi & .qfd) which are mentioned above.

VB.NET
'Create QVFS disk file
QVFS.QVirtualFileSystemManager.CreateQVFSFile("C:\\test.qfi", true);

Add disk files into QVFS:

VB.NET
'Add the file d:\abc.txt to the root folder of QVFS
QVFS.QVirtualFileSystemManager vfm = new QVirtualFileSystemManager();
vfm.OpenQVFSFile("C:\\test.qfi");
vfm.AddNewVFile("D:\\abc.txt","\\" true);
vfm.CloseQVFSFile();

Delete a file from QVFS:

VB.NET
'Delete the file '\abc.txt' from QVFS
QVFS.QVirtualFileSystemManager vfm=new QVirtualFileSystemManager();
vfm.OpenQVFSFile("C:\\test.qfi");
vfm.DeleteVFile("\\abc.txt", true);
vfm.CloseQVFSFile();

Save file in QVFS to disk:

VB.NET
'Save the file "\abc.txt" to "C:\"
QVFS.QVirtualFileSystemManager vfm=new QVirtualFileSystemManager();
vfm.OpenQVFSFile("C:\\test.qfi");
vfm.SaveVFileToDisk("\\abc.txt", "C:\\abc.txt");    
vfm.CloseQVFSFile();

About the DEMO Program

The DEMO program shows the above functions using real codes. It may have some bugs and it is NOT tested carefully. It is only for DEMO. In the other hand, the QVFS library is tested thoroughly, no big function bug is found in the current version.

It shows the folder structure on the left section and shows the detail file's information in QVFS.

Screenshot - DemoPro.jpg

Points of Interest

There are many file systems till now. Here I have just shown a virtual file system based on FAT32. FAT32 is a more "simple" file system compared to NTFS and Linux ext., and it is good enough for the VFS only for storing files except for security requirement, compress requirement and so on. I would like to discuss the latest VFS technology with you. Please give your ideas and suggestions.

History

  • 2004.08.26, Version 0.9.0, VFileSystem version 1
  • 2004.09.24, Version 0.9.5, VFileSystem version 2
  • 2007.10.10, Version 1.0.0, bug fixes & converted into .NET Framework 2.0 library, formal release

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Architect
China China
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionCan we access the file path? Pin
Harshw27-Nov-13 3:42
Harshw27-Nov-13 3:42 
AnswerRe: Can we access the file path? Pin
TonyTonyQ22-Apr-14 22:57
professionalTonyTonyQ22-Apr-14 22:57 
QuestionThanks for this wonderful article. Implemented this QVFS with some major enhancements Pin
Manoj Natarajan28-Oct-13 0:37
professionalManoj Natarajan28-Oct-13 0:37 
PraiseRe: Thanks for this wonderful article. Implemented this QVFS with some major enhancements Pin
Binief Ta9-Nov-16 19:42
Binief Ta9-Nov-16 19:42 
QuestionWhat are the limitations? Pin
Celso Jr20-Feb-13 13:37
Celso Jr20-Feb-13 13:37 
GeneralMy vote of 5 Pin
Tushar Arora13-May-11 23:29
Tushar Arora13-May-11 23:29 
GeneralMy vote of 3 Pin
Dave Kreskowiak21-Jan-11 9:21
mveDave Kreskowiak21-Jan-11 9:21 
JokeMy vote of 5 (contains joke) Pin
MaximilianReisch13-Sep-10 5:00
MaximilianReisch13-Sep-10 5:00 
GeneralMy vote of 5 Pin
Kraeven6-Jul-10 0:33
Kraeven6-Jul-10 0:33 
QuestionExpose Files To Another Proccess Pin
omarooo28-Aug-09 12:16
omarooo28-Aug-09 12:16 
QuestionAccess QVFS file from multiple computers Pin
Kraeven15-Jul-09 10:16
Kraeven15-Jul-09 10:16 
AnswerRe: Access QVFS file from multiple computers Pin
TonyTonyQ15-Jul-09 15:16
professionalTonyTonyQ15-Jul-09 15:16 
GeneralRe: Access QVFS file from multiple computers Pin
Kraeven15-Jul-09 21:06
Kraeven15-Jul-09 21:06 
Generalbug fix... Pin
silverdox13-Jun-09 6:34
silverdox13-Jun-09 6:34 
QuestionAre two files needed? Pin
supercat99-Jun-09 6:32
supercat99-Jun-09 6:32 
AnswerRe: Are two files needed? Pin
TonyTonyQ9-Jun-09 16:25
professionalTonyTonyQ9-Jun-09 16:25 
QuestionISO? Pin
Huisheng Chen8-Jun-09 23:21
Huisheng Chen8-Jun-09 23:21 
AnswerRe: ISO? Pin
supercat99-Jun-09 6:21
supercat99-Jun-09 6:21 
QuestionCopyright infringement? Pin
zlezj8-Jun-09 21:17
zlezj8-Jun-09 21:17 
AnswerRe: Copyright infringement? Pin
Dewey8-Jun-09 21:31
Dewey8-Jun-09 21:31 
GeneralDeleting Files and Folders from QVFS Pin
Kraeven8-Jun-09 5:21
Kraeven8-Jun-09 5:21 
GeneralRe: Deleting Files and Folders from QVFS Pin
TonyTonyQ8-Jun-09 18:33
professionalTonyTonyQ8-Jun-09 18:33 
GeneralRe: Deleting Files and Folders from QVFS Pin
Kraeven9-Jun-09 22:27
Kraeven9-Jun-09 22:27 
QuestionGreat Job! Pin
Kraeven5-Jun-09 4:15
Kraeven5-Jun-09 4:15 
Generalhm... Pin
Member 316538423-Feb-09 7:04
Member 316538423-Feb-09 7:04 
good library...

but seems there are little bug...

in method RenameVFile,
intPos after executing GetVFileEntry, contain position of entry within cluster,
(as example 0 or 64 or 128), code that write name use this value for seeking in global data stream...
it is wrong... we need also use position of cluster

something like
<br />
 strmDataFile.Seek(parentClusterIndex * EachClusterLength + intPos + 1, SeekOrigin.Begin)

(original - strmDataFile.Seek(intPos + 1, SeekOrigin.Begin))


here modification of two methods in QVFS that fix problem
(without fix QVFS will write new name the the beginning off data file at offset 0(or 64 or 128) relative to offset of entry within cluster, and with this fix it will rename file normaly)



Protected Function GetVFileEntry(ByVal VFilePath As String, Optional ByVal IncludeDeletedFile As Boolean = False, Optional ByRef VFileEntryStartPosition As Int32 = 0, Optional ByRef VParentBlockPosition As Int32 = 0) As FileEntry
    Dim memDir As MemoryStream = GetVDirEntry(Path.GetDirectoryName(VFilePath), VParentBlockPosition)
    Dim binReader As New BinaryReader(memDir)
    Dim byteProperty As Byte
    Dim FileInfo As FileEntry
    memDir.Seek(0, SeekOrigin.Begin)

    Dim VFileName As String = Path.GetFileName(VFilePath)

    Do While memDir.Position < memDir.Length
        byteProperty = binReader.ReadByte()

        If byteProperty = &H0 Then Exit Do 'Reach the data end or invaild data, exit

        If (byteProperty <> Properties.EmptyEntry) And (IncludeDeletedFile Or ((byteProperty And Properties.IsDeleted) <> Properties.IsDeleted)) Then
            FileInfo = New FileEntry
            FileInfo.Property = byteProperty
            FileInfo.Name = Encoding.UTF8.GetString(binReader.ReadBytes(FileNameMaxLengthInBytes)).Replace(Chr(0), "")

            If FileInfo.Name = VFileName Then
                FileInfo.CreateDateTime = GetDateTimeFromFourByte(binReader.ReadBytes(4))
                FileInfo.ModifyDateTime = GetDateTimeFromFourByte(binReader.ReadBytes(4))
                FileInfo.ClusterStartIndex = binReader.ReadInt32()
                FileInfo.Size = binReader.ReadInt32

                VFileEntryStartPosition = memDir.Position - EachDirEntryLength  'Already read a directory entry

                Return FileInfo
            Else

                memDir.Seek(EachDirEntryLength - FileNameMaxLengthInBytes - 1, SeekOrigin.Current)
            End If
        Else


            memDir.Seek(EachDirEntryLength - 1, SeekOrigin.Current)
        End If
    Loop

    Return Nothing
End Function

Public Sub RenameVFile(ByVal VFilePath As String, ByVal NewVFileName As String)


    Dim parentClusterIndex As Int32


    Dim bufFileName() As Byte = Encoding.UTF8.GetBytes(NewVFileName)
    If bufFileName.Length > 39 Then
        Throw New Exception("The file name of " & NewVFileName & " is too long")
    ElseIf bufFileName.Length < 39 Then
        ReDim Preserve bufFileName(38)
    End If


    Dim intPos As Int32
    GetVFileEntry(VFilePath, , intPos, parentClusterIndex)//!fix

    strmDataFile.Seek(parentClusterIndex * EachClusterLength + intPos + 1, SeekOrigin.Begin)//!fix
    binDataWriter.Write(bufFileName)
End Sub


in GetVFileEntry parameter VParentBlockPosition added
and in RenameVFile its used

ps
english is not my native =((
hope this comment is readable )

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.