Find files that have the same contents

Introduction 

How to find the files that have the same contents.

Background 

I found  there are many files that have the same contents but have different file name in my computer, i want to clear them and just maint one copy.  Therefore i wrote this small tool to find the same files.

Using the code 

 I  used the dictionary to store the file's mark(calculate by MD5 alg) and the corresponding files:

Dictionary<string, List<string>> dtFiles = new Dictionary<string, List<string>>()

and then find each file under the folder and get it's mark. In order to use the Dictionary.ContainsKey method, i convert the byte[] to base64 string.

    string[] strFiles = Directory.GetFiles(strFolder_);
    foreach (string strFullFile in strFiles)
    {
        if (_bToStop)
            return;

        try
        {
            byte[] byMd5 = Xugd.Hash.XMd5.CalcFile(strFullFile);
            string strMd5 = XConvert.BytesToString(byMd5);
            if (dtFiles_.ContainsKey(strMd5))
            {
                List<string> lstFiles = dtFiles_[strMd5];
                lstFiles.Add(strFullFile);
                dtFiles_[strMd5] = lstFiles;
            }
            else
            {
                List<string> lstFiles = new List<string>(2);
                lstFiles.Add(strFullFile);
                dtFiles_.Add(strMd5, lstFiles);
            }
        }
        catch { }
    }

    // Find in sub dir
    string[] strDirs = Directory.GetDirectories(strFolder_);
    foreach (string strSub in strDirs)
        FindSameFile(strSub, delStart_, dtFiles_);
}

 after process all files, we can check each item in the dictionary and find out the files that has the same contents.

foreach (KeyValuePair<string, List<string>> kvFile in dtFiles)
{
    if (kvFile.Value.Count > 1)
    {
        // found the files
    }
}

 Points of Interest  

This small article is written for those developers who are want to found the same contents file.

History

31 August 2012: First version