Your best solution is to put a Try/Ctach block around the File.Delete() call. This will let you catch the error on the one file you're trying to delete and skip it, continuing with deleting the remaining files in the enumeration. The same is true for subdirectories.
foreach (string filePath in Directory.GetFiles(tempPath, "*.*", SearchOption.AllDirectories))
{
try
{
FileInfo currentFile = new FileInfo(filePath);
currentFile.Delete();
}
catch (Exception ex)
{
Debug.WriteLine("Error on file: {0}\r\n {1}", filePath, ex.Message);
}
}