Click here to Skip to main content
15,887,027 members
Home / Discussions / C#
   

C#

 
QuestionUse GPU from C# Pin
AmbiguousName12-May-14 6:52
AmbiguousName12-May-14 6:52 
AnswerRe: Use GPU from C# Pin
Eddy Vluggen12-May-14 7:43
professionalEddy Vluggen12-May-14 7:43 
GeneralRe: Use GPU from C# Pin
AmbiguousName12-May-14 7:49
AmbiguousName12-May-14 7:49 
GeneralRe: Use GPU from C# Pin
Eddy Vluggen12-May-14 8:38
professionalEddy Vluggen12-May-14 8:38 
GeneralRe: Use GPU from C# Pin
AmbiguousName12-May-14 19:18
AmbiguousName12-May-14 19:18 
GeneralRe: Use GPU from C# Pin
Eddy Vluggen13-May-14 7:04
professionalEddy Vluggen13-May-14 7:04 
QuestionHow can i make this program Parrallel ? Pin
Member 1037568411-May-14 11:51
Member 1037568411-May-14 11:51 
AnswerRe: How can i make this program Parrallel ? Pin
Phil Martin11-May-14 12:57
professionalPhil Martin11-May-14 12:57 
There are a number of ways you can approach this.

The simplest is to execute the body of your outer loop for multiple files at the same time. You can use the Task Parallel Library that is part of .Net for this. The 'Task' classes let you run code in parallel and wait for results at the end.

If you choose to do that, the problem is now "How do I merge results of concurrent indexes into one big index"

Some possible solutions are:

  • Use a lock to protect access to the index
  • Create separate indexes and merge them at the end.


I would go with the second because it uses less shared state, and has less chance of me introducing a difficult to find bug

C#
var indexFilesTasks = (from fname in fnames
                       select Task.Run(() => IndexSingleFile(fname, stopWords, () => worker.CancellationPending))).ToArray();
				
Task.WaitAll(indexFilesTasks);
	
index = MergeIndexes(from t in indexFilesTasks
                     select t.Result);


The IndexSingleFile() method contains all your code in your foreach (var fname in fnames) loop.

The MergeIndexes() method takes multiple Dictionary<string,indexentry>() structures, and merges them together. Below is the code I used for these.

Note this isn't the "best" way, but just one way of approaching the problem.


C#
Dictionary<string,IndexEntry> IndexSingleFile(string filename, HashSet<string> stopWords, Func<bool> isCancelling) {
    var index = new Dictionary<string,IndexEntry>();
	
    // Open file, read in file contents,
    // process each line one at a time
    // split into words, add word to index if not already present
    // increment occurrence count for this file and record line number.
    string[] lines = File.ReadAllLines(filename);
 
    for (int i = 0; i < lines.Length; i++)
    {
        string[] wordsInLineRaw = Regex.Split(lines[i], @"\W+");
 
        string[] wordsInLine = new string[wordsInLineRaw.Length];
 
        for (int p = 0; p < wordsInLineRaw.Length; p++)
        {
            wordsInLine[p] = wordsInLineRaw[p].ToLower();
        }
 
        // Process each word found in line
        for (int j = 0; j < wordsInLine.Length; j++)
        {
            string w = wordsInLine[j];
            // If word in stop list break
            if (stopWords.Contains(w)) continue;
 
            //if word not in index add it
            if (!index.ContainsKey(w))
            {
 
                if (isCancelling())
                {
 
                }
                    //e.Cancel();

                 index.Add(w, new IndexEntry());
 			}
 
            // Update info about word
            index[w].UpdateInfo(filename, i);
		}
	}	
	
	return index;
}


C#
Dictionary<string,IndexEntry> MergeIndexes(IEnumerable<Dictionary<string,IndexEntry>> indexes) {

	var result = new Dictionary<string,IndexEntry>();
	
	foreach (var index in indexes) {
		foreach (var srcPair in index) {
			if (!result.ContainsKey(srcPair.Key)) {
				result[srcPair.Key] = new IndexEntry();
			}
			
			result[srcPair.Key].MergeFrom(srcPair.Value);
		}
	}

	return result;
}


modified 12-May-14 4:49am.

GeneralRe: How can i make this program Parrallel ? Pin
Bernhard Hiller11-May-14 22:06
Bernhard Hiller11-May-14 22:06 
GeneralRe: How can i make this program Parrallel ? Pin
Rob Philpott11-May-14 22:24
Rob Philpott11-May-14 22:24 
GeneralRe: How can i make this program Parrallel ? Pin
Phil Martin11-May-14 22:39
professionalPhil Martin11-May-14 22:39 
GeneralRe: How can i make this program Parrallel ? Pin
Phil Martin11-May-14 22:47
professionalPhil Martin11-May-14 22:47 
GeneralRe: How can i make this program Parrallel ? Pin
SledgeHammer0112-May-14 9:33
SledgeHammer0112-May-14 9:33 
GeneralRe: How can i make this program Parrallel ? Pin
Phil Martin12-May-14 11:15
professionalPhil Martin12-May-14 11:15 
QuestionStored procedures - Northwind database Pin
Vexy11-May-14 9:58
Vexy11-May-14 9:58 
AnswerRe: Stored procedures - Northwind database Pin
Rob Philpott12-May-14 1:33
Rob Philpott12-May-14 1:33 
AnswerRe: Stored procedures - Northwind database Pin
sankarsan parida12-May-14 19:29
professionalsankarsan parida12-May-14 19:29 
QuestionPlot line graph from database Pin
Syafiqah Zahirah11-May-14 4:44
Syafiqah Zahirah11-May-14 4:44 
AnswerRe: Plot line graph from database Pin
V.11-May-14 20:19
professionalV.11-May-14 20:19 
GeneralRe: Plot line graph from database Pin
Syafiqah Zahirah11-May-14 20:52
Syafiqah Zahirah11-May-14 20:52 
GeneralRe: Plot line graph from database Pin
V.11-May-14 21:01
professionalV.11-May-14 21:01 
GeneralRe: Plot line graph from database Pin
Syafiqah Zahirah15-May-14 6:53
Syafiqah Zahirah15-May-14 6:53 
AnswerRe: Plot line graph from database Pin
Bernhard Hiller11-May-14 22:14
Bernhard Hiller11-May-14 22:14 
GeneralRe: Plot line graph from database Pin
Syafiqah Zahirah15-May-14 7:13
Syafiqah Zahirah15-May-14 7:13 
GeneralRe: Plot line graph from database Pin
Syafiqah Zahirah20-May-14 9:24
Syafiqah Zahirah20-May-14 9:24 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.