how to replace vocabulary of document by index it in vocabulary in both document and vocabulary

Question

1.00/5 (1 vote)

See more:

I extract Vocabulary from document with below code.

C#

//document is a string variable 
List<string> Vocabulary= Vocabulary.Union((Regex.Replace(document, "\\p{P}", " ")).
Split(' ')).ToList();

how replace every word of Vocabulay with index it in Vocabulary in both Vocabulay and document Simultaneously with extraction operation in top code
for example
document=>"the book book is are is" change value to "0 1 1 2 3 2"
or this values store in a List<int> variable
Vocabulary[0]=>"the" change value to 0
Vocabulary[1]=>"book" change value to 1
Vocabulary[2]=>"is" change value to 2
Vocabulary[3]=>"are" change value to 3

Posted 2-Jul-15 5:55am

kave2011

Updated 2-Jul-15 8:09am

v2

Add a Solution

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Matt T Heffron · Answer 1 · 2015-07-02T06:59:00

Interesting problem...

This looks like a word-based document "compression" process.

But I'm a little confused as to why you would replace all of the info in Vocabulary from the word strings to the corresponding number. That is the only information of the mapping between the words and the numbers. Without it, you will have no way to reverse the mapping and recreate the original string. So, essentially, the output can be almost arbitrary since there's no way to reconstruct anything useful!

If it is really required to replace the vocabulary values then:

C#

Vocabulary = Enumerable.Range(0, Vocabulary.Count).Select(n => n.ToString()).ToList();

For doing the replacements in document, I'd probably use a Dictionary<string, int> to hold the word-to-number mapping instead of needing to scan the Vocabulary List at every word.

Another option would be to iterate through the Vocabulary list, and apply a Regex.Replace across the whole document for each word with the corresponding number. This will almost certainly misbehave if document can contain numbers that are the same as any of the word replacement values. Also, this is O(N²) on the length of the document.