How to parse / split English Sentence C#

Question

1.00/5 (1 vote)

See more:

VS2010

Please, PROGRAMMER help me... Im so so confused how to create code for parsing English sentence.
how to recognize when one sentence ends and another sentence begins when it has embedded punctuation marks..
For example :

Input :

First sentence. Second sentence! Third sentence? .Yes.
.....Nice to meet you.... I am okay

Output :
arr[0] = First sentence
arr[1] = Second sentence
arr[3] = Third sentence
arr[4] = Yes
arr[5] = Nice to meet you
arr[6] = I am okay

Explanation :
I wanna split { ?, ! , multiple dot or string doesn't have meaning }

This my code

C#

string[] word = new string[100];
string inputRtb = rtbInput.Text;

string plot1 = "";
string plot2 = "";

string[] splitString = inputRtb.Split(new char[] {' ', '\t', '\n'});

int j = 0;
int pos = 0;
for (int i = 0 ; i < splitString.Length; i++)
{
    if (splitString[i].Trim() != "" && splitString[i].Trim() != ".")
    {
        if (splitString[i].Trim()[splitString[i].Length - 1] == '.')
        {
            plot2 = substr(splitString[i].Trim(), 0, splitString[i].Length - 1);
            if (plot1 == "")
                plot1 += plot2;
            else
                plot1 += " " + plot2;
            pos++;
        }

        else if (plot1 == "")
            plot1 += splitString[i].Trim();
        else
            plot1 += " " + splitString[i].Trim();
    }


    if (plot1 != "" && splitString[i].Trim() == ".")
    {
        word[j++] = plot1;
        plot1 = "";
    }
    else if (pos > 0)
    {
        word[j++] = plot1;
        plot1 = "";
        pos--;
    }
    else if (plot1 != "" && i == splitString.Length-1)
    {
        word[j++] = plot1;
        plot1 = "";
    }
}

Posted 8-Mar-13 2:18am

Berry Harahap

Updated 8-Mar-13 4:15am

v2

Add a Solution

Comments

Menon Santosh 8-Mar-13 8:23am

Elaborate your problem

Member 10531704 18-Jan-14 9:11am

what is rtbinput and inputrtb

Richard MacCutchan 8-Mar-13 8:27am

Don't try and do it with basic split methods. You need to write your own parser that recognises specific sentence terminators.

Berry Harahap 8-Mar-13 8:32am

@Santosh : Hallo.
My problem if INPUT : ...Asia
i have difficult to split it, i want the output is OUTPUT : Asia

or INPUT : Thanks...
OUTPUT : Thanks

INPUT : ..Thanks..
OUTPUT : Thanks

INPUT : Thanks
OUTPUT : Thanks

INPUT : Thanks
OUTPUT : Thanks

INPUT : !!!Thanks
INPUT : Thanks

INPUT : Thanks?????
OUTPUT : Thanks

I cant get the code for make it. Any solution?
Im just student.

joshrduncan2012 8-Mar-13 9:14am

I would look at writing your own version of the parser (like what Richard said) using the String.Split() method (you may get some blanks in the array that is created from that. So you would have to throw away the blank elements.

[no name] 8-Mar-13 9:15am

"Being a student" does not mean that you do not have to do some basic research. Did you look up the documentation for Split? Did you try adding SplitOptions.RemoveEmptyEntries?

Berry Harahap 8-Mar-13 9:37am

@Mr. joshrduncan2012
@Mr. ThePhantomUpvoter

If I use SplitOptions.RemoveEmptyEntries, like this example
<pre>
string[] separators = {",", ".", "!", "?", ";", ":", " "};
string value = "The handsome, energetic, young dog was playing with his smaller, more lethargic litter mate.";
string[] words = value.Split(separators, StringSplitOptions.RemoveEmptyEntries);
foreach (var word in words)
Console.WriteLine(word);
</pre>

How to recognize one sentence to another sentence? I need to divide each sentence.
Like
input = "I go to school. She go to market"
output =
arr[0] = I go to school
arr[1] = She go to market

how to recognize when one sentence ends and another sentence begins, index i=0 to i=1 ????

[no name] 8-Mar-13 9:50am

"How to recognize array index i=0 to i=1 ????" what on earth does that even mean? [0] is one sentence and [1] is another sentence. If you are talking about how to recognize when one sentence ends and another sentence begins when it has embedded punctuation marks, then you probably are not going to be able to use Split. You are talking about creating your very own custom sentence parser and that is way beyond the scope of this forum.

Berry Harahap 8-Mar-13 10:09am

@Mr. ThePhantomUpvoter : Good point Mr.
I mean that
"how to recognize when one sentence ends and another sentence begins when it has embedded punctuation marks". Hehe

BUT..
Don't exaggerate this problem scope, Mr @Mr. ThePhantomUpvoter, like this word "beyond".
I'm just amateur student.

[no name] 8-Mar-13 10:18am

like this word "beyond", I am sure that meant something by this but what it is I do not know. This forum is to help people that have specific questions about specific coding issues. "Teach me how to write a natural language parser" does not fit into the guidelines for this forum.
It does not matter if you are a student or not. Being a student does not relieve you of the responsibility to do some research and ask a clear, specific question. Or follow the guidelines set forth for asking questions in this forum.

Richard MacCutchan 8-Mar-13 12:31pm

Well, like I suggested you should not use string.split(). You need to write your own methods to scan the text looking for the characters that terminate a sentence and split them at those points.

Andreas Gieriet 9-Mar-13 1:00am

I think for his task Split is sufficient. It cannot possibly be to detect 100% of English sentences correctly. See also my intro in solution#3. The requirements are rather simple - it seems to be a homework assignment to get the problem converted into some pseudo code and then implemented in C#.
Cheers
Andi

3 solutions

Solution 1

Here is a rough algorithm, but it should get the job done.

public static string[] ParseSentences(string sentence)
{
    char[] terminators = { '.', '?', '!' };

    List<string> sentences = new List<string>(
        sentence.Split(terminators, StringSplitOptions.RemoveEmptyEntries));
    for (int i = sentences.Count - 1; i >= 0; i--)
        if (sentences[i].Trim().Length == 0)
            sentences.RemoveAt(i);
        else
            sentences[i] = sentences[i].Trim();

    return sentences.ToArray();
}

Posted 8-Mar-13 3:36am

MichaelBergsma

Updated 8-Mar-13 4:44am

v3

Comments

Berry Harahap 8-Mar-13 19:03pm

Dear @Mr MichaelBergsma, I'm interested to try this code. But,
List<string> sentences = new List<string>(sentences.Split(terminators, StringSplitOptions.RemoveEmptyEntries)) , There is error in sentences.Split , List<string> sentences = new List<string> doesn't contain a definition 'split' or what reference i should add Mr?

Andreas Gieriet 9-Mar-13 0:30am

Read carefully: there are two variables with almost the same name: sentence and sentences.
Cheers
Andi

Sergey Alexandrovich Kryukov 13-Mar-13 1:24am

That's pretty much it... My 5.
—SA

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Andreas Gieriet · Accepted Answer · 2013-03-08T18:53:00

Solution 3

This task is not so trivial as it sounds: a punctuation character is intrinsically context dependent.
E.g. the dots in at the beginning of this line do not make a sentence each. ;-)
Or "(see item 1. above)" does not terminate after the dot.
There is many more cases also with other punctuation characters.

But it looks like this is not the topic of the question.
So, if you want to simply get the chunks of text between some delimiters, treating repetitions of delimiters as one delimiter, stripping off leading and trainling spaces from the found chunks of text, then the following would do:

C#

string fullText = "..."; // input text
char[] delim = ".?!;".ToCharArray(); // add more single character delimiter as needed
var sentences = fullText.Split(delim, StringSplitOptions.RemoveEmptyEntries).Select(s=>s.Trim());
foreach(var s in sentences) Console.WriteLine(s);

Cheers
Andi

Posted 8-Mar-13 18:53pm

Andreas Gieriet

Comments

Berry Harahap 9-Mar-13 3:30am

==Dear Andreas Gieriet==
Very simple approach and good job, Sir.
Would you mind giving opinion, what do you think about this code? Which one do you choose? Your code or this code? Or any combination? Please give me recommendation.
=================================================
ArrayList arrSentence = new ArrayList();
string temp = inputRtb;
temp = temp.Replace(Environment.NewLine, " ");
char[] arrSeparator = { '.', '?', '!' };

string[] splitInput = temp.Split(arrSeparator, StringSplitOptions.RemoveEmptyEntries);
for (int i = 0; i < splitInput.Length; i++) {
int pos = temp.IndexOf(splitInput[i].ToString());
char[] oChars = temp.Trim().ToCharArray();
char charCollection = oChars[pos + splitInput[i].Length];
arrSentence.Add(splitInput[i].ToString().Trim() + charCollection.ToString());
}
==================================================

Please, sir give me recommendation.
Cheers
Andi :) :)

Andreas Gieriet 9-Mar-13 15:00pm

I never use non-generic containers unless I'm forced to. So ArrayList for me is a no-go.
Naming a variable temp is a no-go too. If you can not think of a reasonable name, then you have a design problem.
You use ToString() at several locations where it is non-sense. Know what types you are using and for what reason!
Calling a variable of type char a charCollection is misleading. This is confusing.

Generally: you have inconsistent naming scheme (why do you call a variable oChar and others without any prefix?) and you have weird type selection or conversions (e.g. see ToString()).

My advise: learn the C# types and choose a naming scheme for variables that carry the meaning of the stored values.

Don't take this critisism personal, take it as advise for improvement.

Regards
Andi

Berry Harahap 9-Mar-13 19:59pm

== Mr Andreas Gieriet ==
Your suggestion is my pleasure. Thanks a lot. :) :) :)

Sergey Alexandrovich Kryukov 13-Mar-13 1:24am

Right, a 5.
—SA

Andreas Gieriet 13-Mar-13 3:08am

Thanks for your 5!
Andi

Shameel · Accepted Answer · 2013-03-08T03:44:00

Solution 2

This one line will do the trick, you just need to add all the line terminators to the array.

C#

string[] splitString = inputRtb.Split(new char[] { '!', '?', '.', '\t', '\n' }, StringSplitOptions.RemoveEmptyEntries);

Posted 8-Mar-13 3:44am

Shameel

Comments

Sergey Alexandrovich Kryukov 13-Mar-13 1:25am

That's pretty much it, too, a 5.
—SA

[no name] 13-Mar-13 2:17am

Thanks Sergey.