|
if i have a text file, and this file contains sentences as lines.
each line my be english or other unicode language, and sometimes both.
now, i want to detect the lines that include english (A-z) and remove it.
Is there any way to detect that there is (A-z) character like calculate the summation of sentence charaters.
considering that i've converted the file encoding to something like UTF-8 or ISO-8859-15...etc
and is a deference of the size of the charater when converting from one encoding to another?
Faris Madi
Nothing Comes Easy (N.C.E.)
|
|
|
|
|
militiaware wrote: Is there any way to detect that there is (A-z) character like calculate the summation of sentence charaters.
Just load in the data from the file and march through it looking for the characters in that range.
militiaware wrote: and is a deference of the size of the charater when converting from one encoding to another?
UTF-8 is an 8-bit encoding. That means that each character is stored in 8-bits. This doesn't allow for many characters and the standard pretty much just has English characters in that set. UTF-16 is a 16-bit encoding which can hold character sets from many different languages. The size of each character in UTF-16 is exactly twice the size of a UTF-8 character.
If you decide to become a software engineer, you are signing up to have a 1/2" piece of silicon tell you exactly how stupid you really are for 8 hours a day, 5 days a week
Zac
|
|
|
|
|
Zac Howland wrote: UTF-8 is an 8-bit encoding.
Its not. UTF-8 is a variable-length encoding, using one to four bytes per character.
Cheers,
Sebastian
--
Contra vim mortem non est medicamen in hortem.
|
|
|
|
|
Sebastian Schneider wrote: Its not. UTF-8 is a variable-length encoding, using one to four bytes per character.
It is an 8-bit encoding, though I probably could have been more specific by saying that it uses a maximum of 8-bits.
If you decide to become a software engineer, you are signing up to have a 1/2" piece of silicon tell you exactly how stupid you really are for 8 hours a day, 5 days a week
Zac
|
|
|
|
|
To my knowledge, UTF-8 is a variable length encoding. You are right in one aspect: As long as you only use the first 127 UC characters, every character will be stored in 8 bits. As soon as you choose any other unicode character, that character will use up to four byte (32 bits).
UTF-8 is NOT a "8-bit encoding able to represent a subset of unicode characters", but a "Unicode encoding designed to be able to represent the most commonly used unicode character with as few bytes as possible".
Cheers,
Sebastian
--
Contra vim mortem non est medicamen in hortem.
|
|
|
|
|
http://www.utf-8.com/
You can find the RFC for it there as well. UTF-8 only holds the lower subset of the UNICODE standard. UTF-16 holds the whole thing.
If you decide to become a software engineer, you are signing up to have a 1/2" piece of silicon tell you exactly how stupid you really are for 8 hours a day, 5 days a week
Zac
|
|
|
|
|
Zac Howland wrote: http://www.utf-8.com/
You can find the RFC for it there as well. UTF-8 only holds the lower subset of the UNICODE standard. UTF-16 holds the whole thing.
No it does not, as you'd know if you actually read the first few lines of your link.
www.utf-8.com wrote:
What is UTF-8?
UTF-8 stands for Unicode Transformation Format-8. It is an octet (8-bit) lossless encoding of Unicode characters.
UTF-8 encodes each Unicode character as a variable number of 1 to 4 octets, where the number of octets depends on the integer value assigned to the Unicode character. It is an efficient encoding of Unicode documents that use mostly US-ASCII characters because it represents each character in the range U+0000 through U+007F as a single octet. UTF-8 is the default encoding for XML.
Which is what Sebastian said. It's 8bits for us ascii, and expands to upto 32 for the remainder of unicode.
|
|
|
|
|
dan neely wrote: Which is what Sebastian said. It's 8bits for us ascii, and expands to upto 32 for the remainder of unicode.
I stand corrected. Sorry, I misread what it was saying the first time around.
If you decide to become a software engineer, you are signing up to have a 1/2" piece of silicon tell you exactly how stupid you really are for 8 hours a day, 5 days a week
Zac
|
|
|
|
|
Happens. I misread stuff all the time. My girlfriend, for example. She writes me a note "I love you. Clean the bathroom, if you find time." and I take it to mean what it says: if I find time, I'll clean the bathroom. Unfortunately, my chainsaw breaks and I have to mend it because I need it the next day... and then I spend the night on the couch 
Cheers,
Sebastian
--
Contra vim mortem non est medicamen in hortem.
|
|
|
|
|
|
Bassam Abdul-Baki wrote: Only 474 pages.
I've seen worse.
_________________________
Asu no koto o ieba, tenjo de nezumi ga warau.
Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
|
|
|
|
|
Yeah, me too. But still, I downloaded it and can't print it. So I have to wait for my SONY reader to come in.
"I know which side I want to win regardless of how many wrongs they have to commit to achieve it." - Stan Shannon
Web - Blog - RSS - Math - LinkedIn
|
|
|
|
|
Bassam Abdul-Baki wrote:
Only 474 pages.
No wonder the explanation in the topology book on my shelf was a hard read; it was less than 10% that size thus must have left out important details 
|
|
|
|
|
Hi,
Do you know any open-source/free implementation/library for circular layout algortihm similar to
this?
Any ideas/links to any information on the topic would be usefull.
Thanks.
Vitaliy Tsvayer
Tikle
|
|
|
|
|
Have you looked around on sourceforge?
"Any sort of work in VB6 is bound to provide several WTF moments." - Christian Graus
|
|
|
|
|
I'm trying to make a 3D scene using OpenGVS, but I need to make the scene looks like the pictures in a black-white TV. I tried to acquire the pixel color buffer before render,and render the pixels just using their luminance.However, it's very inefficient and can't meet the need.
Who can tell me how to solve the problem?Thanks for the help.
|
|
|
|
|
first off, infrared is not a simple greyscale luminance.
Second, you can't get a real infrared from color.
Simulating it means recalculating and restoring all your textures.... or using a shader.
I use a shader, it's very few lines and very efficient, but requires a decent graphics card and you don't have to rebuild the textures, you can use the same textures as true-color, just change the rendering algorithm through a shader.
As for my algorithm.... well.... that is the stuff that builds thesis papers.
However, I googled this[^] discussion so I don't have to talk about my algorithm. 
_________________________
Asu no koto o ieba, tenjo de nezumi ga warau.
Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
|
|
|
|
|
I will start by confessing to the truth.
Yes this is a school project.
Yes I am supposed to do this all by myself
And yes I want your help.
Please help me to express this problem mathematically
This project focuses on the design of a model that could be used to find the shortest path between three points. The shortest distance between any two points is a straight line. The three points are, selected randomly
The selected path is expected to fulfils the following requirements
The selected path should yield the most viable (and shortest?) distance
By pass all settlement with a population density greater than a specified value
By pass all evaluation above a specified height.
By pass all evaluation below sea level
By pass settlements that, although, do not fall in the urban area classification is of importance It is essentially a routing problem.
Any bright idea how I can go about this app?.
ihe
|
|
|
|
|
In general, what you're describing is the traveling salesman problem, which runs in eponential time for an exact answer. Butw with just 3 points a brute force solution checking all routes is possible.
|
|
|
|
|
Yes. So how can I solve it.I need an alogorithms i can use. I do not want to use brute force solution to check all the possible routes.( As the points are not fixed. )
ihe
|
|
|
|
|
If you want an exact solution, brute force is the only option. If you can find a non brute force solution that gives the optimal route in all cases, you've just proved P=NP. If an approximate solution is good enough, and your points obey the triangle inequality(1) then something like the A* algorythm can be used to fine one. Google is your friend.
(1) the distance from A to C is shorter than the distance from A to B plus the distance from B to C for all points A, B, C.
|
|
|
|
|
I'll give you a hint to help you get started ...
Google for the A* algorithm.
If you decide to become a software engineer, you are signing up to have a 1/2" piece of silicon tell you exactly how stupid you really are for 8 hours a day, 5 days a week
Zac
|
|
|
|
|
Thank you for the hint.
ihe
|
|
|
|
|
as with the others... you can use brute-force min-max scoring of paths to try all paths. This is an exponential of data therefore inefficient, but always finds the right solution. Every choice generates a new branch of every other choice it has, and so forth, and so on.
A* and Dynamic A* get into AI decisions of "pre-pruning" paths before evaluating them based on some judgement rules. This is not always the "best" solution, but generates a fast "good" solution.
check path navigation in AI game programming wisdom series of books.
_________________________
Asu no koto o ieba, tenjo de nezumi ga warau.
Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)
|
|
|
|
|
Thank you. Where can find this "AI game programming wisdom series of books" on the web .
ihe
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
