Click here to Skip to main content

Design and Architecture

 
AnswerRe: How do you design this? : with a better example Pinmemberjschell31-Dec-12 10:10 
AnswerRe: How do you design this? : with a better example PinmemberEddy Vluggen31-Dec-12 18:04 
AnswerRe: How do you design this? : with a better example PinmemberKeld Ølykke16-Jan-13 12:23 
QuestionHow to find the similarity between users in Twitter ? How to design a good and efficient idea? Pinmemberldaneil27-Dec-12 8:26 
I am working on a project about data mining. my company has given me 6 million dummy customer info of twitter. I was assigned to find out the similarity between any two users. can anyone could give me some ideas how to deal with the large community data? Thanks in advance
 
Problem : I use the tweets & hashtag info(hashtags are those words highlighted by user) as the two criteria to measure the similarity between two different users. Since the large number of users, and especially there may be millions of hastags & tweets of each user. Can anyone tell me a good way to fast calculate the similarity between two users? I have tried to use FT-IDF to calculate the similarity between two different users, but it seems infeasible. can anyone have a very super algorithm or good ideas which could make me fast find all the similarities between users?
 
For example:
user A's hashtag = {cat, bull, cow, chicken, duck}
user B's hashtag ={cat, chicken, cloth}
user C's hashtag = {lenovo, Hp, Sony}
 
clearly, C has no relation with A, so it is not necessary to calculate the similarity to waste time, we may filter out all those unrelated user first before calculate the similarity. in fact, more than 90% of the total users are unrelated with a particular user. How to use hashtag as criteria to fast find those potential similar user group of A? is this a good idea? or we just directly calculate the relative similarity between A and all other users? what algorithm would be the fastest and customized algorithm for the problem?

AnswerRe: How to find the similarity between users in Twitter ? How to design a good and efficient idea? PinprotectorPete O'Hanlon27-Dec-12 8:39 
GeneralRe: How to find the similarity between users in Twitter ? How to design a good and efficient idea? Pinmemberldaneil27-Dec-12 8:50 
AnswerRe: How to find the similarity between users in Twitter ? How to design a good and efficient idea? Pinmemberjschell27-Dec-12 10:21 
GeneralRe: How to find the similarity between users in Twitter ? How to design a good and efficient idea? Pinmemberldaneil28-Dec-12 9:31 
AnswerRe: How to find the similarity between users in Twitter ? How to design a good and efficient idea? [modified] PinmemberApril Fans27-Dec-12 16:43 
GeneralRe: How to find the similarity between users in Twitter ? How to design a good and efficient idea? Pinmemberldaneil28-Dec-12 9:36 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.


Advertise | Privacy | Mobile
Web02 | 2.8.150327.1 | Last Updated 28 Mar 2015
Copyright © CodeProject, 1999-2015
All Rights Reserved. Terms of Service
Layout: fixed | fluid