|
Hi Ivan. Hi all.
First of all, I had a good time reading big words
about small problems, especially in the posts!
(4th moment and so on)
The temptation of writing what is good and what is
bad in the article is there, but I wouldn't like
to play the same game (big words for small problems).
Anyway, the assumption than votes and weights are
independent is really too strong!
(perhaps they are *uncorrelated*)
Is you say so, you say "all people vote the same way"
(same distribution).
But weights are used because experienced people
(should) vote better...
P.S. I won't tell you my vote, but don't worry,
it has a really small weight!
Luca Piccareta
|
|
|
|
|
You are right
>>Anyway, the assumption than votes and weights are
>>independent is really too strong!
>>(perhaps they are *uncorrelated*)
is a weaker assumption and in this case all conclusions stay the same.
But please note that you have also payed attention to this article and besides, you also wrote some big words about small problems
BTW I really think that people vote the same way as I presume that they vote ideally i.e. depending on the quality of an article.
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
For Movie rating there is no gurus and rookies, but rating is still useful to estimate potential interest of movie based on peoples affinity : if someone like same movies as me, I should see also the movies he likes that I have not seen yet, and vice-versa. Perhaps this affinity theory is also useful for CodeProject’s Article rating ?
|
|
|
|
|
Do you mean that if someone likes the same kind of articles as you do then he will not only read the same article with you but also will vote the same way?
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
Statistically, yes ! If I think I am a guru, I will naturally read other guru's article in priority, and vice-versa.
|
|
|
|
|
I think that this influences ony the distribution of voters' weigts for each certain article.
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
Some speculation on an implementation:
Each user’s vote would have to be stored; as well as their voting history (normalized, probably).
Then, when you are presented with a list of articles, those who vote ‘similar’ to you will have more weight with how your listing is sorted.
This system would depend on how you vote, so those who vote very little should fall back on a default algorthm, or the algorithm should introduce the ‘similar votes’ term as a function of how many votes you’ve cast. As you vote for articles, the system gets more certain of your similitude to other members of the site. Perhaps an n–space could be used to classify users.
This would all be very expensive in terms of CPU time, but I see several optimizations. One would be to run the algorithm only once every few days, perhaps off-server. The vote hisory could be discarded after each one of these runs.
|
|
|
|
|
I entirely agree with You for all points !
n-space could be used to classify users : Yes ! and classifying this n-space would produce a psychological profile of the users ! (for example, for movie rating, an n-space can correspond to movie category : fantasy, science-fiction, ... and for article voting, this can correspond to level of expertise : didactic article or expert article)
CPU time : I think this is the reason why IMDB as not yet a affinity classifier for voting member : it is very expensive of time consuming, but that is worth the blow for predicting movie attractiveness.
|
|
|
|
|
See also : Collaborative Filtering : www.vsnetfr.com/lien.aspx?ID=3923
|
|
|
|
|
No math here :
guru : people with high membership level
rookies : people with low membership level
(I) "Wi and Vi independent" imply : guru and rookie vote in similar way
(D) "Wi and Vi dependent" imply : guru and rookie don't vote in similar way
(given you level of math, and to keep that post short, I don't think I need to proove or explain better the aboves implication, but I can if you ask so)
if (I) is true (that is your hypothesis when you write E(fg) = E(f)*E(g)), then the weigth don't change the vote. That's correct, and nobody need stat to understand that if the guru alone would give the same note that the rookies alone, the combined note would be the same.
if (D) is true (that is code project hypothesis), the weight system allow the combined note to be closer to the opinion of the guru. And also, if (D) is true, your demonstration is false.
So, rather than a show off of mathematical formulas, I expected, from the title of the article, a dicussion on :
Does the guru vote so differently than the rooky ?
or:
Why weights with 1 2 3 4 5 instead of 10 20 30 40 50 ?
or:
Why not a note by membership level in addition to the global one ?
or
Why not discarding the 10 percent lower and 10 percent higher note to reduce data polution.
and there are many others.
As a matter of fact, the ranting system is an ARBITRARY choice, reflecting codeguru personal opinions on the best way to give feedback on article interest.
Weighted sum as they are used seems ok to me, as they give a weighted mean if (D), and the correct mean for (I) too (you only proved that).
But your concusion is wrong. When you say :
"It was discovered that although the weight of each person in the system is taken into account, the mean value doesn’t depend from it."
, you make a 'statistician rooky' mistake, forgetting that a demonstration as no value as proof if the hypothesis are no checked.
Of course, up to you to pretend that it's a matter of personal opinion to think that experienced programmer judge an article the same way than unexperienced one,
and up to me to think that as long as the rating system is logical mathematically correct and make common sens assomptions, I won't find it smart or dumb, but just fitting for the job.
|
|
|
|
|
Hello,
>>(I) "Wi and Vi independent" imply : guru and rookie vote in similar way
>>(D) "Wi and Vi dependent" imply : guru and rookie don't vote in similar way
>>(given you level of math, and to keep that post short, I don't think I need
>>to proove or explain better the aboves implication, but I can if you ask so)
Please provide me with the proof and also explain what are Wi and Vi and why do you speak of their dependency and independency.
>>if (D) is true (that is code project hypothesis), the weight system allow the combined note to
>>be closer to the opinion of the guru. And also, if (D) is true, your demonstration is false.
Right because in this case the vote and weight random variables are dependent.
>>So, rather than a show off of mathematical formulas, I expected, from the title of the article,
>>a dicussion on :
>>Does the guru vote so differently than the rooky ?
>>or: Why weights with 1 2 3 4 5 instead of 10 20 30 40 50 ?
>>or: Why not a note by membership level in addition to the global one ?
>>or: Why not discarding the 10 percent lower and 10 percent higher note to reduce data
>>polution.
>>and there are many others.
These questions are beyond the scope of my article but may be later I will also concern them.
>>Weighted sum as they are used seems ok to me, as they give a weighted mean if (D), and
>>the correct mean for (I) too (you only proved that).
This makes sense. That is why I voted 5 for your post J. But what do you mean by “as they give a weighted mean if (D)” can you predict and estimate the influence?
>>But your concusion is wrong.
>> When you say :
>>"It was discovered that although the weight of each person in the system is taken into
>>account, the mean value doesn’t depend from it."
>>, you make a 'statistician rooky' mistake, forgetting that a demonstration as no value as proof
>>if the hypothesis are no checked.
I would not be so captious if I was you but that is yours opinion. Although I agree that to make a proper model you have to do statistical analysis. This requires a lot of data I do not have access to. That is why I made some assumptions and proved something for them this is better then not to do anything at all.
>>Of course, up to you to pretend that it's a matter of personal opinion to think that
>>experienced programmer judge an article the same way than unexperienced one,
>>and up to me to think that as long as the rating system is logical mathematically correct and
>>make common sens assomptions, I won't find it smart or dumb, but just fitting for the job.
So many men, so many minds … Good conclusion!
Thanks for the valuable feedback!
Ps: By the way the title of your post seems to be strange...
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
BTW. I think that in general you are right and the approach with weights is more general that is why I have fixed my conclusions.
Thanks,
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
Hello,
>>Please provide me with the proof and also explain what are Wi and Vi >>and why do you speak of their dependency and independency.
You are right, I typed too fast. I meant Wt and Vt, and will use Wi and Vi for instances of Wt and Vt.
I speak about dependency because you said in you article :
"If they are independent, then E(fg) = E(f)E(g)"
Well, if they are NOT independent, E(fg) become different from E(f)E(g), and you calculation of E(Rn) can't be done this way because :
sigma(E(WiVi/sigma(Wi))) is not equal to sigma(E(Wi/sigma(Wi)))*E(Vi)
and you can't conclude the E(Rn) doesn't depend from weight.
Definition for statistical independence :
P(A|B) = P(A)
In english : the probability P of the event A knowing the event B is the same as the probaility of the event A alone.
In our case, A=Vi is a vote between the possible vote values, and B=Wi the membership level of the voter.
If Vt is independant of Wt, the probability for a rooky to give a 1 on a specific article is the same as the probability for a guru to do so. And the same for 2, 3, 4 and 5. In plain english, it mean they vote the same way.
If P(Vt|Wt) is not equal to P(Vt), it mean the probability to get a vote Vi is different when you know Wt for at least some Vi. I othere words, it means that the probability to get a given vote is not the same depending on the voter weight.
>>But what do you mean by “as they give a weighted mean if (D)” can >>you predict and estimate the influence?
Absolutly no The impact depend of the the shape of the vote distributions for each weight level, so you would need the "real" distribution of the votes. I would postulate it may be different from one article to the other(normal, lognormal, bimodal etc...), and many article don't get enough vote to assess the distribution.
The simpler way to describe the weighted sum is to say it's equivalent to have people vote as many time as their weight. A level 5 guru vote as the same impact on the average vote as 5 level 1 rookies giving the same vote.
But you made a good point : The need for stats and 'model' arise when you want to make predictions.
>> This requires a lot of data I do not have access to
Yes, you need data to make an analysis, but it does not mean you can always find a model out of the data. In that case, you would have to use the parcimony principle, and give a article's value estimator using as few assumptions as possible : the aritmetic mean
>> Ps: By the way the title of your post seems to be strange...
I would have expected, after you found E(Rn) to ignore weight, that you went back and explain why and in which cases. But you just stopped on a conclusion hurting common sens and made it look as a fact because of the statistical formalism.
Thank for your article, even (or because) it forced me to look deeper in math books (I didn't remebered the mathematical definiton of dependency, while I use the concept all the time)
minox
|
|
|
|
|
Assume a uniform independent distribution of random variables
such that samples are drawn from the following sample space [1,5].
Consider a possible sample population:
{1, 1, 1, 4, 5}
Given equal weights, the mean of the sample population is:
2.4
Using your notation, let a = 2.4.
We therefore expect to find means of our sample independent
of the weights and equal to 2.4, the expected value of our
distribution.
Now, consider the following two weight vectors:
{.3, .3, .3, .05, .05}
The mean of our distribution is now:
1.35
Consider an alternate set of weights:
{.1, .1, .1, 0.35, 0.35}
The mean of the distribution is now:
2.19
Neither mean is equal to the expected mean of 2.4.
|
|
|
|
|
Dear, John Theal
It was very interesting to learn your counterexample.
Unfortunately it doesn''t sound.
First of all:
>> Assume a uniform independent distribution of random variables
>> such that samples are drawn from the following sample space [1,5].
In this case the mean value of the vote is 3.
As far as you have 5 possible values:
1,2,3,4,5
According to your assumption about uniform distribution for each of them the probability equals 1/5.
Thus from the definition of the mean value we have:
1*1/5+2*1/5+3*1/5+4*1/5+5*1/5 = 3
>> Consider a possible sample population:
>> {1, 1, 1, 4, 5}
>> Given equal weights, the mean of the sample population is:
>> 2.4
>> Using your notation, let a = 2.4
>> We therefore expect to find means of our sample independent
>> of the weights and equal to 2.4, the expected value of our
>> distribution.
>> Now, consider the following two weight vectors:
>> {.3, .3, .3, .05, .05}
>> The mean of our distribution is now:
>> 1.35
>> Consider an alternate set of weights:
>> {.1, .1, .1, 0.35, 0.35}
>> The mean of the distribution is now:
>> 2.19
From your reasoning represented above it is obvious that you do not see the difference between the rating value itself and the mean value of the random variable in the sense of the probability theory.
Unfortunately this is completely wrong and it looks like you have to recall at least the definitions of the mean value of the random variable.
The values 1.35 and 2.19 that you have calculated are the values of the rating. They represent the concrete example and have almost noting to do with the mean value of the rating variable.
The point is that there is always dispersion relative to the mean value that can be estimated by calculation of the variance of the random variable.
Sorry for disappointing you but thanks for your interest!
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
|
This only showed that you are not aware of the definition of the mean value.
It seems to me that you didn’t read my article carefully but I am not going to explain everything again. Simply try to read it once more.
BTW information that you have provided about Markov chains is quite known and thus represents nothing new to me.
Ps: It is much easier to write about possibility to do smth than to really do it.
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
(These calculations are trivial and self descriptive)
That line gave me a chuckle. I used to be a math geek, but I never got into statistics so my eyes started to glaze over...
--Mike--
Personal stuff:: Ericahist | Homepage
Shareware stuff:: 1ClickPicGrabber | RightClick-Encrypt
CP stuff:: CP SearchBar v2.0.2 | C++ Forum FAQ
----
Pinky, are you pondering what I'm pondering?
I think so Brain, but if we shaved our heads, we'd look like weasels!
|
|
|
|
|
I just to played the fool not to give the detailed explanation
This is widely used among mathematicians it not it?
But still if something is unclear I can give additional explanations.
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
This is not a proof.
You have shown nothing of interest here other than the mean of
some (unspecified) distribution is equivalent to the mean that
you have assumed. You should look up the implications of leptokurticity. Then you
can calculate an estimator for a (specific) distribution instead of
showing that the mean of a distribution (assumed to have a mean of a) is equal to a.
I don't buy your independence assumption. Something of more interest
would have considered the kurtosis of the distribution, NOT the mean.
|
|
|
|
|
John Theal wrote:
implications of leptokurticity
Hey , I didnt say anything!
|
|
|
|
|
Hi,
>>You have shown nothing of interest here other than the mean of
>>some (unspecified) distribution is equivalent to the mean that
>>you have assumed.
You are right but the mean value itself shows that long time average value of the rating is simply the same as the average value of the vote and doesn't depend on the members’ status.
Even more it is the same as the mean value for other voting types I have discussed in other article. But the way this value is calculated is more complicated.
>>You should look up the implications of leptokurticity.
Sorry I didn't get it. Can you express it in more popular way?
>>Then you can calculate an estimator for a (specific) distribution instead
>>of showing that the mean of a distribution (assumed to have a mean of a) is >>equal to a.
The distribution of the rating might be different from the distribution of the vote itself especially as the weight distribution in explicitly involved. I only showed that the mean value of rating distribution doesn't depend from the distribution of weights although it could be expected.
>>This isn't even a model, this is a textbook exercise.
True!
Even more this is an exercise for the introduction to the theory of probability that anyone should know! But it still allows us to do some analysis.
By the way I called it a model only because I made some assumptions about independency of the random variables etc.
>>I don't even buy your independence assumption, it is more likely that
>>these votes are conditionally heteroskedastic. Something of more interest
>>would have considered the kurtosis of the distribution, NOT the mean.
This sounds smart but meaningless to me. Try to use standard terms to express your ideas.
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
Kurtosis is the 4th order moment of the distribution (the mean being the 1st order moment and the std. dev. the 2nd and variance the 3rd.) It measures the peakedness of the distribution.
Kurtosis, leptokurtic and conditional heteroskedasticity are all common terms used in the literature. You cannot read articles in the statistical journals without knowing them.
You can find more information here: http://mathworld.wolfram.com/Kurtosis.html[^]
|
|
|
|
|
Thank you for provided information.
Why do you think that the 4th order moment is so interesting for the voting system? What about the 2th order moment?
Ivan S Zapreev
cerndan@mail.ru
|
|
|
|
|
All you showed in this article is that assuming independence of votes and weights, the expected rating has the same value as the expected vote.
This seems quite obvious, and in fact I wouldn't trust a rating algorithm that didn't have a similar property. Yet in your conclusion you say that this algorithm is not good. Care to explain why?
Besides, you assumption of independence is incorrect. When a user votes, he can see the current rating and his vote is influenced by it. Also, it's quite possible that people with different weights have different voting distributions.
|
|
|
|
|