BleuPanda wrote:Beatsurrender24 wrote:Hi there. Thanks a lot for creating this spreadsheet – it's a really useful way of discovering great songs that I may have missed last year.
I'm a little confused by some of the maths, though. From what I can tell from a quick look over the formulae, the final song on a list receives the same score as the final song on any other list, regardless of how long the list is. So, for example, Something Just Like This by The Chainsmokers was named the 50th best song of 2017 in Uproxx's Top 50, and it therefore has approximately the same score as On + Off by Maggie Rogers, which Pitchfork named the 100th best song of 2017 in their Top 100.
Doesn't this mean that the spreadsheet is biased towards longer lists? The song that Pitchfork named as the 50th best of 2017 (Show You the Way by Thundercat) would have received twice the score of Something Just Like This, even though they were both at the same position (number 50) on their respective lists.
If you follow that through to its logical conclusion, that would then mean that, for example, being named the third best song on Reactor 105.7's list (106 entries long) is worth almost 11 times more than being named the third best song on The Music's list (only 10 entries long), heavily biasing the spreadsheet in Reactor 105.7's favour. This seems a little unintuitive to me. Surely a song's position on a list should be worth the same score irrespective of how long the list is, or how many other songs are beneath it?
Anyway, those are just my thoughts. It's your spreadsheet, so you can compile it however you want. Apologies if I'm reading the maths completely wrong and misrepresenting your calculations. Thanks again for the great work!
The problem is, if you were to simply make every position worth the same no matter the size of the list, it harms a lot of songs that don't make smaller lists simply due to lack of space. I believe the main purpose to having such a scale is so that not appearing on a list is generally the same value. I'm not sure what the numbers in this case are, but imagine a scale where rank 1-100 were given 100-1 points depending on their rank. In a list with 100 songs, the difference between being #100 and #101 is essentially 1 point. If a list only contains 10 songs, the difference between #10 and #11 would be 91 points! That would give a lot of weight towards lists that frankly give us less information to actually work with, and gives an unfair advantage to those few songs that manage to make short lists. The issue isn't raw value as much as the difference assigned between positions.
BleuPanda wrote:andyd1010 wrote:Interesting. So longer lists allocate way more points than shorter lists - per song in addition to the larger number of songs. I didn't realize that. I can see the pros and cons of both approaches. I have a similar project I've mentioned before, and I initially went with Sweepstakes Ron's approach before switching to Beatsurrender24's approach. But I just don't give any points to songs that don't appear on a list, which makes it a little different than the problem BleuPanda mentions about assigning an appropriate amount of points to songs that didn't make the cut.
Well, it's not like anyone's assigning points to songs that don't appear on a list; the problem is how much higher than zero the songs that are on the list are getting. Being #3 on a list of the top 100 songs is different than being #3 on a top 10. In the end, what really matters is the point difference, not the total value of points. So, in my example distribution, every song ranked outside the top 100 would be 98 points less than song #3; but there are 97 songs that have smaller differences! The problem is that smaller lists have an additional 90 songs that we gain no information on. To me, with a top 100 list, I can go ahead and agree that the point distribution between an imagined #101 and #200 would be so small that it wouldn't matter; but for a top 10 list, I think the imagined difference between the non-existent #11 and #101 does matter. Somewhere out there is a song that should be getting 90 points that isn't (if I'm using 100 as the ideal size).
The easiest solution is to lower that size difference; it's messy, but you need some method that accommodates for different list sizes. In other words, it's better to treat every unranked song as #11 if the list only has 10 songs. Why should they be treated as #101, other than the fact that other lists go to 100? That's essentially what using a flat rate suggests. A flat rate statistically gives more weight to smaller lists; which I hope we would agree is a bad thing, right?
In a way, you are always assigning every song points; zero is a value. It's all relative.
Of course, this is just me giving my own analysis for why there should be different weighing based off the size of a list; Henrik likely has entirely different reasons.
I decided to make a new thread so this doesn't end up cannibalizing the EOY songs thread.
First of all I want to say how much I appreciate the work you've done, Sweepstakes Ron, and to everyone else who has worked hard on this site to create these lists we all enjoy! I'm not saying I necessarily think you should change your method, but I think it's worth discussing especially since I use a different method with my own spreadsheet, and maybe I can be convinced that my method isn't ideal.
Well BleuPanda, it does seem like this system assigns points to songs that don't appear on a list - you're saying the songs outside that hypothetical list all get credit for being #11, etc. But that isn't what I meant with my post - A longer list already has more weight by virtue of being longer, even if each song of equal rank is weighted the same. So then to devalue the lower ranks of smaller lists creates an even bigger difference in the impact those lists have on the final results.
I guess you're arguing otherwise because of this: Say a 100-song list and a 10-song list have the exact same top 10. You'd argue that with equal point allocation the 10-song list has "more weight" because those 10 songs gain so much on the field with the 10-song list, while with the 100-song list many of their competitors in the spreadsheet appear later in the list and get a reasonable amount of points, so the top-10 songs gain less drastically on the field. Am I understanding the argument correctly or is there more to it?
My feeling is that songs that don't appear on small lists aren't at too much of a disadvantage since only 10 songs gain on them. Regardless of the allocation, a song that does not appear on either list is always impacted more negatively by the inclusion of the 100-song list, since 100 songs gain on it and the top songs are still gaining at least as much on the 100-song list as they do on the 10-song list, if not more, right? So other than my one example I don't see how smaller lists could be seen as having more weight.
How does it help to come up with hypothetical points for songs that might have appeared if lists were longer, rather than just using a formula that spits out a number of points for each rank regardless of list size and giving 0 points to songs that aren't listed (isn't that what we do for most of our polls, where lists of various lengths are accepted)?
Thanks in advance for the clarification!