Alternative Film Poll Formula Discussion

Post Reply
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Alternative Film Poll Formula Discussion

Post by BleuPanda »

We've sparked a discussion in the 2010s film poll results topic about issues with the formula we have been using - I think it would be for the best if we move the discussion to a separate thread so as not to distract from the results.

I think the basic issue has been that underseen films with one fan but otherwise largely unappreciated inherently outdo films that most of us have seen, even if it appears our consensus votes prefer the latter film - essentially, films are rewarded for not being watched, which feels contrary to the spirit of the poll. There must be some way to accommodate lesser known films without giving such an obvious advantage. The question is, how?

So, here is a place to throw out suggestions.

My idea was that the 'unseen score' should fluctuate for those landing in the lower half. If a film is in the upper half, the 50% mark feels fair. But in the lower ranks, the unseen score actually drags movies upward. My method would take a bit more effort to determine, but my idea would be to use the average percentage rank for each film. So, instead of giving every film a flat assumption of an exact middle rank, the assumption is instead based off what those who have seen it think. As far as I'm concerned, the current unseen score works for the top results but falls apart on the back end, and this would mitigate that issue.

I would still use 50% for those with a better average ranking, as we would want to avoid a movie with only 2 or 3 viewers who all loved it inexplicably placing it at the very top - after all, there must be some reason stopping others from having seen it. So, my suggestion is either 50% or the average percentage, whichever is lower.

(others have made their own suggestions but I now have to step away from the computer)
stone37
Different Class
Posts: 323
Joined: Mon Feb 20, 2012 12:39 am

Re: Alternative Film Poll Formula Discussion

Post by stone37 »

I think BleuPanda's suggestions are good ones. I wondered if it might be a good idea to take just 10 or 12 films from each year and not ask for bonus selections. Personally, if there were fewer films, I would be more apt to track down the pics I have not seen yet. I watched more than two dozen films for this poll, but I was not prepared to drop over 200 US dollars to rent the remaining films.
User avatar
Petri
Feeling Good
Posts: 1107
Joined: Fri Feb 10, 2012 6:13 pm
Location: Helsinki, Finland

Re: Alternative Film Poll Formula Discussion

Post by Petri »

BleuPanda wrote: Tue Jun 21, 2022 9:25 pm
My idea was that the 'unseen score' should fluctuate for those landing in the lower half. If a film is in the upper half, the 50% mark feels fair. But in the lower ranks, the unseen score actually drags movies upward. My method would take a bit more effort to determine, but my idea would be to use the average percentage rank for each film. So, instead of giving every film a flat assumption of an exact middle rank, the assumption is instead based off what those who have seen it think. As far as I'm concerned, the current unseen score works for the top results but falls apart on the back end, and this would mitigate that issue.

I would still use 50% for those with a better average ranking, as we would want to avoid a movie with only 2 or 3 viewers who all loved it inexplicably placing it at the very top - after all, there must be some reason stopping others from having seen it. So, my suggestion is either 50% or the average percentage, whichever is lower.

(others have made their own suggestions but I now have to step away from the computer)
This sounds a good idea. I would add few options:
If the movie is a wildcard nomination we exclude the affect of the nominator (for the score of unseen not in the score itself). So for example the 'unseen' Bol votes would be situated at the 98.61% point instead of the 79,22% or 50% point.
I would also consider if there's a need for wildcard nominations at all or at least we should diminish the number of movies people can nominate.

I would also recommend weighting the score by how many movies the voter have seen. The weighting (more info, check the polls I have hosted and find "correction factor") will do less harm for the movies in the lower half of the lists of the people who have seen only few movies (it also decreases the impact of voter who have seen less movies which is fair in my opinion).
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Alternative Film Poll Formula Discussion

Post by BleuPanda »

Petri wrote: Wed Jun 22, 2022 11:31 am
BleuPanda wrote: Tue Jun 21, 2022 9:25 pm
My idea was that the 'unseen score' should fluctuate for those landing in the lower half. If a film is in the upper half, the 50% mark feels fair. But in the lower ranks, the unseen score actually drags movies upward. My method would take a bit more effort to determine, but my idea would be to use the average percentage rank for each film. So, instead of giving every film a flat assumption of an exact middle rank, the assumption is instead based off what those who have seen it think. As far as I'm concerned, the current unseen score works for the top results but falls apart on the back end, and this would mitigate that issue.

I would still use 50% for those with a better average ranking, as we would want to avoid a movie with only 2 or 3 viewers who all loved it inexplicably placing it at the very top - after all, there must be some reason stopping others from having seen it. So, my suggestion is either 50% or the average percentage, whichever is lower.

(others have made their own suggestions but I now have to step away from the computer)
This sounds a good idea. I would add few options:
If the movie is a wildcard nomination we exclude the affect of the nominator (for the score of unseen not in the score itself). So for example the 'unseen' Bol votes would be situated at the 98.61% point instead of the 79,22% or 50% point.
I would also consider if there's a need for wildcard nominations at all or at least we should diminish the number of movies people can nominate.

I would also recommend weighting the score by how many movies the voter have seen. The weighting (more info, check the polls I have hosted and find "correction factor") will do less harm for the movies in the lower half of the lists of the people who have seen only few movies (it also decreases the impact of voter who have seen less movies which is fair in my opinion).
For the bonus films in the all-time poll, I am going to limit the pool to those that landed in the 31-60 range in their decade poll. I think something similar going forward could be good for the decade polls - something like it must land in the top 30 for the year and have at least two (maybe three, depending on the number of voters) votes. Which I think was the original intent of the bonus films - to rescue films that get drowned out in strong years.
Humphrey9
Superunknown
Posts: 4
Joined: Mon Feb 28, 2022 10:47 am

Re: Alternative Film Poll Formula Discussion

Post by Humphrey9 »

BleuPanda wrote: Tue Jun 21, 2022 9:25 pmessentially, films are rewarded for not being watched, which feels contrary to the spirit of the poll. There must be some way to accommodate lesser known films without giving such an obvious advantage. The question is, how?
Watch them?
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Alternative Film Poll Formula Discussion

Post by BleuPanda »

Another method I have considered is taking the full spread of points but having everyone start at the middle instead of from the top down. So, say we have 250 entrants and you rank 50 - your choices would occupy the range of 101-150.

The big issue I see with this is it starts heavily benefitting those who have seen everything due to the scaling at the top. I want to try something out with the all-time poll. I think I will try this method:
1. All entrants start from the center point, which should naturally give more weight to people who have seen more films
2. To curb this back a bit, with ~300 films in the final list, the ranking will only process the top 125 and bottom 125. Essentially, everyone who ranks 250 films and everyone who ranks 300 films will have the same weight. All the excess films in the middle will share the middle points like an unranked film with our current methodology.

I think this method both encourages people to try to watch more of the entrants without making it feel like you have to seek out every single one to be fully counted. I think this still matches our current method but in a way that better accommodates both ends.
User avatar
prosecutorgodot
Keep On Movin'
Posts: 1547
Joined: Tue Apr 14, 2015 5:53 am
Location: SF Bay Area, California

Re: Alternative Film Poll Formula Discussion

Post by prosecutorgodot »

Pretty good proposal, BP.

I don't 100% understand your correction for scaling, but my thought was to leave an initial "buffer zone" of 50/300 in the middle (in that case 126-175), so you'd start scoring around that buffer zone - so 50 ranked would have scores of 101-125 and 175-200. After surpassing 250 you'd start filling in that buffer zone.
Post Reply

Return to “Films, Movies, Motion Pictures...”