Acclaimed Video Games site

User avatar
Moonbeam
Full of Fire
Posts: 2543
Joined: Mon Feb 06, 2012 11:40 am
Location: Sydney, Australia

Acclaimed Video Games site

Post by Moonbeam »

Some of you may remember a poster with the username Moeboid who used to post on AMF. He's taken inspiration from Acclaimed Music and launched a version for videos games with Acclaimed Video Games. Since I've seen a few video game posts in this forum, I thought it might be of interest, particularly as it's from an AMF alumnus! :happy-partydance: Full confession: Moeboid reached out to me to ask for a few pointers on the ranking algorithm, but he's taken the little bits of advice I gave and run a marathon with them. He's said the site is still in its early stages visually, but the data used to inform the rankings is quite extensive.
User avatar
Krurze
Movin' On Up
Posts: 877
Joined: Tue Aug 12, 2014 10:10 pm
Location: Hamburg

Re: Acclaimed Video Games site

Post by Krurze »

Thanks for sharing! I have huge respect for these kinds of projects and it's always fun scanning these lists for your personal favourites.

Also I had a little déjà-vu when opening this thread, as another AMF member has created a website with a very similar concept some time ago. It's called PlayThatGame and it was shared on here on AMF in this thread.


There's probably some methodological differences and both are clearly labours of love, but I still thought maybe this would be interesting for Moeboid if he sees this :)
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Wow, PlayThatGame is seriously impressive. And I'm embarrassed to say that I had not even heard of it before reading the above post. All this time I was under the impression that my approach was unique in its ability to take into account such a wide variety of lists. But PTG is able to take an even wider variety of lists into account than I can. It even has genre exclusive lists, which aren't even on the horizon for me at this stage.
Honestly, my first thought upon seeing PTG was that my list was obsolete and that all the time I've spend over the past few years putting it together had been wasted. But just from the results I can see that there are strong methodological differences between us. Enough to justify both of our lists existing.
Anyway, thanks for the plug, Moonbeam. And hello to everyone who remembers me. I had to check when I last posted regularly here, and it was all the way back in 2009. The passing of time is so strange.
Now excuse me while I add all the PTG lists that I can include.
User avatar
Krurze
Movin' On Up
Posts: 877
Joined: Tue Aug 12, 2014 10:10 pm
Location: Hamburg

Re: Acclaimed Video Games site

Post by Krurze »

Yes, I thought it might be a great resource for your own project. Glad to hear you're motivated to keep working on your website!
User avatar
ValenN
Wannabe
Posts: 103
Joined: Sat Sep 10, 2016 10:29 pm

Re: Acclaimed Video Games site

Post by ValenN »

Congrats on the site!
User avatar
VacantJoy
Leader of the Pack
Posts: 10931
Joined: Mon Oct 05, 2020 1:09 pm
Contact:

Re: Acclaimed Video Games site

Post by VacantJoy »

Moeboid wrote: Sun Apr 02, 2023 9:23 am
how long did it take you to create the site?
DONATIONS TOWARDS ME -> https://revolut.me/anastagf0s
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Vacantjoy wrote: Fri Apr 07, 2023 2:40 am how long did it take you to create the site?
The website or the ranking algorithm? Because the answer is very different. I've been working on the ranking algorithm, on and off, for a few years now. It would be hundreds of hours of coding and data entry in total. But once all that was done, the website didn't take long at all. I didn't do it because I don't have the skills for it, but I'm told it took around 15 hours.
User avatar
VacantJoy
Leader of the Pack
Posts: 10931
Joined: Mon Oct 05, 2020 1:09 pm
Contact:

Re: Acclaimed Video Games site

Post by VacantJoy »

Moeboid wrote: Fri Apr 07, 2023 5:25 am
Vacantjoy wrote: Fri Apr 07, 2023 2:40 am how long did it take you to create the site?
The website or the ranking algorithm? Because the answer is very different. I've been working on the ranking algorithm, on and off, for a few years now. It would be hundreds of hours of coding and data entry in total. But once all that was done, the website didn't take long at all. I didn't do it because I don't have the skills for it, but I'm told it took around 15 hours.
a few years ? can you you be more specific? :romance-kisscheek: what did take you more time to find the lists or coding them?
DONATIONS TOWARDS ME -> https://revolut.me/anastagf0s
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Vacantjoy wrote: Fri Apr 07, 2023 6:18 am
Moeboid wrote: Fri Apr 07, 2023 5:25 am
Vacantjoy wrote: Fri Apr 07, 2023 2:40 am how long did it take you to create the site?
The website or the ranking algorithm? Because the answer is very different. I've been working on the ranking algorithm, on and off, for a few years now. It would be hundreds of hours of coding and data entry in total. But once all that was done, the website didn't take long at all. I didn't do it because I don't have the skills for it, but I'm told it took around 15 hours.
a few years ? can you you be more specific? :romance-kisscheek: what did take you more time to find the lists or coding them?
Well it's a project that's slowly become more ambitious over time, so it's hard to say when I first started working on the project in its current form. The first attempt at aggregating critic lists was back in 2020, but that was just for games of the previous decade, and I certainly didn't think I'd be launching a website for it. I just wanted to see what I could do.
That first version of the program was very poor. Data entry was tedious, the aggregation method wasn't great, and my Python code was slow to the point where I didn't even think it would handle all time lists. So I let it sit for a year or so.
The second attempt was written in C# (once I'd learned how to use it!), and it addressed most of the previous issues. Data entry was a lot easier, and it was fast enough to handle all time lists. This is when I did most of the data entry. However, the aggregation method still had a few weaknesses that I couldn't figure out, so again I let it sit for a year or so.
The version that I'm using now was written in 2022, and it uses a much more sophisticated aggregation method. It's not perfect, but I'm still really proud of it, and I wanted to launch a website for it because I think it's better than what some other websites are doing.

Anyway, to answer your question more directly, I would guess that the coding took the longest, but I can't really be sure. It's been so long that I don't really remember how much time everything took. And it's possible that the coding only took longer because I'm not very good at it. I'm sure I spent more time looking up how to do stuff than actually doing it, so maybe it took a lot longer than it should've, or longer than it would've taken someone else to do the same thing. I'm not sure, but it's also not important to me. What matters is that I really like what I've got now, and I think it's worth sharing with people.
User avatar
VacantJoy
Leader of the Pack
Posts: 10931
Joined: Mon Oct 05, 2020 1:09 pm
Contact:

Re: Acclaimed Video Games site

Post by VacantJoy »

Moeboid wrote: Fri Apr 07, 2023 8:24 am
Vacantjoy wrote: Fri Apr 07, 2023 6:18 am
Moeboid wrote: Fri Apr 07, 2023 5:25 am

The website or the ranking algorithm? Because the answer is very different. I've been working on the ranking algorithm, on and off, for a few years now. It would be hundreds of hours of coding and data entry in total. But once all that was done, the website didn't take long at all. I didn't do it because I don't have the skills for it, but I'm told it took around 15 hours.
a few years ? can you you be more specific? :romance-kisscheek: what did take you more time to find the lists or coding them?
Well it's a project that's slowly become more ambitious over time, so it's hard to say when I first started working on the project in its current form. The first attempt at aggregating critic lists was back in 2020, but that was just for games of the previous decade, and I certainly didn't think I'd be launching a website for it. I just wanted to see what I could do.
That first version of the program was very poor. Data entry was tedious, the aggregation method wasn't great, and my Python code was slow to the point where I didn't even think it would handle all time lists. So I let it sit for a year or so.
The second attempt was written in C# (once I'd learned how to use it!), and it addressed most of the previous issues. Data entry was a lot easier, and it was fast enough to handle all time lists. This is when I did most of the data entry. However, the aggregation method still had a few weaknesses that I couldn't figure out, so again I let it sit for a year or so.
The version that I'm using now was written in 2022, and it uses a much more sophisticated aggregation method. It's not perfect, but I'm still really proud of it, and I wanted to launch a website for it because I think it's better than what some other websites are doing.

Anyway, to answer your question more directly, I would guess that the coding took the longest, but I can't really be sure. It's been so long that I don't really remember how much time everything took. And it's possible that the coding only took longer because I'm not very good at it. I'm sure I spent more time looking up how to do stuff than actually doing it, so maybe it took a lot longer than it should've, or longer than it would've taken someone else to do the same thing. I'm not sure, but it's also not important to me. What matters is that I really like what I've got now, and I think it's worth sharing with people.
thanks for sharing and congrats!
DONATIONS TOWARDS ME -> https://revolut.me/anastagf0s
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Little announcement about the project: the first ever update to the list will be happening very soon, hopefully within the next couple of days. The update reflects the addition of about 50 new lists, many of which are all time lists (i.e. the ones that have the biggest impact on the rankings), a much more rigorous weighting algorithm, and a few corrections/bug fixes (including one fairly major one). This all resulted in some pretty significant changes, including a new no#2. (No, it's not Tears of the Kingdom--TotK is too new to have appeared on any lists yet, though I expect once the lists are out it will place extremely highly.) I'll keep the game that replaced it a secret, but I am curious to know what people think once it's all live.
Nassim
Full of Fire
Posts: 2793
Joined: Fri Feb 10, 2012 2:35 pm
Location: Lille (France)

Re: Acclaimed Video Games site

Post by Nassim »

Moeboid wrote: Mon May 15, 2023 12:10 pm Little announcement about the project: the first ever update to the list will be happening very soon, hopefully within the next couple of days. The update reflects the addition of about 50 new lists, many of which are all time lists (i.e. the ones that have the biggest impact on the rankings), a much more rigorous weighting algorithm, and a few corrections/bug fixes (including one fairly major one). This all resulted in some pretty significant changes, including a new no#2. (No, it's not Tears of the Kingdom--TotK is too new to have appeared on any lists yet, though I expect once the lists are out it will place extremely highly.) I'll keep the game that replaced it a secret, but I am curious to know what people think once it's all live.
Great news !
Well the new GQ list says it should be the Last of Us, but I'd bet on the Witcher 3. Nothing from the current top 10 would shock me as the 2nd greatest game ever in any case !
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Update is finally live! Took longer than expected, but the extra time was spent making sure that it will be easier and faster to update next time. Well worth doing.
New no#2 could be a surprise for some people. It's not The Last of Us or The Witcher III, sorry Nassim. Actually on the previous version of the list it wasn't even in the top 10. If anyone's curious as to how it moved up so much, I'll probably have a blog post about it that I could share soon. It's not as simple as it might seem.
Nassim
Full of Fire
Posts: 2793
Joined: Fri Feb 10, 2012 2:35 pm
Location: Lille (France)

Re: Acclaimed Video Games site

Post by Nassim »

Moeboid wrote: Sun May 21, 2023 1:12 am Update is finally live! Took longer than expected, but the extra time was spent making sure that it will be easier and faster to update next time. Well worth doing.
New no#2 could be a surprise for some people. It's not The Last of Us or The Witcher III, sorry Nassim. Actually on the previous version of the list it wasn't even in the top 10. If anyone's curious as to how it moved up so much, I'll probably have a blog post about it that I could share soon. It's not as simple as it might seem.
Oh yeah wow... Great game for sure but having those 2 at the top looks like a bit of recency bias, any way the calculation could favor recent games ?
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

The presence of Forbidden West and Pokemon Legends Arceus in the all-time top 100 feels really off.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

Like, just comparing Elden Ring at #2 and Ocarina of Time at #3; Elden Ring has 7 all-time list mentions and Ocarina of Time has 58. And of those 7 Elden Ring lists, Ocarina of Time is higher on 3 of them. These numbers simply don't make sense to me. It's important to balance the fact that new works can't appear on old lists, but this feel far, far too weighted in the other direction.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Mon May 22, 2023 8:04 pm Like, just comparing Elden Ring at #2 and Ocarina of Time at #3; Elden Ring has 7 all-time list mentions and Ocarina of Time has 58. And of those 7 Elden Ring lists, Ocarina of Time is higher on 3 of them. These numbers simply don't make sense to me. It's important to balance the fact that new works can't appear on old lists, but this feel far, far too weighted in the other direction.
There are a few things happening here. The first is that the list makes no attempt to compensate for recency bias, or sample size error. If a list were released tomorrow with a 2023 game in first place, then that game would be first overall as well (and by a huge margin). The previous version of the list actually placed recent games more conservatively because of this, but I stopped doing that for this update. I don't necessarily prefer it this way, I'm just still in the process of experimenting with the formula.

Anyway, to answer Nassim's question more directly, no, the formula is not biased towards recent games in any way, but recent games are naturally a lot more prone to sample size error. Once more lists come in, I suspect quite a few recent games will go down in the rankings.

The other thing that's happening is I think people are overestimating how acclaimed Ocarina of Time is. Elden Ring has only not been mentioned on one of the eight lists that it was eligible for so far, where as for Ocarina of Time that number is 16/74. Most of those are either short lists, or lists with a one-per-series limitation (the formula accounts for this), but not all of them, and the combined effect is that Ocarina of Time is just a lot more beatable than I think people realise.

Ocarina of Time's ranking has also dropped significantly since Breath of the Wild's release. Here is a comparison of where every publication that I have data for ranked Ocarina of Time before and after Breath of the Wild:
Edge: 8th in 2015, 9th in 2017
Game Informer: 20th in 2009, 39th in 2018
Gaming Bolt: 1st in 2013, not even mentioned in 2022 (or 2023)
IGN: 8th in 2015, 25th in 2019 (34th in 2022)
Jeuxvideo: 18th in 2011, 55th in 2017
Popular Mechanics: 11th in 2014, 12th in 2019
Slant: 1st in 2014, 12th in 2018
Stuff: 2nd in 2014 (I think), 40th in 2017
And in each case the formula favours the more recent list.

I don't know what's going to happen once more lists come in, obviously, but if I had to guess then I would say that Elden Ring is going to remain ahead of Ocarina of Time (unless OoT is the beneficiary of BotW and TotK trading blows). I wouldn't even be particularly surprised if a few other games overtake it as well.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

Sorry, but the fact that a single list putting a 2023 game at #1 right now would make it #1 on your compiled list by a huge margin makes no sense at all. This is a horrible way of compiling data and you really should reconsider how you are weighing lists. Your entire formula appears weighted to give absurd advantages to modern releases. This entire site feels incredibly misleading as is.

Right now, it reads as if scores are simply calculated by what percentage of potential lists it could have been on; a list published this year should not benefit a more recent game at a lower rank than an older game at a higher rank. Your formula is not grading games on the same criteria. Also, due to this structure, I imagine recent games will rapidly decay in their ranking over time as soon as they miss a few lists. Also, are you accommodating for the number of people involved in the list-making process? That GQ list polled about 300 people, correct? As such, I'm hoping it has a lot of weight compared to the others. It feels like games are punished more for missing lists than they benefit from making lists; how would an older game that became a sleeper hit ever overcome your system if it's apparently being judged against every list that has ever existed?

I just don't get how you can see something like Shin Megami Tensei V get featured on a single all-time list at #66 and say that is proof it is the 187th best game of all time, when your other yearly lists don't even put it in the top 10 just for 2021. And then have Dead Space lower despite being on several more all-time lists, many at a higher rank.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

http://www.acclaimedvideogames.com/games/27/

http://www.acclaimedvideogames.com/games/29/

Like, just look at the God of War games. Both made the same number of lists from 2023, in addition to GoW 2018 having others (not to mention GOW 2018 making the massive GQ poll while Ragnarok did not). Not only that, GoW 2018 has 14 EOY lists where it is ranked #1 compared to GOW: Ragnarok only topping three 2022 lists. Yet, somehow, you are using this data to claim Ragnarok is more acclaimed?
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Tue May 23, 2023 4:48 am Sorry, but the fact that a single list putting a 2023 game at #1 right now would make it #1 on your compiled list by a huge margin makes no sense at all. This is a horrible way of compiling data and you really should reconsider how you are weighing lists. Your entire formula appears weighted to give absurd advantages to modern releases. This entire site feels incredibly misleading as is.
Two points:
1) The above hypothetical--a game placing first on the only list that it had been eligible for--was brought up purely to illustrate a point. In practice, I ignore games that have only been eligible for a single list.
2) That's just how averages work. A game that had a single review 100% review on Metacritic would be first overall on their list, too (if they took games with a single review seriously, that is. Obviously, they don't).
BleuPanda wrote: Tue May 23, 2023 4:48 amRight now, it reads as if scores are simply calculated by what percentage of potential lists it could have been on; a list published this year should not benefit a more recent game at a lower rank than an older game at a higher rank.
It doesn't.
BleuPanda wrote: Tue May 23, 2023 4:48 amYour formula is not grading games on the same criteria.
Yes, it is.
BleuPanda wrote: Tue May 23, 2023 4:48 amAlso, due to this structure, I imagine recent games will rapidly decay in their ranking over time as soon as they miss a few lists.
Correct.
BleuPanda wrote: Tue May 23, 2023 4:48 amAlso, are you accommodating for the number of people involved in the list-making process? That GQ list polled about 300 people, correct? As such, I'm hoping it has a lot of weight compared to the others.
Lists are weighted based on a number of factors. Number of people involved is one of them, yes.
BleuPanda wrote: Tue May 23, 2023 4:48 amIt feels like games are punished more for missing lists than they benefit from making lists; how would an older game that became a sleeper hit ever overcome your system if it's apparently being judged against every list that has ever existed?
The system is biased in favour of newer lists precisely for this reason.
BleuPanda wrote: Tue May 23, 2023 4:48 amI just don't get how you can see something like Shin Megami Tensei V get featured on a single all-time list at #66 and say that is proof it is the 187th best game of all time, when your other yearly lists don't even put it in the top 10 just for 2021. And then have Dead Space lower despite being on several more all-time lists, many at a higher rank.
Many at a lower rank, too. One of its mentions is 340th, for example, which might not even be an advantage given that it's 189th overall. Anyway, because I've got so much more data for Dead Space, its position is going to be a lot more stable. Missing out on the next few all time lists probably won't impact its ranking all that much, where as if the same thing happens to SMT V, it's likely to drop pretty significantly. Again, that's just how averages work.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Tue May 23, 2023 6:32 pm http://www.acclaimedvideogames.com/games/27/

http://www.acclaimedvideogames.com/games/29/

Like, just look at the God of War games. Both made the same number of lists from 2023, in addition to GoW 2018 having others (not to mention GOW 2018 making the massive GQ poll while Ragnarok did not). Not only that, GoW 2018 has 14 EOY lists where it is ranked #1 compared to GOW: Ragnarok only topping three 2022 lists. Yet, somehow, you are using this data to claim Ragnarok is more acclaimed?
GOW 2018 didn't have a game like Elden Ring to compete with for its end of year lists, and both of GOW Ragnarok's mentions are extremely high (8th and 18th). Thus, the formula thinks that GOW Ragnarok's performance is marginally better, on average.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

So how does GoW Ragnarok's failure to make the GQ list impact its average?
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Wed May 24, 2023 5:30 am So how does GoW Ragnarok's failure to make the GQ list impact its average?
Had GQ had included it as the 101st game on their list then it would've ranked 18th overall.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

I guess I don't understand what you want this site to represent.

You compare it to Metacritic with "averages," but the whole point of Metacritic is to capture the immediate response to a work, where an average makes sense.

The point of Acclaimed Music, TSPDT, and Play That Game is to show what works have historically been treated as the most important in their medium, but your use of averages makes games almost irreversibly treated as less relevant with time.

What you are doing intentionally does neither. What is the benefit of using your site compared to Play That Game when it features obviously questionable listings like Horizon Forbidden West and Pokemon Legends Arceus in the all-time top 100 which you yourself admit will rapidly decay with time? There is a reason no other site like this takes "averages," because that is not how anyone measures historical impact. You've set it up where the work decays over time instead of the lists. I'm not trying to be rude, but I truly cannot imagine an average gamer who wants to see an aggregate listing of the most acclaimed video games sticking around with something like Pokemon Arceus in the top 100. Absolutely no one puts that game on that pedestal, and the fact that some random website that I just learned has *9* Twitter followers included it on a list doesn't change that.

Literally 9 twitter followers. I'm more relevant than the source that you are using to justify Pokemon Legends Arceus being listed as one of the most "acclaimed" games ever made. I just stumbled onto their Facebook page after typing this which sure has more followers but appears to only post clickbait videos. Even their main social media platform doesn't seem to find their reviews relevant enough to share, yet they are almost singlehandedly propping up a random game on your website.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Wed May 24, 2023 12:49 pm I guess I don't understand what you want this site to represent.

You compare it to Metacritic with "averages," but the whole point of Metacritic is to capture the immediate response to a work, where an average makes sense.

The point of Acclaimed Music, TSPDT, and Play That Game is to show what works have historically been treated as the most important in their medium, but your use of averages makes games almost irreversibly treated as less relevant with time.

What you are doing intentionally does neither. What is the benefit of using your site compared to Play That Game when it features obviously questionable listings like Horizon Forbidden West and Pokemon Legends Arceus in the all-time top 100 which you yourself admit will rapidly decay with time? There is a reason no other site like this takes "averages," because that is not how anyone measures historical impact. You've set it up where the work decays over time instead of the lists. I'm not trying to be rude, but I truly cannot imagine an average gamer who wants to see an aggregate listing of the most acclaimed video games sticking around with something like Pokemon Arceus in the top 100. Absolutely no one puts that game on that pedestal, and the fact that some random website that I just learned has *9* Twitter followers included it on a list doesn't change that.

Literally 9 twitter followers. I'm more relevant than the source that you are using to justify Pokemon Legends Arceus being listed as one of the most "acclaimed" games ever made. I just stumbled onto their Facebook page after typing this which sure has more followers but appears to only post clickbait videos. Even their main social media platform doesn't seem to find their reviews relevant enough to share, yet they are almost singlehandedly propping up a random game on your website.
A few things here:
1) Admitting that the rankings of recent games are bound to be a lot more volatile due to sampling error is not the same as admitting that the rankings of Horizon Forbidden West and Pokemon Legends: Arceus, specifically, will rapidly decay. I'm sure that the rankings of some recent and highly ranked games will decay, but I'm not making any predictions about those two games because I know nothing about them.
2) Using averages does not mean that a game's rank necessarily peaks when it is new. Sampling error is just as likely to underrate a game as it is to overrate it, so while some games' ranks will drop precipitously as more lists come in, others will rise just as fast. You're more likely to be looking at the overrated ones because overrated games are naturally a lot more visible than underrated ones, but the underrated ones still exist.
3) I said very clearly before that how the formula handles recent games is still a work in progress, so can we not use words like "justify" and "prove" in reference to the rankings that it spits out? You're implying a confidence, and a finality, that just isn't there. Don't mistake me telling you why it spits out certain results with me being happy with said results. There are still parts of the formula that I'm trying to get right.
4) Ultimately, the only standards that I'm trying to meet here are my own. I don't mind taking feedback and discussing the project, but the most important thing for me is making a list that I'm happy with. Hopefully in the process I make something that others think is useful too, but if not then that's not something that bothers me. If PlayThatGame is already doing what you think my list should be doing, then just use PTG. Or if you think PTG has its own weaknesses, what are they? I might try to address them.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

My problem with PTG is the opposite - it takes far too long for games to feel reasonably represented there. PTG is not reactive enough (gaming has only been around for a few decades, it's reasonable for new games to crack the top 100 after a couple years or maybe even immediately for the heavy hitters), while your current system is too reactive. I want to see a site that finds the right balance between these two extremes. Which, of all these types of websites, Acclaimed Music is the only one that I feel truly pulls that off.

From all I have gathered, it sounds like your methodology takes large influence from the number of lists a work has appeared on compared to the number it could have appeared on. As such, an older game receiving a high rank on a new list doesn't benefit all that much while a new game gets an astronomical boost. Like, let's say SVG was a highly notable site - even then, how does their single placement at #59 push it to #95 on what should be a consensus list? Meanwhile, Sonic the Hedgehog 2 appears at #6 on that very same list and is only #133 on your website. These rankings don't seem to reflect what SVG is saying - you are using their data to largely flip the relative ranking between these two works.

Which is why I want to circle back on 'decaying.' With your methodology, older games are simply at a disadvantage. I imagine a better way to tackle this is to instead have the lists decay over time - a #6 placement on the same list in 2022 should benefit Sonic the Hedgehog 2 more than a much lower place on the same list for something else, regardless of their relative age. Meanwhile, sure, Ocarina of Time shouldn't get too much of an advantage if it happened to take the top spot on a list from the year 2000. But that's more an issue with the age of the list.

Another question that I think gets overlooked with these formulas - where do you place your zeros? As in, when you enter a list with 100 entrants, how far below do you place every other game that could have been on that list? This is why I feel like you may be hurting older games more than you realize - there might just be a massive, invisible gulf piling up against them simply due to the volume of lists. If most of these lists only cover 100 games, that can create an insurmountable barrier for all but the biggest games. This is probably why new games overperform; there's an intrinsic gulf that gets bigger the further back you go.
User avatar
Henrik
Site Admin
Posts: 6439
Joined: Sun Feb 05, 2012 10:09 am
Location: Älvsjö, Stockholm, Sweden
Contact:

Re: Acclaimed Video Games site

Post by Henrik »

Gratulations Moeboid to a great website!

I think it could be wise for you to include a sample size correction. But it also seems like you have put a lot of great thought behind your algorithm! I, for one, can say that it is really not easy to find a balanced way to compare new and old stuff, given that the sources are so different (mostly eoy vs mostly alltime lists).

(But it’s fun! 😜)
Everyone you meet fights a battle you know nothing about. Be kind. Always.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Henrik wrote: Wed May 24, 2023 9:03 pm Gratulations Moeboid to a great website!

I think it could be wise for you to include a sample size correction. But it also seems like you have put a lot of great thought behind your algorithm! I, for one, can say that it is really not easy to find a balanced way to compare new and old stuff, given that the sources are so different (mostly eoy vs mostly alltime lists).

(But it’s fun! 😜)
Thank you!

I'm almost certainly going to reintroduce a sample size correction at some point, it's just a matter of deciding how to do it, and how much to do it. I really struggle with arbitrary decisions like that. I wish there was a clear right answer, but at the end of the day it's just a taste thing. The placing of recent games being a bit rough actually doesn't bother me particularly, but clearly not everyone thinks that way.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Wed May 24, 2023 7:28 pmFrom all I have gathered, it sounds like your methodology takes large influence from the number of lists a work has appeared on compared to the number it could have appeared on. As such, an older game receiving a high rank on a new list doesn't benefit all that much while a new game gets an astronomical boost.
The number of lists a game has appeared/not appeared on is not directly relevant. The way it works by tracking how often a games have been ranked ahead/behind each other.
BleuPanda wrote: Wed May 24, 2023 7:28 pmLike, let's say SVG was a highly notable site - even then, how does their single placement at #59 push it to #95 on what should be a consensus list?
Most other publications haven't weighed in on where Pokemon Legends: Arceus should be in the all time rankings, so I don't know what the consensus is.
BleuPanda wrote: Wed May 24, 2023 7:28 pmMeanwhile, Sonic the Hedgehog 2 appears at #6 on that very same list and is only #133 on your website. These rankings don't seem to reflect what SVG is saying - you are using their data to largely flip the relative ranking between these two works.
I have a lot more data for Sonic the Hedgehog 2. The consensus is basically known.
BleuPanda wrote: Wed May 24, 2023 7:28 pmWhich is why I want to circle back on 'decaying.' With your methodology, older games are simply at a disadvantage.
No, they're not. They're less prone to sampling error. That's all.
BleuPanda wrote: Wed May 24, 2023 7:28 pmI imagine a better way to tackle this is to instead have the lists decay over time - a #6 placement on the same list in 2022 should benefit Sonic the Hedgehog 2 more than a much lower place on the same list for something else, regardless of their relative age.
1) The lists do decay over time.
2) The age of the game has nothing to do with how much it benefits from its position, so Sonic the Hedgehog 2's position at 6th is much more advantageous to it than Pokemon Legends: Arceus' placement at 59th. The difference is that I have a ton of data telling me that Sonic the Hedgehog 2 shouldn't be at 6th, where as the data for Pokemon Legends: Arceus is rather limited.
BleuPanda wrote: Wed May 24, 2023 7:28 pmMeanwhile, sure, Ocarina of Time shouldn't get too much of an advantage if it happened to take the top spot on a list from the year 2000. But that's more an issue with the age of the list.
The formula already works like this.
BleuPanda wrote: Wed May 24, 2023 7:28 pmAnother question that I think gets overlooked with these formulas - where do you place your zeros? As in, when you enter a list with 100 entrants, how far below do you place every other game that could have been on that list? This is why I feel like you may be hurting older games more than you realize - there might just be a massive, invisible gulf piling up against them simply due to the volume of lists. If most of these lists only cover 100 games, that can create an insurmountable barrier for all but the biggest games. This is probably why new games overperform; there's an intrinsic gulf that gets bigger the further back you go.
Again, it works by tracking how often games have been ranked ahead/behind each other. If a game is 59th then it is marked as having placed ahead of every game in the database except the 58 that are ahead of it (also excepting any that hadn't been released at the time of the list's publication). There is no pre set "distance" between games that are mentioned on a list, and games that didn't make it.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

If this all works as you say, then I really think the solution is to just hold off on adding new games until a certain all-time list threshold has been met since their release. There's a reason Metacritic waits until at least 4 reviews have been added, and it's the most recent years that make your list feel off in a way that the rest of the data does not.

So, just so I am fully understanding this process:
Based on the SVG list, Pokemon Legends: Arceus beats out every single game but the 58 above it
Meanwhile, based on the two GamingBolt lists, GQ, Parade, and USA Today lists released in the same time period (and any other from 2022 and 2023), it is marked as losing to every game that made those lists?

If that is how it works, that is ostensibly fine, but you really should be waiting for more data points before declaring something like Pokemon Legends: Arceus as one of the 100 most acclaimed games of all time. You shouldn't be making statements like "I don't know what the consensus is" when running a website claiming to present consensus data. Hold off until you actually can make that claim, even if it means having to wait a couple years before new games are implemented. The fact that you are not familiar with this game and its general reception has me convinced you're not realizing just how bad its presence so high up makes the rest of your data look (literally the second highest Pokemon game?), when the real issue appears to be that you simply listed it too early.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Thu May 25, 2023 2:37 pm If this all works as you say, then I really think the solution is to just hold off on adding new games until a certain all-time list threshold has been met since their release. There's a reason Metacritic waits until at least 4 reviews have been added, and it's the most recent years that make your list feel off in a way that the rest of the data does not.
Current plan is to add games along with the end of year lists, and I can't see myself changing that, sorry. Recent games will be ranked more conservatively once I find an algorithm that does that to my satisfaction, but if it still doesn't seem right to you then you'll just have to deal with it. I don't think it's too much to ask to just take the rankings of games from the past year or so with a grain of salt. If the rankings of games prior to that seem correct then the same will be true for recent games eventually as well.
BleuPanda wrote: Thu May 25, 2023 2:37 pmSo, just so I am fully understanding this process:
Based on the SVG list, Pokemon Legends: Arceus beats out every single game but the 58 above it
Meanwhile, based on the two GamingBolt lists, GQ, Parade, and USA Today lists released in the same time period (and any other from 2022 and 2023), it is marked as losing to every game that made those lists?
That's how it works, yes.
BleuPanda wrote: Thu May 25, 2023 2:37 pmIf that is how it works, that is ostensibly fine, but you really should be waiting for more data points before declaring something like Pokemon Legends: Arceus as one of the 100 most acclaimed games of all time. You shouldn't be making statements like "I don't know what the consensus is" when running a website claiming to present consensus data. Hold off until you actually can make that claim, even if it means having to wait a couple years before new games are implemented. The fact that you are not familiar with this game and its general reception has me convinced you're not realizing just how bad its presence so high up makes the rest of your data look (literally the second highest Pokemon game?), when the real issue appears to be that you simply listed it too early.
1) I can only represent the consensus to the extent that it is known. If people don't understand that, then that's their problem. I don't think I'm making any false claims, especially if that false claim amounts to being honest about when data is poor.
2) If picking PLA is so bad then shouldn't you be more focused on the publication that picked it, rather than my website for including their list as part of the consensus? Besides, if you're confident that it's a bad pick, and that no one else will pick it, then just sit back and wait for its ranking to go down. It's a temporary problem.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Rankings have been updated again. The new rankings reflect the addition of a few new all time lists, updated publication weightings, and a smoothing algorithm that places recent games more conservatively. I think you will like the new rankings a lot better, BleuPanda. Elden Ring's position has dropped to 9th, and every other game from 2022 is now out of the top 100. The amount of smoothing is ultimately arbitrary, so feel free to give feedback.
http://www.acclaimedvideogames.com/

Edit: Tweaked a few things and God of War: Ragnarok is now back in the top 100 (73rd).
Last edited by Moeboid on Thu Jun 08, 2023 11:00 am, edited 1 time in total.
Nassim
Full of Fire
Posts: 2793
Joined: Fri Feb 10, 2012 2:35 pm
Location: Lille (France)

Re: Acclaimed Video Games site

Post by Nassim »

I like seeing Hades at #10, I think it might actually be THAT good.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Nassim wrote: Thu Jun 08, 2023 9:48 am I like seeing Hades at #10, I think it might actually be THAT good.
I actually wasn't crazy about it, but a lot of people whose taste I respect love it, so I guess 10th is fair. My favourite Supergiant game is actually Bastion, which is criminally underrated at 343rd. Hades is better mechanically, but it never held my attention the way Bastion did. I loved the music, the visuals, the way the levels construct themselves as you move, the clever use of narration... And it wasn't just for an aesthetic, all the elements came together to tell a story and deliver a message that I actually thought was worthwhile. Absolutely loved it. Probably top 20 for me.
Nassim
Full of Fire
Posts: 2793
Joined: Fri Feb 10, 2012 2:35 pm
Location: Lille (France)

Re: Acclaimed Video Games site

Post by Nassim »

Moeboid wrote: Thu Jun 08, 2023 11:17 am
Nassim wrote: Thu Jun 08, 2023 9:48 am I like seeing Hades at #10, I think it might actually be THAT good.
I actually wasn't crazy about it, but a lot of people whose taste I respect love it, so I guess 10th is fair. My favourite Supergiant game is actually Bastion, which is criminally underrated at 343rd. Hades is better mechanically, but it never held my attention the way Bastion did. I loved the music, the visuals, the way the levels construct themselves as you move, the clever use of narration... And it wasn't just for an aesthetic, all the elements came together to tell a story and deliver a message that I actually thought was worthwhile. Absolutely loved it. Probably top 20 for me.
I had Hades at 23 and Bastion at 107 the last time I did a ranked list. I love all the games in my top 200 to some extend but I think Hades is an improvement on all fronts.
I tried Pyre but the gameplay wasn't for me (seems pretty great on the narrative side though) and I never tried Transistor
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Nassim wrote: Thu Jun 08, 2023 5:05 pm I had Hades at 23 and Bastion at 107 the last time I did a ranked list. I love all the games in my top 200 to some extend but I think Hades is an improvement on all fronts.
I tried Pyre but the gameplay wasn't for me (seems pretty great on the narrative side though) and I never tried Transistor
Yeah I'm clearly in the minority with Hades. When I'm in the mood for that style of game, I play Dead Cells, though that game has had so many updates since I last played it that I can't guarantee I'd rate it as highly now.

Speaking of games that I don't rate as highly as others, The Witcher III is now in second place. I'm not entirely sure how that happened since none of the changes affected it directly, but as far as I can tell it's all working, so I guess there was just some indirect advantage somewhere. The games between 2nd and 5th are all quite close, so it doesn't take much to shift their rankings.

Is this top 200 of yours online somewhere? I'd be curious to see what else is on it.
Nassim
Full of Fire
Posts: 2793
Joined: Fri Feb 10, 2012 2:35 pm
Location: Lille (France)

Re: Acclaimed Video Games site

Post by Nassim »

Moeboid wrote: Fri Jun 09, 2023 4:47 am
Nassim wrote: Thu Jun 08, 2023 5:05 pm I had Hades at 23 and Bastion at 107 the last time I did a ranked list. I love all the games in my top 200 to some extend but I think Hades is an improvement on all fronts.
I tried Pyre but the gameplay wasn't for me (seems pretty great on the narrative side though) and I never tried Transistor
Yeah I'm clearly in the minority with Hades. When I'm in the mood for that style of game, I play Dead Cells, though that game has had so many updates since I last played it that I can't guarantee I'd rate it as highly now.

Speaking of games that I don't rate as highly as others, The Witcher III is now in second place. I'm not entirely sure how that happened since none of the changes affected it directly, but as far as I can tell it's all working, so I guess there was just some indirect advantage somewhere. The games between 2nd and 5th are all quite close, so it doesn't take much to shift their rankings.

Is this top 200 of yours online somewhere? I'd be curious to see what else is on it.
We had a poll on the forum 3 years back viewtopic.php?p=142601&hilit=Future#p142601

Still haven't played The Witcher 3 but I have it ready on my SteamDeck
ManPerson
Let's Get It On
Posts: 157
Joined: Fri Oct 01, 2021 8:21 pm

Re: Acclaimed Video Games site

Post by ManPerson »

When do you think the recency bias will be fully adjusted for?

Great job, btw.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

ManPerson wrote: Fri Jun 09, 2023 10:51 am When do you think the recency bias will be fully adjusted for?

Great job, btw.
Thanks!

If you're asking how long it will take for the rankings of recent games to settle, then it's not a matter of time as much as it's a matter of getting my hands on more all time lists. If a lot of major publications update their all time lists soon, then the rankings of current games could settle very quickly, but when these lists actually come out is anyone's guess.

However, if what you're asking is when I'm planning on implementing some sort of compensation so that the rankings of recent games are less volatile, then I have actually already done so. This is the main reason why the rankings of so many recent games dropped after the last update. At the moment, the amount of compensation is pretty negligible, but I may increase it in the future if people think that would be better.

Also, just so that it's clear, the formula is in no way biased towards recent games. The rankings of recent games aren't artificially any higher or lower than they should be, they're just imprecise due to how few lists there are to base them on. This imprecision is just as likely to underrate a game as it is to overrate it, but because the overrated games are a lot more visible, the overall list appears to be consistently overrating them. Don't let that feel you though, the underrated games definitely exist, they're just outside the top 500.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Nassim wrote: Fri Jun 09, 2023 6:05 am We had a poll on the forum 3 years back viewtopic.php?p=142601&hilit=Future#p142601

Still haven't played The Witcher 3 but I have it ready on my SteamDeck
Very interesting! Thanks. I see The Witcher III didn't even make the AM forum's top 100. That is correct, in my opinion.
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

I've been thinking a lot about the 'recency bias' this afternoon, which I think doesn't give the full picture of what's happening.

Let's return to Arceus, just as a point of reference:
Pokemon Legends: Arceus was eligible for all lists since its release and appeared on one at #59
For that list alone, I think we can say Arceus scored a single point against every game in existence other than the 58 above it
On every other list published since its release, Arceus loses to anything featured on those lists

So, just looking at Arceus:
It beats out every single game in existence that has not been mentioned on any list since 2022 on, 1-0
Elden Ring, which appeared on the same list at a higher spot and also on 6 more, beats Arceus 7-0
So, every old game now needs to appear on a new list to 'tie' Arceus 1-1, and appear on at least two new lists to outright beat it 2-1 (assuming Arceus does not appear on any more)

So I think all of this has to be considered in relation to the total number of games listed on all-time lists since each game was released.

Pokemon Legends: Arceus is only in competition with the ~200 unique games that have featured on all-time lists since its release. Meanwhile, Tetris is facing off against presumably 1000+ unique games. It would be literally impossible for Pokemon Legends: Arceus to fall out of the top 500 until 500+ unique games appear on new lists. So, the moment a new game appears on a single all-time list, it has an insurmountable buffer until that 500+ unique game requirement is met. It is literally impossible for a game from the last two years to appear on a single all-time list, no matter the source, and not be in the top 500 (unless that list is massive, I guess).


So, I think the big problem here is that Pokemon Legends: Arceus has thousands of direct matchups that read as 1-0 victories. A lot of weight is being given to those 1-0 pairs; but what if that single ranking is a fluke? The only way this fluke will be righted is for dozens upon dozens of new all-time lists to be produced, all featuring a mass assortment of games - and, again, note that this direct matchup system means every classic game has to appear on two separate lists before it can beat out Arceus. This is why, when you look at the website, it goes from Arceus at 150 to nothing at all beyond that for 2022. 2021 is the same way, SMT5 at 154 and then nothing from 2021 beyond that. Their ranking is less a measure of how many games they beat out and more a measure of how many unique games have been featured on all-time lists since their release, of which they lucked into being included a single time. When most all-time lists consist of a core group of games, it can take an extreme amount of time for outliers to fall into place.

So, I think this all comes down to the assumption that these 1-0 pairings are anything more than statistical noise. Again, this is all an issue of weighing; why is this single ranking from SVG able to hold so much power over every other point of data you have collected? It's not an issue of gathering more lists, because this problem will simply reset every single year.

While I think this direct matchup system is interesting, it is clear from the results that there simply aren't enough all-time video game lists being produced every year to make it work while also trying to implement recent releases, especially if 1-0 pairings are going to be used as evidence of relative strength. So it's not so much a recency bias as an issue with sample sizes. Older games have large sample sizes, new games have very small sample sizes, but these sample sizes are being treated with equal weight.

This is also why I have an easier time believing your Elden Ring ranking; an implicit 7-0 ranking against most games in existence reads a lot more clearly than a bunch of 1-0s, as those 7 lists back up each other. SVG, as is, stands completely alone in its praise of Arceus. I think all the problems with your formula disappear if you simply require a game appear on a certain number of lists before trying to implement it. You may not be aware that you are doing so, but you are making gigantic extrapolations based on single placements of recent games. You asked me earlier to blame SVG for ranking Arceus, but from everything I have learned about statistics, a system like this should easily be able to look past their sole ranking as noise until backed up by other data. It's as simple as saying "There is not enough data to judge this game's relative placement until it appears on at least three
all-time lists." Because there really isn't enough data, and this should be a really simple fix. Even game #500 on your list has managed to place on 5 all-time lists, so 3 really isn't that much to hold out for if a game truly deserves to be listed among the best of the best.

And, just to be clear, I am saying all this because I want your website to succeed.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

BleuPanda wrote: Fri Jun 09, 2023 8:15 pm So it's not so much a recency bias as an issue with sample sizes.
Yes. Some of your assumptions about how the formula works are incorrect, but what you essentially have a problem with is how it ranks games in the case of bad data. Waiting until more lists come in is one way to deal with this, but not the only way...
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

I have updated the website again, this time with a formula that I believe does a much better job of handling small sample sizes. It does this by focusing on areas where sample sizes are not small, which for newer games is the end of year lists. By referencing these end of year lists, it can better predict what the all time data should look like. I'm sure it's not perfect, but I think it's a lot better than what it was before.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Another update. The last one was an improvement, but there were still a few odd rankings. Now those are almost completely gone. I'm extremely happy with it now, and for the first time I'm feeling like it's just a matter of refining things, rather than making drastic changes. Curious to hear what you all think.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

Another major update. The website now pulls metadata from IGDB, including images, and the formula has been massively improved. Interestingly, the new formula basically agrees with what BleuPanda (and others) thought the previous formula was getting wrong: Elden Ring has dropped from 2nd to 18th, Pokemon Legends: Arceus has dropped form 95th to 422nd, Horizon: Forbidden West has dropped from 56th to 317th, God of War: Ragnarok has dropped from 27th to 217th, and Shin Megami Tensei V has dropped from 187th to 430th. These moves are partly due to more lists being added, but they're mostly due to the formula just being a lot more rigorous now.
The previous formula was, understandably, accused of having a sort of recency bias, but that really wasn't the issue. The problem was that it assumed all matchups to be independent, which they clearly aren't. A single all time list will affect literally thousands of matchups, and these would then reinforce each other as the formula tried to cross reference everything. The matchups that it had more data for were weighted more heavily, but it wasn't enough to outweigh the thousands of matchups affected by the all time lists. My first attempts at addressing this involved arbitrarily excluding certain matchups, but this new method gets more to the heart of the issue: it guarantees that each list only gets counted once, regardless of how many matchups it affects.
Something that I'm really happy about is that it's no longer necessary to make an all time list in order to place inside the top 500. It really bothered me before that Immortality wasn't in the top 500, because it was clear from the 2022 end of year data that it should've been there (this is why I was so adamant that the issue wasn't a recency bias). It's since placed on an all time list anyway, but even if that list didn't exist, the new formula is still smart enough to see that it deserves to be in the top 500 (without its all time mention, Immortality be ranked 460th, which I think is fair considering how few opportunities it's had to place on an all time list).
Looking forward, the next thing I want to do is fix up the website. It still doesn't display very well on mobile, it's not fetching the developer data from IGDB properly, and I want to turn the home page into a sort of news page. After that I'll be looking to include the platform specific lists. Lots of work, but it's all looking very positive!
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

Took a glance and the rankings definitely look a lot more believable now.
ManPerson
Let's Get It On
Posts: 157
Joined: Fri Oct 01, 2021 8:21 pm

Re: Acclaimed Video Games site

Post by ManPerson »

Agreed, nice work on improving the formula Moeboid.
User avatar
whuntva
Keep On Movin'
Posts: 1838
Joined: Wed Sep 19, 2012 11:46 pm

Re: Acclaimed Video Games site

Post by whuntva »

As someone who did follow PlayThatGame for a while, I am not sure which site would have the "better" formula. It is all subjective of course, but I want one with some precedent to pass off as the big one.

Though I will say Breath of the Wild at #1 feels wrong. It's too recent.
" Ah, yes! Our meager restitution"
User avatar
BleuPanda
Higher Ground
Posts: 4717
Joined: Thu Aug 30, 2012 1:20 am
Location: Urbana, IL

Re: Acclaimed Video Games site

Post by BleuPanda »

whuntva wrote: Tue Aug 15, 2023 1:26 am
Though I will say Breath of the Wild at #1 feels wrong. It's too recent.
Oh, great arbiter of worthiness, please let us know how many years must pass before we are allowed to praise a work.

It's already embarrassing when people clearly latch onto the supposed canon of film and popular music without actually analyzing their qualitative worth, but to try and apply that same ideology to a medium that is still essentially in its youth is so incredibly toxic. Who exactly does it help when you put games like Ocarina of Time on a pedestal? To decide the entire medium of video games peaked in its first couple decades?

The NES came out in 1985. Except for a small handful, most of the games that have ever been in serious discussion as among the greatest ever are on the SNES or later. That's about 30 years of serious analysis, of which BotW has existed for 20%. That's a significant chunk of time.

I don't think anyone serious about gaming would cite Play That Game as a realistic picture of the state of gaming. Anyone paying attention to the actual discussions around the medium knows the conversation has moved on to games like Breath of the Wild and Elden Ring. So, sure, BotW at #1 may look off, but certainly not as off as it only being #39 on PTG (or, even more absurd, nothing from 2019 even being in the top 250 even after end of decade lists).

Like, I struggle to think of a game that was massively acclaimed that did not sustain that acclaim. Among the few are Heavy Rain and Bioshock Infinite (and even then, the backlash for those happened within a few months of release). So, I think you are wrong to accuse people of jumping the gun when people have actually been pretty consistent about the consensus on gaming since the beginning. I don't see how you can look at the Metacritic score for BotW and also the sheer number of lists that it has topped and think it's somehow unworthy of that spot.
Moeboid
Strange Fruit
Posts: 52
Joined: Sun Jan 06, 2013 12:10 am

Re: Acclaimed Video Games site

Post by Moeboid »

whuntva wrote: Tue Aug 15, 2023 1:26 am As someone who did follow PlayThatGame for a while, I am not sure which site would have the "better" formula. It is all subjective of course, but I want one with some precedent to pass off as the big one.

Though I will say Breath of the Wild at #1 feels wrong. It's too recent.
I make no secret of the fact that my website is a work in progress, and while that's the case I'm definitely not expecting anyone to take it seriously. I've made drastic changes to the formula no less than four times since launching it less than six months ago, and I'll have to stop doing that before I can claim to be an authoritative voice on the most acclaimed video games. How can people trust a list so unstable? I'm pretty happy with where the formula is at now, and since the last update feedback has also been a lot more positive, but I'm still not at a point where I can guarantee that things won't change going forward.

One thing that I will guarantee, however, is that Breath of the Wild deserves to be in first place. My formula works by tracking how often games have been ranked ahead/behind each other, and by that metric, Breath of the Wild has an enormous lead on literally every other game in the database. Here are Breath of the Wild's five worst win rates, and the games that they are against:
60.62% Tetris
62.04% DOOM (1993)
62.65% Super Mario 64
65.15% Super Mario World
66.99% The Last of Us
For comparison, here's the same data for The Last of Us (currently in second):
33.01% The Legend of Zelda: Breath of the Wild
49.45% Half-Life 2
53.99% Tetris
54.18% Super Mario 64
61.47% Super Mario Bros. 3
And for Tetris (currently third):
39.38% The Legend of Zelda: Breath of the Wild
45.67% Elden Ring
46.01% The Last of Us
55.45% Half-Life 2
55.77% The Legend of Zelda: Ocarina of Time
This is just a small part of what the formula takes into account, but it's enough to see just how strong Breath of the Wild's results are. In order to dethrone it, I would have to either abandon the above metric altogether (which is what allows me to fairly aggregate lists covering vastly different time periods), or introduce such a strong bias in favour of old games that recent games would barely be represented. And this isn't case of bad data, either; Breath of the Wild has been eligible for more than 1/3 of the all time lists that I have in my database, as well as a huge number of end of decade lists. It may seem strange to see such a recent game in first place--especially if you're used to music and film lists--but that's just what the data says.
Post Reply

Return to “General Discussion”