Mitch's 2012 wood coaster poll

Jeff said:

Because that's what statistical significance is. That is science. You seem to get hung up on that every time this comes up.

Not really, I get hung up on the idea that creating a poll ranking favorite roller coasters works best based entirely on scientific method.

Flat out - it can't be an accurate list of the most popular coasters if some coasters are excluded. It just can't.

It can be scientific, but it's not complete. Which, in turn, makes it inaccurate - it's just a list of the most popular coasters that got enough votes to meet the criteria of being statistically significant.

The opposing argument would be that using the statistically insignificant makes a poll inaccurate as well. But extrapolation isn't exactly voodoo. And, given the task at hand, I'll always feel better about making assumptions based the available info for the sake of inclusion than ignoring the available info for the sake of scientific accuracy.

Isn't the overarching point to create a list that ranks all the roller coasters of the world in order of popularity?

The second you exclude a ride from the list, you're not doing that. To me, you miss the point of what the poll tries to achieve by getting stuck in the methods used to reach that point.

----

As a total aside:

I'd be interested in seeing the list in the 3 other possible states:

1. With all voters included and no cutoff on the number of coasters votes. (the free for all)

2. With the number of coaster votes needed still in place, but all voters included.

3. With the voter cutoff in place, but all coasters included.

How different would those three lists look?

Lord Gonchar said:
Isn't the overarching point to create a list that ranks all the roller coasters of the world in order of popularity?

Isn't the overarching point of the polls to generate discussions like the ones in this thread? You take a subjective determination like which roller coaster is "best" and apply objective numbers, averages and ordered rankings to that determination to make it appear to be more objective. So now, in addition to being able to discuss the merits (or lack thereof) of steel/wood, airtime, lateral forces, etc. in terms of what makes a coaster good, there also can be a discussion about what objective standards are best at making that subjective determination appear less subjective.

There is a ride in Asia with exactly *one* rider in Mitch's pool (eliminated for not having enough riders, but...). Let's just say that the individual decided it was the best wooden coaster ever....or that they wanted to give the park some publicity. Without any lower bound, that would be the highest-ranked wooden coaster in the specified poll. Let's say that rider hadn't ridden ANY other wooden coasters...

Not arguing whether that's right or wrong, only pointing out how easily it could be skewed if there are no real criteria...

The overarching Point of these polls is to make Enthusiasts feel better about rides they've ridden, whilst giving Parks something to market. Today was the first time I saw Mitch's poll 'legitimized' by a park. SFGAdv tweeted results since El Toro won.

rollergator said:

There is a ride in Asia with exactly *one* rider in Mitch's pool (eliminated for not having enough riders, but...). Let's just say that the individual decided it was the best wooden coaster ever....or that they wanted to give the park some publicity. Without any lower bound, that would be the highest-ranked wooden coaster in the specified poll.

But what if it really is the best coaster in the world. Or even a very good one. It gets ignored because of the sample. The list is immediately wrong in spite of having some data to work with.

That's where I think Mitch's system of comparisons makes sense. You can certainly infer a lot from a big ranked list.

Again, given the task at hand, it will always make more sense to me to include some degree of conclusion and/or reasoning of the data you have to fill in the blanks instead of just ignoring the info.

Is one opinion enough to form a consensus? Obviously not. But if you're looking for opinions on the quality of roller coasters, a handful is better than none.

Let's say that rider hadn't ridden ANY other wooden coasters...

Well then there'd be nothing to extrapolate from and no comparisons to make inferences from. At that point even I would concede that a determination of ranking cannot be made.

Not arguing whether that's right or wrong, only pointing out how easily it could be skewed if there are no real criteria...

Of course it could be, but only if you look only at the numbers. Which is exactly what I'm suggesting you don't do. The number form the clay. It's up to the presenter to mold it into something useful.

(Discussions like this make me feel like I'm suddenly on The Big Bang Theory discssuing some mundane thing with Sheldon.)

Oh good lord.

A lot of conversations here at CoasterBuzz are like that, Gonch. That's why it's so much fun.

Lord Gonchar said:
But what if it really is the best coaster in the world. Or even a very good one. It gets ignored because of the sample. The list is immediately wrong in spite of having some data to work with.

Yes, that's the way science works. I don't make up those rules, but thousands of years of people who are very smart with numbers generally agree that this is the case. The goal is not truth, it's a reasonable degree of accuracy, which can only be achieved with a sample size that can generally account for a reasonable margin of error. If one person popping Pez (from a Darth Vader head dispenser) is cured of cancer, we don't jump to the conclusion that Pez cures cancer.

Of course not...because it's Darth Vader that cured the cancer.

Jeff said:

If one person popping Pez (from a Darth Vader head dispenser) is cured of cancer, we don't jump to the conclusion that Pez cures cancer.

Yeah, you just typed that. Roller coasters. Cancer.

And it's a horrible analogy because we know Pez doesn't cure cancer. But we don't know that any given coaster might not be the best (there's an opening for someone to make a joke about traditionally hated coasters - you're welcome). If you didn't know whether or not something cured cancer, you'd certainly add it to a list of things that could and give it further consideration.

The goal is not truth

Clearly.

it's a reasonable degree of accuracy

Ok. But a list of the most popular roller coasters is immediately inaccurate if one isn't included. You stop chasing accuracy long before I would have.

...which can only be achieved with a sample size that can generally account for a reasonable margin of error.

Even though the subject is benign enough and the gaps small enough that one can reasonably fill them in with little repercussions or errors? Again, this isn't curing cancer, it's making a list of roller coasters.

Yes, that's the way science works. I don't make up those rules, but thousands of years of people who are very smart with numbers generally agree that this is the case.

Agreed. All I've been trying to tell you for years now is that science and people who are very smart with numbers suck at making decent, complete, inclusive roller coaster popularity lists.

I still think you're stuck on the methodology and missing the greater point. But I'm realizing that's just the result of us seeing the goal differently.

I don't know why I get pulled into this every year. It's just the way you seem so sure that 'scientifically valid' is the only way to do it when, to me, it misses making what I'd call 'a good list' by a bit.

Jeff said:

I suppose I left out my previous frustrations with that poll though, that being sample size. Remember the years where some obscure coaster would be #1 because four people who have been on almost all coasters would influence the results? That I don't agree with, because it's not statistically significant. That's why I ditch any ride that hasn't seen at least half of the riders on it, and any rider that has been on less than half of the average number of rides.

Damn your logic, Spock!

Perhaps I can articulate the heart of the issue:

The math can tell you what margin of error you will have for a given sample space. This is not up for debate.

The math cannot tell you what margin of error is acceptable for your application. This is a judgment call.

Leave it to someone else to articulate my point in a way that haven't been able to in years. Well done, Andy.

My mind has just been blown that Andy was able to articulate the core issue so succinctly! Good job my friend.

Mitch's steel poll results are posted.

http://www.ushsho.com/steelrollercoasterpollresults2012.htm

Wow, not sure how I got lucky enough to stumble onto this thread, but when I saw all the talk about "evolution" and "big bangs" I got really excited and curious. I was hoping someone might be able to answer that age old question, "how did we get here??" And while they're at it, they could also explain how a ride like Millennium Force gets left off of that top 12 list ?? Does it have anything to do with the science/numbers/math thing that Gonch and Jeff were debating ? Because most of that was so far over my head it looked like it was written in Japanese.

Is it all just yesterday's news that I failed to keep up with ?

Or is it that according to the experts that took part in the poll, MF just flat out doesn't measure up to the rest - which I guess brings us right back to square one as to the accuracy of the poll itself ?

It's nothing more than my opinion (really, all of this poll stuff is just personal opinion) but the Mitch poll has always seemed "elitist" to me. As such, I'm not surprised a ride like MF falls out of the top 12.

Does Mitch work for Intamin (8/12)? ;)

In the words of P. Diddy - "I'm shocked at this information!" Oz-Iris, arguably the most blah inverted ranked higher than Dragon Challenge? Egad!

MF isn't in my top 10.