Mitch's Poll - Wood Coaster Results

Jeff said:
But with these sample sizes, it's simply not.

The only way to increase the sample size is to either get more people to go to Korea or eliminate certain coasters based on ridership. Mitch already does based on HIS (probably somewhat, just a little arbitrary) number but if the threshold to get on the list is too low for you just pick a (probably somewhat, just a little arbitrary) number for yourself and dump those rides from the results.

Charles Nungester said:
Two years from now, Voyage will be right back where it belongs. TPR is the same group that got served Beer for free at one park and ranked it over voyage even though it craws thru the turns.No Offense to T Express. I need a much bigger diverse sample than 6 of 9 verses over 250 people..The good ones and well maintained ones keep their status. Even though on this list. 50th or in some cases 100th could be someones personal favorites.

I was on the TPR Korea tour last year and ranked Eagle Fortress my number 1 on the steel coaster poll . Strangely, I don't remember being served "Beer for free" (and I don't drink beer to start with). Since I was on a TPR tour, that means my opinion is automatically not good? I'm going back to Everland next march on my own, will that make my opinion "valid", since I won't be "influenced" by a group?

The ECC is doing Japan and South Korea next year, so if T Express and Eagle Fortress keep their spots, will the blame start toward them?

I can't believe we don't have a certified numbers dork around here who can give us an actual scientific answer.

You rang?

Unfortunately, there isn't a "single answer" to the question, because it depends on the sources of error. There's a pretty decent (but not terribly accessible) paper on the topic here:

http://www.stat.uiowa.edu/techrep/tr303.pdf

At the end of the day, the poll can't ever be used for a rigorous statistical test/sample, because it has a selection bias---in layman's terms, only people who give a rat's patootie about relative coaster rankings ever respond to the poll. Folks with axes to grind are a lot more likely to take the poll than "regular enthusiasts" (if such a thing exists.)

The real question is "what if 14 people decided to vote T-Express as the #1 in the world just because the voting system would allow for it?" If such a small sampling could end up voting a relatively unridden (North Americans anyhow) and pushing it to #1, what''s stopping them?

I'm sure T Express is a great ride, but let's not throw away the fact that a small number of jokers could have done it just for shiggles.

There you have it.

You guys, or actually, just Matt, are unfairly giving Mike a hard time. He's not making any suggestion about whether or not a coaster is at Cedar Point or whatever. Heck, with 300+ votes for Mean Streak, it's clear to me that a crappy coaster is a crappy coaster, and numbers don't help it.

The problem is that the claim that the poll "separates quantity from quality" is false. Quantity does in fact play a huge role, because high rankings from a few people who have been on a butt load of coasters creates the bias that Brian is talking about.

Mitch explains that the cut off is 1% of ballots have to include the ride to count it. That's seven votes! Seven whack job, patch-wearing schmucks who went to Korea become taste-makers for thousands of regular Joe Six-Packs. That skews the bias of the poll because it no longer represents some theoretical average, it only represents the niche.

I've done a lot of marketing related study work with various consultants, and generally we wouldn't work with any less than 300 responses. And even then, that sample had to be representative of various criteria that matched our overall audience (in our case, things like location and various socioeconomic indicators). Similarly, I know that in the papers Stephanie had published back in the day, she needed at least that many [insert critter here] to pass the scrutiny of peer review.

^But in this case, the seven whack-job patch-wearing schmucks really aren't taste-makers for thousands of regular Joe Six-Packs, they're taste-makers for the couple dozen weirdos that have the time and resources to travel all over the world with no better agenda than riding roller coasters...or, as someone stated earlier, if I'm travelling around the world, I've probably got SOME other stuff to see/do...like museums, temples, or natural wonders. Not that there's anything wrong with that, either. A suitable quote from Ghandi for those who take these things WAY too seriously - "Honest differences are often a healthy sign of progress." ;)

Fair enough, though the very fact that this comes up year after year tells me that "it" does attempt to speak for a broader audience, whether that's by design or not.

But still no one is suggesting a better way to do it.

The point remains that the poll is voluntary and that of those who volunteered, only 14 had ridden T Express and they pretty unanimously enjoyed it.

Just because the fix could be in, doesn't mean it is. I still say that given the importance of the subject and the data obtained that this is a satisfactory conclusion.

Jeff:
Quantity does in fact play a huge role, because high rankings from a few people who have been on a butt load of coasters creates the bias that Brian is talking about.

1. I'm not quite sure that's the bias Brian spoke of.

2. I think there should be bias toward people who've ridden more coasters - they create more comparisons for the data set and they're arguably more qualified to compare due to greater coaster 'experience' (for lack of a better term).

That's the beauty of Mitch's poll, the more coasters on your ballot the more you affect the outcome. The tradeoff is that the less people that have ridden a coaster, the more weight each rider carries in that coaster's ranking.

Those two factors play off of each other quite nicely, but when the two combine in a weird way you end up with little anomolies like T Express reaching the number one spot with just 2.2% of the voters having ridden it. (and while the site might say the cutoff is 1%, it's an old number...this year the cutoff is clearly 2% if you look throught the rankings)

And all of this discussion just starts to sound an awful lot like the ideas thrown around in this thread (members only) and I'm not sure anything was decided then other than there's a gazillion ways to run the data through an equation that all seem relatively valid in some way.

And hell, your post at the top of page two in that thread predicts this very issue, Jeff:

The track record size as a straight weighting would be too severe I think, again leading to the problem of John Q. Austrian rating some obscure ride highly.

So obviously, it's a personal pet peeve of yours. :)

The same discussion played out there - how to compensate and how much data was needed before it became relevant. I don't think there is necessarily a 'correct' answer. It's a judgement call.

EDIT - spelling

I don't mean to embarrass myself with my basic knowledge on the subject, but I have some education/experience on the topic. To my knowledge, the factor that taints a sample isn't necessarily the quantity of the sample. It's the selection of the sample, as I believe Brian was pointing out. And it's the representation of the sample to the overall population as I believe Gonch was pointing out.

So it seems to me that the question that would have to be answered is what characteristics of the overall population (people who ride coasters) are considered relevant for representation in a sample for a poll to determine which coasters are best? How would you even begin to answer that question? Is it demographical? Is it experience in riding coasters? Is it some measurement of enthusiasm?

What makes those 14 different from you besides their opinion... and is that better or worse for this poll?

Lord Gonchar said:
But still no one is suggesting a better way to do it.

Irrelevant. I don't know how to fix the economy, but that doesn't mean it isn't broken. You can do better than that.

I'm approaching this from a statistics standpoint. Even if I didn't, why is experience an important attribute? If it is important, does it suggest that less experienced riders have irrelevant opinions?

You're right, it is a judgment call, so let's not pretend that there's statistical relevance where there isn't any.

As Carrie said, I'm not even sure if there is a way to concretely define the enthusiast "community" -- whatever that is -- so that a reliable sample could ever be taken from that population. But I will say that the best way to screw up any chance of a poll having validity is to have the members of the sample self-select. You'll find more wisdom gazing at your navel.

Jeff said:
Irrelevant. I don't know how to fix the economy, but that doesn't mean it isn't broken. You can do better than that.

Not at all and no I can't.

Until someone shows us the correct way, how do we know this isn't the best way? We've seen the economy run unbroken, it's easy to tell that something's wrong and even still, it doesn't mean it was perfect before.

We've yet to see a coaster poll that's less broken than Mitch's, so how do we know this isn't the right way...or at least as correct as trying to quantify coaster enthusiasts' collective opinions of rides and express it in plain numbers can get?

If it is important, does it suggest that less experienced riders have irrelevant opinions?

Nope, just less informed or, as I said before, less 'experience' with coasters. The same way you question the validity of 14 riders opinion putting T Express at #1, I'd question the validity of someone who's only been on Mean Streak and Son Of Beast claiming that Mean Streak is the best coaster out there.

The opinion isn't irrelevant at all, but it's less informed (or experienced) and should carry less weight. That's why you ask as many people as possible. I still feel that 629 people is plenty of people for relevance. Unfortunately, only 14 of them have ridden T Express. Sometimes good enough has to be good enough.

You're right, it is a judgment call, so let's not pretend that there's statistical relevance where there isn't any.

And on the same note, let's not insist there isn't relevance where there very well could be (even though no one seems to be able to provide a cold, hard answer on that one).

And yes, it's a judgement call. So if your judgement is that 14 is too low a sample to be relevant, then put the asterisk in that spot as well on your scorecard.

Carrie M. said:
So it seems to me that the question that would have to be answered is what characteristics of the overall population (people who ride coasters) are considered relevant for representation in a sample for a poll to determine which coasters are best? How would you even begin to answer that question? Is it demographical? Is it experience in riding coasters? Is it some measurement of enthusiasm?

I think the answer is that it's a poll of people who give a crap about ranking coasters for people who give a crap about ranking coasters.

In the sense of, "Hey everybody. Let's all try to figure out what rides we like the best!"

In other words the only characteristics one needs is an interest in ranking coasters and the time to submit a ballot.

Taken for what it's worth, the whole thing is very relevant. It tallies the opinions of people who care a lot about coasters and shares those results with people who care a lot about coasters. I've always assumed that was the point - maybe I've been misinformed?

2. I think there should be bias toward people who've ridden more coasters - they create more comparisons for the data set and they're arguably more qualified to compare due to greater coaster 'experience' (for lack of a better term).

I agree. I have not been on nearly as many coasters as a lot of other people have. But it seems that the opinions of less experienced riders are irrelevant at least part of the time.

For example, I saw that the Grizzly was #176 on the poll. When I made my only trip to Great America and rode it when I was 15 yrs old it seemed like a fun ride. Perhaps due to the very small number of wooden coasters I have been on, which are: Colossus, Psyclone, Ghostrider, Grizzly, Giant Dipper. Would my choice of "best wooden coaster" mean much compared to the opinion of someone who has rode all, most, or even half of them?

A solution! We get a grant from the government to fund 300 members from the 'enthusiast community' to go on a 'research trip' around the world to these controversial places like Everland, Mirabilandia, Nagashima Spaland, Holiday Park, Alton Towers, Cedar Point, etc. etc. We select these 'members' randomly and we use to grant to send them all to these places, so that once and for all we can know the truth!

Of course, something tells me we would just find new things to argue about in this poll: "I didn't get to go on the trip, therefore I don't believe any of it. Everland sux!11!1!"

Charles Nungester said:
Two years from now, Voyage will be right back where it belongs. TPR is the same group that got served Beer for free at one park and ranked it over voyage even though it craws thru the turns.No Offense to T Express. I need a much bigger diverse sample than 6 of 9 verses over 250 people..The good ones and well maintained ones keep their status. Even though on this list. 50th or in some cases 100th could be someones personal favorites.

There are several new wooden coasters opening up next year( Prowler, Terminator, Fireball), and possibly several more the year after that. It's about more than just how well a seasoned coaster is maintained, it's also about how it holds up to the new batch. That's why polls are fun and informative, but should not be over-'anal' ized. Suppose Voyage gets rough like Legend? Then what? There are no absolutes and that's exactly what I love about Mitch's poll. Small sampling or not, it weeds out the "knee jerk" fanboys, and forces them to take notice of coasters outside their backyard.

Absimilliard said:

Charles Nungester said:Two years from now, Voyage will be right back where it belongs. TPR is the same group that got served Beer for free at one park and ranked it over voyage even though it craws thru the turns.No Offense to T Express. I need a much bigger diverse sample than 6 of 9 verses over 250 people..The good ones and well maintained ones keep their status. Even though on this list. 50th or in some cases 100th could be someones personal favorites.

I was on the TPR Korea tour last year and ranked Eagle Fortress my number 1 on the steel coaster poll . Strangely, I don't remember being served "Beer for free" (and I don't drink beer to start with). Since I was on a TPR tour, that means my opinion is automatically not good? I'm going back to Everland next march on my own, will that make my opinion "valid", since I won't be "influenced" by a group?The ECC is doing Japan and South Korea next year, so if T Express and Eagle Fortress keep their spots, will the blame start toward them?

No, Im just saying a bigger sample of mutual riders is needed. T Express looks like a wicked ride. Probably deserves its status. Id just like to see a lot more than 9 people decide that. If a group who generally prefferes steel coasters or very smooth ones go to a park that has a smooth woodie. Well.....Like what you want. Im not judging anyones opinion. Just my favorites tend to keep thier status without every new park I visit having the No. 1 coaster. I was kinda joking on the beer thing. That video shows Alvey saying, Free Beer before riding. It has to be no 1. I don't care. The coaster im on at the time Unless it hurts is my favorite :) Its about having fun.

John Knotts. I agree. However Legend last year expecially at HWN kicked butt again. All Im saying is the good ones keep a high ranking where the actual *FLASH IN THE PAN* is short term. Tonner De Zues? Wheres that in the poll now? 20th? 30th? 50th? But it was a small samples FAVORITE for a couple years anyway. Given the rides Timbers is capable of. That coaster could still crack most top ten or even five list but I think its fair to say that now that a bigger sample have ridden it and most haven't ridden it in *TOP Condition* Which I think it's fair to say Timbersfest gave for several years. Plus the new batch of coasters have dropped it some.Am I mad that T express won? HECK NO. Makes me want to check out what the fuss is about and decide on my own. Most of the times these coasters are EXCELLENT although I have been anticipointed at least once. (Boulderdash) Which IMHO is a fine ride but the whole return might as well be magnum lite.Chuck, who's actually ranked some coasters low for several years until I got the ride they were capable of. Thunderbolt at Kennywood Id visited five times before I GOT IT! It was never bad, just never great to me. Now its among my favorites :)

Lord Gonchar said:
Until someone shows us the correct way, how do we know this isn't the best way?

Still irrelevant. You're arguing adequacy, not statistic validity.

That's why you ask as many people as possible. I still feel that 629 people is plenty of people for relevance. Unfortunately, only 14 of them have ridden T Express. Sometimes good enough has to be good enough.

Asking 14 people isn't asking as many as possible. You're saying get numbers for completeness unless you can't. That's a contradiction.

Your entire argument is that no one has demonstrated a better way, which makes this way adequate. That has zero to do with my assertion that this methodology doesn't stand up against the scrutiny of the scientific method or basic statistics. As far as I'm concerned, that's the measure of adequacy. If rainbows and feelings of fuzzy adequacy are good enough for you, super, but they're not for me.

Regarding Tonnerre de Zeus and its lower rankings, very easy explanation here, would apply to a woodie in the middle of the US too if it happened there: The park doesn't maintain the ride AT ALL. Grease? Blah! Makes it go too fast! PTC trains that die after 7 years? New trains are needed in 2010 too, 14 years after it opened. Retracking? What's that? The ride is still up and that's all the park care about!

Look at Expedition GeForce, Balder, Katun... They kept their high rankings, even after everyone said "wait until more people ride them!".

Jeff said:
If rainbows and feelings of fuzzy adequacy are good enough for you, super, but they're not for me.

For an ordered list of which coasters enthusiasts seem to enjoy the most - it's more than enough for me. :)

When parks base their entire marketing stategy on something like the Golden Tickets, this is rocket science and brain surgery combined in comparison.

(rocket surgery? brain science? :) )