Mitch's Poll - Wood Coaster Results

Are you guys still talking about this? Go to bed, for pete's sake. Or Santa Claus may just bypass your house . . .

It might not matter. Correct?

Ah, but in science, your results are guilty until proven innocent. Essentially, you want your results to leave no reasonable doubt in the reader's mind.

Or Santa Claus may just bypass your house

I'm happy to report that Santa has come and gone, and in record time. Usually, Santa has a few hours of work ahead of him by this time of night.

Charles Nungester said:

eightdotthree said:How is a tour from TPR that rode T-Express any different than coaster enthusiasts traveling to Holiday World for the various events that happen there. If The Voyage were #1 I don't think anyone would be debating that when in fact a bias exists there as well.

Much larger sample with a much greater variety in taste than just 14 people who generally preffer the same type of ride. Hers a question for you. Where did favorite steel rank in those 14 people? I Bet in every case its withing the top five. In my case the top steel isn't in my top ten, Why, because I like to leave the seat, get tossed about and latterals. .Like I said, this group done it before. They'll do it again. I don't really care. They are getting out and ridding coasters and having fun. If T express really is that good. It will hold its status. No worries.Chuck

It's not the poll's fault more people didn't travel to Korea to ride T-Express. Everyone had equal chance to ride it, and due to whatever circumstances, chose not to. For those who went, it was a slam dunk.

The way I interpret the poll, I simply cancel out the overseas votes since they do not apply to me, and bam, *my* #1 coaster among those i've ridden is Voyage. I don't blame the poll for not getting to a coaster that others did and loved.

If Voyage had won the top spot, all of this would be a non issue. People just can't accept seeing Voyage being knocked down, especially by a coaster that they probably never will ride.

I think most people here agree that the Theme Park Review trip carried the most weight in putting T-Express as the #1 wood coaster. I wonder how much influence the comradery of the TPR trip had. If all the TPR people went to ride T-Express individually, or at an ERT night, would it still have the same ranking? Perhaps there was something other than just the pure quality of the ride that put the ranking to #1. Additionally, there have been studies that show some people's opinions are infulenced by other peoples opinions. If you think T-Express is the best ride ever, then I would be more likely to rate T-Express highly. I guess this all comes back to sample size. Even though 14 people all rated T-Express the best wood coater of 2008, most if not all of those people were on the TPR trip. One group of people is not a sufficiently large sample.

John Knotts said:
It's not the poll's fault more people didn't travel to Korea to ride T-Express.

You're personifying the poll now, which is not something anyone in this discussion is doing. Read what Brian and I have been talking about regarding statistics. The absence of data doesn't validate the data that is available. That's not how statistics work.

Coasterphan said:
If Voyage had won the top spot,all of this would be a non issue. People just can't accept seeingVoyage being knocked down, especially by a coaster that they probablynever will ride.

Oh yeah, that's what it is. Thanks for adding that.

0g actually brings up a whole host of issues that would not stand up to scientific scrutiny. The flaws in the poll's methodology are two-fold, from the statistical insignificance to the selection criteria (or lack thereof).

Look, no one disagrees that the poll is entertaining and good water cooler talk. It's even a better attempt at polling than the typical popularity contest (or worse, the Amusement Today joke). But as an accurate measure of opinons, it's totally flawed.

I'd call this one at least questionable.~"CALIFORNIA, USA"~"RICHARD HARBISON"Abrupt Airtime_____________________________,y,y,1Floating Airtime___________________________,y,y,2Out of Control Feeling/Directional Changes_,y,y,3Sensation of Speed_________________________,y,y,4Lateral Forces_____________________________,y,y,5Positive G's/Loops_________________________,y,y,6T Express Everland KR,y,y,1El Toro SFGAd NJ,y,y,2The Voyage Holiday World IN,y,n,3My Favorite Steel Anypark Anywhere **,y,y,4Ravine Flyer II Waldameer PA,y,y,5Kentucky Rumbler Beech Bend KY,y,n,6New Mexico Rattler Cliff's NM,y,n,7Ozark Wildcat Celebration City MO,y,n,8The Legend Holiday World IN,y,n,9Raven Holiday World IN,y,n,10, AlphaGhostRider Knott's Berry Farm CA,y,y,11Colossus S.F. Magic Mountain CA,y,y,121 out of the states woodie ridden, El Toro with no Rolling thunder vote.Ravine Flyer II without a Comet vote. Colossus without a cyclone vote (although thats probable)I didn't say its a lie, I said itts a world wide collection for having ridden only 12 woodies. Probably on YOUTUBE.

Allow me to attempt to sum up, but without the needless numbers game...

Mitch's poll is what it is. A poll of enthusiasts for enthusiasts. Chances are, regardless of mathematical significance of the sampling, it speaks well for our shared obsession. I had always considered that it was the ONE poll that throws away the "most ridden = most votes" factor. Is it perfect? No, period.

Now, consider the FACT that El Toro is a VERY well liked coaster, and has been ridden by a much larger percentage of Mitch's respondents, and finished #3. Based solely on pictures (since I have not ridden either Intamin Plug & Play in question) T-Express certainly LOOKS like a better ride than Toro. This makes me believe that T-Express actually has the potential to be #1, regardless of the percentage of North americans who have ridden it.

Haha! Just last year and every year that preceded it, everyone was praising this particular poll and saying how accurate the results were. Now everyone favorite coaster gets bumped and outrage ensues.

I personally don't care what some poll says. The coaster that I enjoy will be different than other enthusiasts anyway. If YOUR favorite coaster was #1 on Mitch's poll, you are not a winner or a loser. It matters not.

My favorite wooden coaters is still the Beast. It's not #1 on the poll anymore and my feeling are not hurt about that at all.

(Once I ride Voyage, my #1 may change though, but that's for another topic.)

Jeff said:

John Knotts said:
It's not the poll's fault more people didn't travel to Korea to ride T-Express.

You're personifying the poll now, which is not something anyone in this discussion is doing. Read what Brian and I have been talking about regarding statistics. The absence of data doesn't validate the data that is available. That's not how statistics work.

Sorry Jeff, but I think you're speaking for yourself and Brian here, and trying to apply that to the entire conversation. There are two different threads of thoughts happening here, and what I'm responding to is what Chuck posted. I'm staying out of the statistical part of this conversation because like others have said, I can't connect to it unless someone can offer a better alternative. No one can.

Let me try to answer the question concerning adequateness of sample size so that the results could be considered statistically significant. The answer is..... it doesn't matter. Why?? Because regardless of the sample size, the results would not be statistically VALID. It wouldn't matter if you had n=14 or n=14,000. The results could never be considered statistically valid due to the way the data was collected. Statistical analysis requires random sampling of a defined population. It assumes that the sample is representative of the greater population as a whole and was randomly obtained. As Brian has pointed out, this poll is not a random sampling; people have preselected themselves to participate in the survey. Without a random sampling, it would be statistically invalid, regardless of the number of people in the survey. Hence can we please stop referring to statistical significance for this poll? In this particular case, "size DOESN'T matter." Which is actually a shame, because that at least is one thing thing I have going for me. :) ;)

That being said, it is still a fun poll to look at and should be taken with a proverbial grain of salt given its inherent flaws. I look at it and think, "Wow, there seems to be alot of fun-sounding rides out there that I had never heard of." Beyond that it doesn't matter. As someone else has said, the best coaster is the one you happen to be riding at the time.

sws said:
The results could never be considered statistically valid due to the way the data was collected. Statistical analysis requires random sampling of a defined population. It assumes that the sample is representative of the greater population as a whole and was randomly obtained. As Brian has pointed out, this poll is not a random sampling; people have preselected themselves to participate in the survey.

Ok, I apparently don't get this. I understand what you guys are saying, but I still don't entirely see the difference.

When you sample people at random, they still have to choose to participate. No matter the method, you have to find willing participants. A random sampling of a defined population, right?

So what exactly is the difference in asking people in the defined population until you have enough willing to particpate or just asking everybody in the defined population who are willing to participate to do so? (aside from Brian's scenario about ulterior motives)

In other words if you need 600 samples to be valid, what's the difference in asking random people in a defined population until having the offer accepted by 600 (getting the necessary data) or just asking everyone in the defined population who be interested to participate and getting the same 600 (or more) to participate? It's still 600 points of data from the same definied population.

How many of us here knew of the ongoing poll but didn't participate? How many did? Wouldn't that itself be a random sampling of the defined population?

What am I missing that defines the difference between the two?

Gonch, it goes back to what Brian was talking about with selection bias.A group of 14 coaster geeks go to Korea and have the time of their lives. They have such happy memories of their fabulous trip that they all immediately participate in Mitch's poll and their selections reflect the overall wonderfulness of their trip. Their participation could not be considered a random sampling given their preselcetion bias. Thus bias is entered into the data collection. There doesn't even have to be any malice involved. It could be all warm and fuzzy bunnies and sweet smelling flowers.

So how could you get a random sampling from which you could determine statistical significance? You define your population ahead of time. For instance, Homeland Security is afraid of a terrorist attack at a major theme park and have thus used there extensive resources to collect personal information about all people who bought season passes for theme parks in the last five years. Some marketing company steals Homeland Security's data base and has telemarketers randomly call people on the list and obtain their opinions. Thus you could randomly sample the larger population and then potentially use statistical analysis to interpret the data. However the end result would be that the coaster geek community would learn that Homeland Security is watching them, and all of the geeks would be afraid to leave their mothers' basements and all of the parks would go bankrupt. :)

I don't have a problem with 14 coaster geeks going, making a judgment and being fair in that judgement. I have a problem with pollsters deleberately trying to skew the results like the one I posted above. T Express the only Out of Country woodie ridden. Ridden Ravine Flyer II and not comet and another example of that in that vote. Theres also one that has Raven, Legend and Voyage ranked as their least favorites while T Express is No 1 Probably somebody with a grudge getting escorted out for bad behavior.. In nobodys book are those the three worst woodies on the planet I like the poll, Think its highly accurate but in about 1 percent of the vote. those votes should be void as they are deliberate attempts to skew the results.

So what exactly is the difference in asking people in the defined population until you have enough willing to particpate or just asking everybody in the defined population who are willing to participate to do so? (aside from Brian's scenario about ulterior motives)

That is the difference, because at the end of the day, everyone has an ulterior motive---and those with the strongest motives (whatever they may be) are most likely to respond, giving you only the outliers. Response bias is generally much milder in form than my example---I purposely constructed a scenario in which it is obviously bad. But any time you design a survey instrument that has the response bias problem, your results are effectively unpublishable in a reputable venue.

Personally, I think holding Mitch's results to that standard of rigor isn't necessary.

But, if I'm reading Jeff's point correctly, I think he's suggesting that even though Mitch's results are better than a straight-popularity vote, they still may be "wrong" in some objective sense, for the two reasons brought up so far. One: it has response bias. Two: the cutoff for "statistically significant" (10 riders) is arbitrary and without justification. There are some statistical tools you can use to see what your cutoff should be, but I don't use them often enough to apply them here intelligently. Thankfully, I rarely have to study people's attitudes, and mostly just measure objectively instead. ;)

Yes, those are my two points. And I personally don't care what the results are, but people are right to question the methodology on the basis of a relative unknown being #1 because 14 people make it so.

Understand also that because of my media background where I've worked in a lot of surveys and polls (we used to live and die by Arbitron in radio), I have an appreciation for the "right" way to conduct them.

So what is the right way to conduct this poll, Jeff? How should it be done in the future? (not a challenge, but a sincere question given this is a coaster ranking poll)

You'd have to eliminate the selection bias and have a large enough sample size, as we've been talking about.

I hear you. How do you do that with this poll?

sws:
A group of 14 coaster geeks go to Korea and have the time of their lives. They have such happy memories of their fabulous trip that they all immediately participate in Mitch's poll and their selections reflect the overall wonderfulness of their trip. Their participation could not be considered a random sampling given their preselcetion bias. Thus bias is entered into the data collection. There doesn't even have to be any malice involved. It could be all warm and fuzzy bunnies and sweet smelling flowers.

But asking people at random and still including those 14 in the end would be statistically valid?

I have a hard time with that.

Brian Noble said:
Personally, I think holding Mitch's results to that standard of rigor isn't necessary.

And you're one step ahead of me. That was my next move.

...which is what I've been saying all along. For the purpose this is supposed to serve, the sampling is entirely fine...and as we've already established it's entirely possible that the number of samples could be high enough to be valid.

Jeff said:
Yes, those are my two points. And I personally don't care what the results are, but people are right to question the methodology on the basis of a relative unknown being #1 because 14 people make it so.

Again, it seems you just can get past the idea that 14 people could respresent the whole.

What about the fact that a vote in Mitch's poll isn't just a vote for a coaster, it's a vote for a comparison of two coasters? For example, when someone lists T Express at #1 on the ballot and The Voyage at two, those two pieces of information are used to create one data point.

How many people in the world have ever ridden both T Express and the Voyage? Probably few enough that these comparisons are respresentative of everone who has.

It's not just 14 people saying a coaster is the best and it getting put at #1. (in fact, not all of those polled did say it was the best) It's about people qualified to make a decision as to which is better doing so.

Those 14 people could very well represent the majority (or the entirety in some cases) of people who've ridden T Express and any other given coaster.

Understand also that because of my media background where I've worked in a lot of surveys and polls (we used to live and die by Arbitron in radio), I have an appreciation for the "right" way to conduct them.

Is this like the FICO discussion where indirect past experience equates to authority on the subject? ;)