Mitch's Poll - Wood Coaster Results

Well, more than 14 people have ridden Voyage to compare it to other rides, so right there that would make it more statistically relevant . . .

So a ride can't be the best because enough people haven't ridden it?

That's my main problem at this point with the 'statistic irrelvance' argument. Even if a ride is the best, if it's a rarer credit, it can't be ranked as such.

Seems like a major flaw right there.

^ That's the point I was trying to make Gonch. No matter how you try and throw around numbers to suit your point, it still comes down to "I don't agree with the results, so let's discredit the poll". People are much more transparent than they think.

Every coaster poll I have ever read leaves out a huge chunk of the enthusiast community, enough to actually change the results of the poll. There are a few that never even include coasters outside of the US. Nothing is flawless, it isn't humanly possible. But until someone can top what Mitch has done, this is the one I will use as a base.

Lord Gonchar said:
So a ride can't be the best because enough people haven't ridden it?

That's how statistics work. Why are you so hell bent on not accepting that?

Because it makes no sense.

Everyone in the situation who is qualified to judge something judges it and the aggregate says it's the best of its kind...but that's not good enough because it's not enough people saying it's the best.

That's absurd...no matter how many times I read it.

You claim that statistics in general says you can't judge it - the contest is fixed, there's no way you can be the best even though everyone qualified to say so says so.

Call Mitch's process flawed all you'd like, but I see way more wrong with a system where you can't 'win' even after being declared the winner.

It does make sense, for all of the reasons that smarter people than I have indicated. Selection bias and sample size do matter.

Well, again, no one of Those The We Know And Trust™ seems to have a problem with the sample size except you.

The sample bias has been clearly explained and I understand. However, it was also confirmed that using all the 'correct' methods for obtaining a sample that this exact data could be obtained.

A potentially adequte sample size pulled from data that could be obtained in a proper scientific method - I'll make no apologies for that being good enough for me.

Seems a little nitpicky to be that scrutinous of it, especially given the unimportance of the subject matter in the big scheme of things. Sounds like science and statistics (as you're arguing) say that a list like this just can't be black and white by the nature of what it tries to accomplish. I mostly agree with that.

But it also sounds like you're arguing that if the absolute answer can't be found there's no point in trying to get as close to the answer as possible and any attempt to get that answer is useless. I just can't agree with that.

We're officially going in circles with the next comment, but...

It's more than enough for the purpose it serves.

Nothing we talk about here matters that much, so what's your point? We do it anyway. Why do you want to take the discussion there?

You're totally reading into what I'm saying. I'm only saying it's a flawed survey and not likely an accurate reflection of any real generalized opinion. And as far as the science of polling goes, my opinion is absolutely reflective of it. If you disagree, and you find it adequate, that's super, but it doesn't change the fact that it does not stand up to the scrutiny of any scientific method.

I can't think of very many polls that really do hold up...especially polls that try to quantify the 'best' something or the 'favorite' something. So if that's the point, we're just kind of stating the obvious.

If not, what is the point of pointing out the scientific shortcomings? I guess that's where you lose me.

I don't think anyone (or myself at the very least) expected this to hold up to bonafide scientific scrutiny, so it just feels like pointing out the shortcomings implies a better way to do it.

Your argument for it not being the best way are the scientific shortcomings, but offer no better solution or reasoning (nor do you have to). You don't have a better answer. I don't have a better answer. This entire thread doesn't have a better answer. At least Mitch has an answer.

I still believe that a 'good' answer is better than no answer. That same 'good' answer is not better than the right answer. No one seems to know how to find the right answer or even if a right answer even exists. The 'good' answer is the best answer at that point.

I'd rather have the best possible answer than no answer. In fact, to me that's the scientific way. You use the info you have to come to the best conclusion possible. Maybe down the road more info or new methods become available and your solution is updated. At any given time in history the knowledge we have and the info available and the solutions we come to are merely the best available. There's a lot of information that seems silly and antiquaited now that once stood up to the scrutiny of scientific method...just like ideas of today will in the future. But that's just fruity, philosophical crap and I digress.

Sometimes it doesn't matter how you get there. Mitch might not be using his GPS to find the front door, but he's used a compass to get to the street.

Luckily, he's not looking for the cure for cancer at a specific address, he just needs to find the street fair. :)

And no, it doesn't change the fact that it does not stand up to scientific scrutiny, but at the same time, I'm not sure anyone expected it to.

With all of that said, I still suspect it gets closer to 'correct' than you think it does. And that's the second half of my twofold take on the whole thing:

1. It's not strictly scientific, it doesn't need to be.
2. It's probably a lot closer to science than you're giving credit for.

I don't know what else to say. We've been around the circle a couple of times now. That's where it stands.

(and I've typed the word 'answer' so many times that it's starting to not look right to me)

Lord Gonchar said:
I can't think of very many polls that really do hold up...

The folks at Gallup would beg to differ. The thousands of papers published every year on various academic pursuits, subject to peer review, tell a different story.

If not, what is the point of pointing out the scientific shortcomings? I guess that's where you lose me.

What's the point of anything we talk about here? That's an irrelevant point. It doesn't make the poll suddenly more valid.

I don't think anyone (or myself at the very least) expected this to hold up to bonafide scientific scrutiny, so it just feels like pointing out the shortcomings implies a better way to do it.

It does imply a better way to do it, but we've already discussed how it would be difficult to get a statistically significant sample size to do so. (Which brings up another issue that our "sport" is hopelessly small in scope.)

Your argument for it not being the best way are the scientific shortcomings, but offer no better solution or reasoning (nor do you have to). You don't have a better answer. I don't have a better answer. This entire thread doesn't have a better answer. At least Mitch has an answer.

Again, I can't tell you how to fix the economy either, but it doesn't mean I can't demonstrate that it's borken. Hell, there are problems I can't solve in my own life, but it doesn't make them non-problems. This argument that, "If you don't have a better solution, you can't criticize the poll," is stupid. Mitch's "answer" is a fine attempt, sure, but the whole point is that it can be skewed in a way that makes it less valid.

I still believe that a 'good' answer is better than no answer...I'd rather have the best possible answer than no answer. In fact, to me that's the scientific way. You use the info you have to come to the best conclusion possible.

That's not science. People once thought the world was flat, because that was the best conclusion possible.

1. It's not strictly scientific, it doesn't need to be.
2. It's probably a lot closer to science than you're giving credit for.

Probably is inconclusive, and I disagree anyway. Enthusiasts are not as important as they think they are, first of all, so I'd much rather have the aggregate opinion of Joe Six Pack (or those people from Greenland in the Burger King commercials). Enthusiasts are notorious for fanboyism, as we've seen for as long as this hobby has been discussed on the Internet. Just given that fact, selection bias absolutely distorts the measure of opinion. It doesn't matter if it's Cedar Point, Holiday World, or a bunch of nerds who follow their hero to Korea... any subset that can single-handedly change the results disrupts the measurement.

So you can redefine science if you want, but you'll still be wrong. :)

So what if the Joe Six-Pack poll ends up with Son on Beast as the #1 simply because it's "so gosh-darn big, it's gotta be the best!" Would that be scientific? It goes right back to "bigger is better" and the most-ridden coaster gets most votes. That's the whole thing Mitch is attempting to eliminate with his poll. The fact he reaches to the enthusiast community to gather his information shows that he wants a more informed opinion.

Coaster Enthusiasts certainly aren't the most informed group in general. However, they have a better understanding of what "makes" a roller coaster good than Joe the Plumber does. Because a comparison of 150 total roller coasters ridden is a better comparison of 12. That's what an enthusiast brings to the table, fanboy or not. Besides, the fanboy factor is WAY skewed toward Joe the Plumber because of that small sample size, he can only compare his paltry number where an enthusiast has so much more to draw from.

Is Mitch's poll science? No, I already stated it.
Is Mitch's poll a really good attempt to get rankings in a way that is different from all other polls? Yes.
Is T-Express the #1 wooden coaster in the world? I dunno, but it looks to me like it has the potential for it.

I think Mitch's poll would hold up better (and perhaps be considered more scientific) if it could get a better response from enthusiasts world-wide, not just the West.

BTW, I can't wait for the next podcast. Should be an interesting discussion between the two of you.

Jeff said:
What's the point of anything we talk about here? That's an irrelevant point. It doesn't make the poll suddenly more valid.

But that still doesn't get to the point. In fact, it skillfully avoids it. Why does it matter that this poll doesn't meet the scrutiny of the scientific method? I'm trying to figure out why that's such a sticking point given that:

1. There is no better answer available
2. People don't seem to care about the scientific validity and accept the results as a 'good enough' indicator of which coasters are most liked.

Like I said before, I'm trying to understand the goal of your statement. So it's not valid...now what? Or is pointing out that the poll is not done using proper textbook scientific method the point? And if so (also like I said before), I don't think anyone thought it necessarily was or expected it to be.

Again, I can't tell you how to fix the economy either, but it doesn't mean I can't demonstrate that it's borken. Hell, there are problems I can't solve in my own life, but it doesn't make them non-problems. This argument that, "If you don't have a better solution, you can't criticize the poll," is stupid. Mitch's "answer" is a fine attempt, sure, but the whole point is that it can be skewed in a way that makes it less valid.

On the flip side of the bad analogy spectrum, I can't take apart and rebuild an automobile engine or worse - build one from scratch, but I know when one is running well and getting me where I need to go. You seem to be complaining that it doesn't sound right even though it still runs.

I never said you couldn't criticize - just the opposite. I'm just wondering why you are. You keep talking about how difficult it would be to do something like this correctly and on the next breath criticize the poll for only doing it as well as possible.

It's like a circular logic that hurts my head.

1.This poll is not valid and useless.
2.A more valid poll does not exist.
3. See 1

That's not science. People once thought the world was flat, because that was the best conclusion possible.

That's exactly my point. At one time the idea that the Earth was flat held up just fine to scientific scrutiny of it's day. Science is nothing more than the best we can do at any given time. Mitch's poll does just that - the best that anyone is doing at this time.

Face it, it's not exactly a field that draws a lot of interest. :)

It doesn't matter if it's Cedar Point, Holiday World, or a bunch of nerds who follow their hero to Korea... any subset that can single-handedly change the results disrupts the measurement.

Can or could is different than do or did.

Look, I know the score here. The TPR tools voted up the ride. Great. It still doesn't mean it wasn't a good ride. Right now those tools are the only people qualified to make the comparisons needed in Mitch's poll.

How many people in the world do you think have ridden both T Expres and The Voyage? Probably not many.

Sadly, these people do represent the majority of people who can provide the needed data. They are the ones you want to sample. It's not a bias if these people pretty much make up the entire sample pool.

So you can redefine science if you want, but you'll still be wrong. :)

I'd still rather be wrong and have a decent solution (one that works for the needs) in my hands than be right with no solution. :)

So to take the argument to the extreme, if these 14 people are the only people in the entire world who have ridden both T-Express and Voyage, would they be a "big enough" sample space?

Also, is there a way to quantify the uncertainty? Can we say T-express is probably not the best wood coaster or probably is or might be or has a 32.58% chance of being the best wood coaster?

It seems like the fundamental argument is Jeff saying T-Express's true place on the poll is pretty uncertain and Gonch saying T-Express's true place on the poll can't be too far from where it is.

ApolloAndy said:
Also, is there a way to quantify the uncertainty? Can we say T-express is probably not the best wood coaster or probably is or might be or has a 32.58% chance of being the best wood coaster?

It seems like the fundamental argument is Jeff saying T-Express's true place on the poll is pretty uncertain and Gonch saying T-Express's true place on the poll can't be too far from where it is.

If you look at the "winning percentages" of the highest six coasters in the poll, they are all greater than 97%. Given that small, insignificant difference in the percentages, one could not say with certainty that there is any real difference between any of them. Does anyone actually believe there is a true difference between 1.000, 0.9885, and 0.9886? Even if this was a highly controlled scientific study (that we all agree it is not) and had a HUGE sample size (which we agree it does not), it would be nearly impossible to prove a statistically significant difference between the entries. Even with a sample size in the tens of thousands, the differences are too small.

So I agree with you Andy. I also agree that Jeff is right and Gonch is also right. The data does not support any firm conclusions. I'm embarassed to say I've probably wasted more time evaluating Mitch's data than any article I've read in the New England Journal of Medicine in the last year.

sws said:
I also agree that Jeff is right and Gonch is also right.

Which is weird because it seems like any time Jeff and I are on opposite ends of things like this that this is always the conclusion someone mentions.

Just one of the reasons I'm glad I know the guy. :)

Lord Gonchar said:

sws said:
I also agree that Jeff is right and Gonch is also right.

Which is weird because it seems like any time Jeff and I are on opposite ends of things like this that this is always the conclusion someone mentions.

Just one of the reasons I'm glad I know the guy. :)

I'm convinced you guys just like to argue like an old married couple. You two remind me of my grandparents. :)

Imagine if we did an uncensored grumpy bastard podcast. I don't think anyone would listen, but it would be fun to us.

I thought we already did the old grumpy bastard vesion of an enthusiast podcast.