"Can" be controlled sure, but afer reading all these pages, plus the origional thread .. one persons controlled test is another sloppy job to skew numbers. At times it seems no-one is happy unless their product is tests the best.

Me, I like to see what people are getting on the dynos. It's interesting to see how they compare in "generally" uniform tests. You have to take a margin of error, but that's the nature of the beast.

I don't like when people go through the work to have a dyno day, and collect this info, only to have it bashed. So i'm just suggesting another outlet for the comparison. That's all. Besides, who wouldn't want to see a bunch of MINIs run the track.

 

I dont see how a problem would arise.....he's not on the M|U staff. He's just a member of the team. He will not be part of the testing. If we feel that his involvement in FTR will harm him being a TEAM member(not staff), we will discuss....but i really dont see how it could.

 

In this regard, I don't see why the denial of admission to Will [an innocent and virtuous guy- a good guy] should be held out as an example of integrity engendered into M|U. Detlman shall be of no different stature than Will in the very near future- in violation of these rules.

Admittedly, I was TMMZ. I once left to join in at M|U. Admittedly, it isn't all that I had hoped- it isn't any less clicky.

You're a wonderful group of friends and a good group of peeps, and that's the only reason I'm telling you. I trust that my point is made.

/End O/T [sorry]...

 

Actually, that is pretty inconsitent.

Obviously, all vendors who are participating in a test seem to care about is that a person involved in the team has something to do with their competition. I say this with confidence because of how much of a big deal was made out of Will being part of TMMZ and this article. He wasn't an M7 employee at the time the tests were conducted and did not have anything to do with the article or the test setup, yet it was still an issue. Vendors do not care about different levels of team involvement or what rules you use to determine is someone gets a special symbol by their screen name. All they see is someone on this team is part of their market.

If it is going to be made into an issues with one person or one test, it will come up again and someone will make somthing out of it. All of us that know each other know that their is/will be no bias, but someone taking their time to point out an affiliation and make an issue out of it with this test just goes to show that some people are not going to take your word for it.

 

What is the proper way to say there's a better way to get number? Or to say that one should consider measurment error when drawing conclusions?

Here's another way to put it, if the flaws of measurement technique aren't pointed out, the chance that the same mistakes will be repeated are much larger.

I may be wrong here, but I think you're misinterpreting the intent of those of us who commend the effort, but have insights into how to do it better, if something like this is done again.

I did a test on three throttle bodies. It took 15-16 runs to get enough data to know that measurements are repeatable, and do a baseline check. This study had 8 runs for 4 configurations, and no baseline check.

I know I go off the deep end on this kind of stuff, it's one of my hot buttons. But the people who spend the $ on modding their cars want the numbers that they use to base their decisions to mean something. I guess I'm just barking up the wrong tree in trying to help those without expertise in measurement technique, data analysis and statistics to learn how to get better numbers.

I'll quite down now. If anyone who wants to do some measurements wants my imput, feel free to PM me. I'm happy to share....

Matt

 

At this time, FBT Racing is not selling any products.

 

Matt, I totally agree with you that more runs, more data collected, the more accurate the information is. There's no doubt about that. Unless someone is willing to take the time and money to run, what may be a weekend of tests, with all the systems out there, run on the same car... we won't know how these systems really compare.

BUT, and maybe I missread the MU report, but it seems clear that under the conditions A, B and C as used, for results were that was posted. I'm willing to accept those results for those conditions. I'm not beliving those results to be law, however. It just seems as tho these results, or any that people post, are somehow seen as the end-all-be-all numbers but some people, and completely discredited by others.

All these tests have to be considered with a grain of salt. I'm coming from a completely stock MCS, and looking for the info that's out there for the mods. From what I see, they're all pretty damn close, maybe 1 or 2 hp, but factoring in a margin of error, that's negligible. My thought, put them on a track, 0-60 runs that kind of stuff, if nothing more, it'd be fun.

I'm not trying to mix things up, and cause more debate than what there already is.. jsut trying to give my opinion as an outsider (stock MCS driver) looking at the data.

 

As i have said in a few previous posts in this thread. You are certainly right. These numbers should only be seen as just that, numbers. They are not "laws" or 100% truth, they just allow you to see how they each compare to each other in this particular scenario.

It would be interesting to see if there is a difference in 0-60 or other things. We will definitely keep that in mind Everyone loves things that involves the chance to drive

Hope you guys all have nice days

 

what most of you are missing, and what the Dr is emphasizing, is that you have to discredit any/all of the results, comparative or not, if they are not shown to be repeatable. That was my point in looking at Scudermini's thread where the same car same dyno same parts on two runs showed a 6hp difference between runs (and he threw out one lower run!).

so this conclusion: "...they [the numbers] allow you to see how they each compare to each other in this particular scenario. " is still not valid, anymore than if you said the same thing about Scudermini's postings.

 

JennB,
Thanks for explaining the Will thing. We didn't have all the info about his involvment. But i think for future tests it would be good to know all involved so no one can complain or discredit the results because of someone involved.

Also can we see the first runs?

 

I think that's quite fair . Props to Alta.

 

Errr... from Hollis' introduction to the article:

 

Knowing what we knew previous to the posted MU info, it was a concern. But after the posted info was revealed, it made a little more sense.

Like i said, it doesn't bother us because it was said that he didn't "influence" things, and we have to trust judgment. It shows from other comments here that others are also concered about these types of affiliations on test. I think to keep future test inline, this is something to consider.

As much as i want to setup the dyno day test for the header shootout, i don't think we should. We and all other vendors do nothing but send parts to someone not affiliated with the parts being tested, and stand back.

 

Thats sounds like a piece of the proper methodology, but I'd suggest that perhaps the testing protocol should be detailed and hopefully agreed upon prior to the test taking place. Perhaps that would lower the level of critiquing, though clearly it wouldn't silence it.

Again, I think testing, any testing, has to be taken as a guide, not as fact. It was stated at the outset of the test that car in question was making what appeared to be abnormally high HP to begin with. It could easily be argued that the entire test should be thrown out on this basis alone. Why was it making that HP? Should they have performed a compression test or volumetric check to see if this particular engine was truly representitive of what most of us have? I could easily argue that had 4 runs been performed, that all we'd have is better numbers for this particular and possibly special car and that the test should have been perfomed across 3 other stock cars as well. Of course, if these gentlemen where prepared to go to these lengths, at that point, they should have just pulled the damn motors out and put them on a real dyno and eliminated all the variable inherent in mearsuring at the wheels.

If I think about the most commonplace auto testing, producing the kind of results that people actually lay out 10s of thousands of $$$ on the basis of, its far more flawed than anything seen here. 0-60, 1/4 mile times, skipad and slalom tests performed 12 months a year at different venues, accompanied by text like that this months CD comparision of the Exige and Cayman something like, 'jeez we we really surprised that the Exige 0-60 time was a second slower than the last Elise we tested despite being lighter, it must have been the track surface... ' Cripes, I'm surprised they didn't blame it all on the Exige's aero kit causing excessive drag.

So, again for me, I could be a complete idiot, but I accept that the results do indicate there is some value to adding a CAI, and personally given the limited runs, its probably reasonable to extrapolate from there. Sure, there is vaguery here, but heck thats life.

 

There is an old proverb: "For example is not proof"

If one is going to draw technical/scientific conclusions, then one should really try to employ the scientific method. Of course that includes an understanding of measurement principles, statistics (e.g., sample size, intra-individual variation, etc.), amongst others. These points have been aptly stressed by jlm and Dr Obnx. Those who think that the methodology posts are bashing are being rather myopic, rather than heeding the implicit advice and constructive criticism.

When limited, and possibly flawed, data are presented, there are those who will take the posted results as verified fact, which is not the case. Thus, the applicability of the above quotation. To reiterate what I requested in a previous post: would the data analyzers please post all the dyno curves for each car? That way, the results would clearly be shown, and the reader could interpret the graphs as they so choose.

In spite of the very apparent limitations in the study, I still applaud the organizers and workers for all their effort. Their experience and the myriad of comments should be helpful to others planning on some form of comparative parts testing ...

 

Bob,

Any word on WMS testing of the AGS? It looks loke you posted the last comment on the WMS forum regarding this.

 

I'm sure that all involved are just thrilled to have your applause. lol:. Now that's some really passive aggressive stuff right there. Whew.

 

Webb's board says that he's waiting on a V3 (!) tube before releasing results. I thought that that was kinda weird, so I asked M7. Short story, as far as what was told to me, is that all V2 tubes right now are HANDMADE/finished, and that makes the sort of mass production that Randy Webb's testing might bring essentially impossible. So, apparently, the new V3 is going to be machines out of metal [I believe aluminum, but don't hold me to that, of course!] to the spec of the V2. Their materials costs go up, I'd guess, but it's worth the time and effort [again, I suppose]. I believe they've asked WMS not to release anything until they can fill the orders .

Hope this helps... And, I hope I don't get in trouble! Heh.

 

to quote MRT. Web:
jan 28: :We do have testing, but with the V1 tube. We are waiting for the V3 tube after speaking with Peter on Thursday. It will be an aluminum bit instead of a rotary molded plastic piece.

As soon as ALL of our testing is completed, we will release our independent data.

Hope that helps!
Randy"

which brings up the question:
what version was tested here, what changed between v1,2,3?

 

[SIZE=2]The main difference was how much it leaked...
[/SIZE]

 

V2 was tested here.

There was a redesign in the opening of the tube that offered more airflow over V1

 

I'm pretty sure we tested V2....we will have to wait for hollis to chime in and say for sure. As far as i know, this V3 is a very new thing.

 

Just to verify. the tested part was indeed V.2.
We will most likely release the V.3 next week.

peter
Team M7
562-608-8123

 

For what it is worth I'd like to address performance modification comparisons from a different angle. I am curious to know what others think so let me know if I'm on to something, off base or just full of ****. Everyone please jump in - M7, Alta, WMS, NAM members.

I use similar criteria to evaluate MINI performance products as I do at work when evaluating new medications. One of my job responsibilities as a hospital pharmacist is to select which drug from a particular class to stock and use on our patients. The criteria used for comparative evaluation of medications and MINI performance products are listed below:

1. Efficacy

Is the new medication significantly more effective than older drugs from the same class?
Does the new performance product offer significant advantages over similar products already on the market?

2. Safety

Does the new medication have an enhanced safety profile over similar older drugs?
Does the new MINI product (e.g., CAI, exhaust, pulley) have any reported problems?

3. Reliability of the Manufacturer

Does the drug manufacturer provide good customer support? Have any of their drugs been withdrawn from the market? Was there adequate clinical testing prior to launch?
Does the MINI product manufacturer provide good customer support? Have any of their products been pulled from their catalog? Was there adequate testing and R&D before bringing the product to the market?

4.Cost
Does the drug cost more than similar drugs from the same class? This includes acquisition cost as well as preparation and administration cost.
Does the performance product cost more than similar products? This includes the product cost plus installation and maintenance cost.

Please jump in....

 

but the weighting is different. Replacement drugs need to do the same thing, with minimal risks. Car parts usually are bought to do something different, with slight risks.

But the idea of a personal criteria for evaluation of alternative ways of spending your money are a great idea, but for car adders, the differ a lot person to person... For example...

I only care a little about who I buying from.

I care A LOT about function.

I care a bit about how things look.

I don't care at all if a vendor has made a different part badly.
But if they did, I care a lot about how they delt with it.

I only car a very itty bitty little bit about sound, and it's non-linear. I'm more concerned about things that are louder, than less loud.


This is because I'm a function guy, I don't buy tires or wheels because of how they fill the wheel well, nor do I get brake kits because the spokes on my car would show a small rotor. I don't shop for exhausts based on sound, but I did take a very high flow exhaust off because it could wake the dead! (and made my ears ring). I've found that if you really make a motor and suspension perform, you'll get a car that sounds and looks like it has power and handles, all for a free extra.

But others are very concerned about aesthetics, and wiegh things very differently than I do. And since this isn't medicin, but a fun and practical hobby, that's OK.

Matt