Monday, March 27, 2006

Paper Problems

One of the more annoying types of papers is what I like to refer to as the "What The Heck Were They Thinking" (WTHWTT) paper. These papers will have bizarre protocols, an odd selection of what types of controls they will utilize, where they apply said controls, and make far-reaching claims based on their data.

The most annoying parts are:
A: They still get published in prestigious journals.
B: You finish reading the paper and realize that as frustrating as the paper is, they're probably still right.

Tonight's example--a paper I am reading in class this week--will not be named, because that would be rude. I am still a lowly undergrad, and I must freely admit that these people DO have a better idea of what they're doing than I do. Nonetheless, I must confess to being irritated by a few aspects of their approach. I'm at the point where I've read enouugh papers on drug addiction that I know what the protocols look like. Also, this paper involves a technique that my lab does a lot of, although I am forced to admit that I have never performed these experiments myself.

First off, they utilize two control groups: an untreated group and a sham group (a group which undergoes the same treatment the experimental group does, with saline replacing the drug in question). That's fine: especially since the procedure involves a survival surgery, it's sensible--and applaudable, since it's not always done--to make sure that the surgery itself is not responsible for any of the changes they monitor. However, although the untreated group is used as a control for all of their experiments, the sham group is only used in a few randomly selected experiments. This is the reverse of how it's usually done: one usually compares the exprimental group directly to a sham group, and then utilizes an untreated group for a few key experiments (if at all) to demonstrate a lack of difference between the sham group and an untreated group. I'm forced to conclude that I missed out on the great saline shortage of the late 90s.

Even though they've previously demonstrated a difference in response when utilizing an electrical stimulation vs. application of an agonist for the receptor in question, they only utilize electrical stimulation in this experiment and interpret their results assuming that agonist application would result in the same data, except where they don't. They claim that further confirmation of these presumptions is impossible with their current protocol--with no more detailed explanation, even though this claim makes no sense--and then confirm it using agonist application for one of the half dozen or so experiments they perform. Furthermore, if their protocol prevents them from getting the data that's actually important to their hypothesis, they should either use a different protocol or restrict their claims about results to what they've actually demonstrated. That's why papers have a "discussion" section in which you "discuss" potential extrapolations of your results and future avenues of research.

Almost all of their data is from brain slices in baths containing a few ingredients (only added to the experimental groups) that I haven't seen added in similar papers, and I'm pretty sure we don't use our selves. It's really quite possible that I'm just not as up on how this is done as I think I am, or that it was just done differently at the time than it is now. However, they only do one experiment where they compare the result of these ingredients when added to the baths for untreated slices, and they never perform an experiment where they show results without the ingredients in the bath. Again, in their defense, if they're right and I'm wrong it would in fact throw off the data.

They use one of their experimental groups first for experimental data, and then claim it as a control for their previous experimental data. It's possible this is thoroughly acceptable, but it feels like a bit of a no-no.

But their worst offense is going to be difficult to explain. The first experiment performed demonstrates that their primary experimental group and the untreated group are different in the magnitude of the basic response being studied. What does this mean, in your human english?

It means that statistic analysis indicates that the two groups are intrinsically different in terms of this response. It means that when you look at the data from these two groups--measure the magnitude of a response--you can safely say that you're actually dealing with two groups in terms of this response, that the differences are not due to chance. Now further experiments performed are based on--comparing how the two different groups respond to an inhibition of the response, or to an enhancement of the response. That's pretty safe, since the differences between the groups in those experiments are--shall we say--"more different" than the two groups are without modulation.

But then they perform an experiment in which the slices are sorted into groups based on the basic magnitude of their responses, which they then compare. And I'm forced to go: huh? Didn't they just demonstrate that these two groups are not, in fact, directly comparable in this respect? And to top it off, half of the groups have responses of magnitude 1.5-2.5 times greater than the highest magnitude achieved in the initial experiment, with no claim that the protocol was somehow different or even any acknowledgement that this might seem odd.

So, yeah: it's been a frustrating night.

By the way, I should make my presumed caveats clear at this point:

  • I'm not claiming the researchers are incompetent. I assume, in fact, that they are much more competent than myself--although perhaps not very good writers. It's possible that I'm just being overly paranoid about their approach, and these are all acceptable approaches I just haven't encountered or just don't understand. I do, in fact, intend to bring up my questions in class tomorrow and attempt to find out what I'm missing.

  • What they're doing is very very hard. I happen to have it on good authority that the type of experiment performed is one of the hardest to be found in neuroscience. I've been told that even people who have been great at it for years will suddenly hit dry patches where they can't get any data out of it for a month. I can understand the urge to cut some corners.

  • I have some fair amount of confidence in their data. I know that much of this has been confirmed by later research. Much of the rest, seems to be consistent with other results I've seen in similar areas. I just think it comes across as more than a bit sloppy, leaving a fair number of loose ends to be tied up.

5 Comments:

At 30 October, 2015 00:40, Blogger ninest123 Ninest said...

ninest123 10.30
longchamp, gucci outlet, oakley sunglasses, michael kors outlet, ugg boots, cheap oakley sunglasses, michael kors outlet, kate spade outlet, oakley sunglasses, nike outlet, prada handbags, polo ralph lauren outlet, ray ban sunglasses, christian louboutin outlet, replica watches, longchamp outlet, kate spade handbags, tiffany jewelry, ugg boots, prada outlet, louis vuitton outlet, chanel handbags, michael kors, louboutin, coach outlet, nike free, polo ralph lauren outlet, louis vuitton, jordan shoes, tory burch outlet, longchamp outlet, michael kors outlet, replica watches, tiffany and co, michael kors outlet, oakley sunglasses, burberry, nike air max, louis vuitton, ugg boots, burberry outlet online, oakley sunglasses, michael kors outlet, ray ban sunglasses, louis vuitton, louboutin outlet, ray ban sunglasses, louis vuitton outlet, louboutin shoes

 
At 30 October, 2015 00:44, Blogger ninest123 Ninest said...

lacoste pas cher, abercrombie and fitch, vanessa bruno, sac guess, ray ban pas cher, nike free, nike huarache, nike roshe run, michael kors, hermes, new balance pas cher, converse pas cher, tn pas cher, timberland, longchamp, lululemon, ralph lauren pas cher, hollister, hollister, ray ban uk, air force, north face, nike roshe, air max, hollister pas cher, coach factory outlet, hogan, nike air max, vans pas cher, north face, longchamp pas cher, true religion outlet, nike free run uk, sac longchamp, michael kors, nike blazer, louboutin pas cher, air jordan pas cher, michael kors, oakley pas cher, nike air max, true religion jeans, ralph lauren uk, michael kors, nike trainers, nike air max, true religion jeans, mulberry, true religion jeans, coach purses, burberry

 
At 30 October, 2015 00:47, Blogger ninest123 Ninest said...

reebok shoes, soccer shoes, ferragamo shoes, oakley, ray ban, louboutin, beats by dre, valentino shoes, iphone 6s plus cases, mont blanc, herve leger, bottega veneta, asics running shoes, nike air max, north face outlet, ghd, hollister, ralph lauren, chi flat iron, s5 cases, nfl jerseys, new balance, lululemon, vans, nike roshe, iphone 5s cases, iphone cases, baseball bats, ipad cases, vans shoes, mcm handbags, iphone 6 plus cases, instyler, p90x workout, gucci, babyliss, soccer jerseys, timberland boots, birkin bag, celine handbags, insanity workout, iphone 6 cases, abercrombie and fitch, jimmy choo shoes, iphone 6s cases, wedding dresses, converse, mac cosmetics, hollister, north face outlet, nike air max, giuseppe zanotti, coach outlet

 
At 30 October, 2015 00:51, Blogger ninest123 Ninest said...

doke gabbana outlet, canada goose uk, thomas sabo, converse outlet, canada goose outlet, moncler, moncler outlet, swarovski, moncler, canada goose, doudoune canada goose, supra shoes, bottes ugg, pandora charms, moncler, links of london, hollister, juicy couture outlet, ugg pas cher, replica watches, karen millen, ugg,ugg australia,ugg italia, pandora charms, pandora jewelry, montre pas cher, louis vuitton, toms shoes, sac louis vuitton pas cher, canada goose, marc jacobs, coach outlet, michael kors outlet, michael kors handbags, lancel, canada goose outlet, juicy couture outlet, moncler, louis vuitton, wedding dresses, moncler, moncler, swarovski crystal, michael kors outlet online, louis vuitton, canada goose, barbour jackets, ugg,uggs,uggs canada, ugg boots uk, louis vuitton, pandora jewelry, moncler
ninest123 10.30

 
At 11 December, 2015 02:00, Blogger mmjiaxin said...

air max shoes
wedding dresses uk
michael kors uk outlet
ralph lauren outlet
ugg boots sale
chanel handbags outlet
cheap nfl jersey
michael kors outlet online
michael kors uk
basketball shoes,basketball sneakers,lebron james shoes,sports shoes,kobe bryant shoes,kobe sneakers,nike basketball shoes,running shoes,mens sport shoes,nike shoes
cyber monday deals
tiffany and co
louis vuitton outlet store
ferragamo outlet
louis vuitton outlet store
uggs outlet
fitflops sale
canada goose jackets
tiffany outlet
cheap soccer jerseys
mm1211

 

Post a Comment

Links to this post:

Create a Link

Send Haloscan trackback ping

<< Home

Academics Blog Top Sites