Inferences made in research, objective?

[buffalopc7]

I hesitate to discuss research, however I came across this article today and was compelled to comment, out of disgust. While I cannot copy the entire article, here is the citation...

Title: Difficulties in Comprehending Causal Relations Among Children With ADHD : The Role of Cognitive Engagement, By: Pugzles Lorch, Elizabeth, Eastham, David, Milich, Richard, Lemberger, Clarese C., Sanchez, Rebecca Polley, Welsh, Richard, van den Broek, Paul, Journal of Abnormal Psychology, 0021843X, February 1, 2004, Vol. 113, Issue 1

In brief, children with and without ADHD were asked individually to watch two episides of a TV show. For one show, the TV was surrounded by toys, for the other, the toys were absent. The children were then questioned in order to assess their recall and understanding of the show and characters. The results showed that the children with ADHD demonstrated less attention to the show when the toys were surrounding the TV. Ok, thats not a shock to me. The children with ADHD also tended to answer the comprehension questions less accurately when the toys were present.
The thing that caught my attention (lol) was the inferences made by the researchers regarding the results...
"Four pieces of evidence converge to support the conclusion that the difficulties the children with ADHD have in understanding causal relations in the toys-present condition are due to less sustained cognitive engagement."

First off, the most glaring error here is the researchers' choice to make an inference based on data, unfortunately that practice is all too common. I could, after reading the study, provide a myriad of alternative inferences, however mine would not weigh on the negative possibilities. Children (and adults) with ADD/ADHD often have higher than average intelligence and the children here who have ADHD may have simply lacked interest in the television show and/or in teh researchers questions. Were they engaged? Maybe, maybe not. I know if I had been one of those children, I would have been bored to tears and obstinate enough to give erroneous answers simply to break up the monotony. While there could easily be dozens of possibilities, the researchers in this case chose to (in my belief) perpetuate a widely held belief that children with ADHD have a deficiency in understanding. Sad.

[Hyperion]

I think you may be misinterpreting what the researchers mean by "cognitive engagement." They were not saying that the children had less cognitive abilities or that there was a deficiency in understanding. That would clearly make no sense since if they did fine without toys.

I think what they meant was that children with ADHD often see A and B, but they have difficulty seeing the A->B action because they don't pay attention (or "sustain cognitive engagement" in the action) long enough to notice it. So the ADHD kid sees A, then sees the toys and gets distracted, then sees B. He does not understand the causal relationship between A and B because he's not paying attention to the TV show when it occurs.

Note that they specified that this was a lack of understanding during the toys-present condition that was not there during the no-toys condition. They probably did this specifically to rule out the possibility that the problem was that the people with ADHD couldn't understand the material. They understood it just fine when there weren't distractions.

And finally, note that they said "cognitive engagement," not "cognitive ability." "Cognitive ability" would be how smart they were, "cognitive engagement," or "sustained cognitive engagement" refers to whether they were actually focusing their cognitive abilities on the television.

So I actually think that the researchers were trying to make the same point that you were, that it isn't a true lack of understanding because they can understand the material just fine without the toys present, but that the appearance of a lack of understanding of what had occurred on the TV was because they simply weren't paying attention or focusing their cognitive abilities on the TV when there were toys around.

[meadd823]

Originally Posted by buffalopc7

I hesitate to discuss research

HeHeHe Welcome to the forums, you have now passed the initiation process obviously. Sorry can’t remember when I took my medications last so it is meadd823 in “raw” ADHD form. No telling what will pop out.

Originally Posted by buffalopc7

First off, the most glaring error here is the researchers' choice to make an inference based on data, unfortunately that practice is all too common. I could, after reading the study, provide a myriad of alternative inferences, however mine would not weigh on the negative possibilities.

Thank you, and agreed. If you were standing in front of me I would hug you for this. Positive stuff in ADHD (or any neurological difference) is always over looked in favor of more "problem orientated" findings. So all science can provide is a bunch of negative stuff about ADD. They can give us pages (some times entire books) of stuff ADDers suck at but not one shread of ability we could aime for. Doesn't give us much to work with does it? (a personal pet peeve of mine)

I have often wondered if they tested non-ADDers on how ADHD they were I think we would find they don’t do ADD any better than we do non-ADD. They call this circular logic; I think one should use logic weather straight or slightly bent as it is so hard to come by these days.

I like these words I read in one of the several books I read about executive functions.



“The Executive Brain”
Elkhonon Goldberg (forwarded by Oliver Sacks)
Page 180

Our culture traditionally approaches neurological disorder as a deficit a loss. This is reflected in our terminology: aphasia – loss of language; amnesia loss of memory. Hypermnesia and hyperverbality, when they occur are viewed by society as a mnemonic or literary gift, not as a pathology. But if the norm is defined as the population average, then talent is by definition a deviation from the norm. The relationship between talent and psychopathy has intrigued and beguiled both clinicians and those afflicted (or blessed)

It is common to distinguish in neurological and neuropsychological conditions between “negative” and “positive” symptoms. Negative symptoms reflect the loss of some thing that should normally be present, such as the ability to walk, talk, and see things. Positive symptoms reflect the presence of some thing that is not part of normal cognition, such as hallucinations or tics. Negative symptoms are more readily comprehensible, easier to conceptualize, measure quantify and subject to rigorous scientific examination. Positive symptoms are usually more elusive more mysterious but also more intriguing and challenging. They hint at an inner world that is different and not merely impoverished – at the presence of a neurological condition which not only robs but also endows.
***End Quote


Gee I haven’t the desire to sit watch TV, even as an adult. As a child I rarely sat through a show even those not lasting over 30 minutes. So I am impaired, if not being a couch potato is impaired (okay what ever)

The toys were distracting duh wonder how much money it cost to figure this out? I could have told them that for free. Spending money to figure out children especially ADD children find toys around a TV distracting when compared to not having toys around a TV should be considered an impairment. (IMHO)

Originally Posted by Hyperion

I think what they meant was that children with ADHD often see A and B, but they have difficulty seeing the A->B action because they don't pay attention (or "sustain cognitive engagement" in the action) long enough to notice it.

However what they meant isn’t crystal clear, I agree.

Originally Posted by Hyperion

Note that they specified that this was a lack of understanding during the toys-present condition that was not there during the no-toys condition. They probably did this specifically to rule out the possibility that the problem was that the people with ADHD couldn't understand the material. They understood it just fine when there weren't distractions.

How do we know ADD children find toys any more distracting than those who do not have ADD. I failed to notice a control set being given the same test. Which is why we are left to infer because for all we know by reading this is all children would be equally distracted by the toys. (IMHO)

Were the shows with the toys and without played one after the other? Were the toys present on the first or second show? Were the shows the same of different. Was one a documentary and the other a cartoon? Does buffalopc7 have a hyperlink to the whole thing. Nosey ADDers want to know I think you also have a point, not really enough information provided and a few things seem to be missing here and there so we are left with doubt as to the meaning of this entire study. Maybe the eight reasearchers just wanted to watch TV during class time.

[Imnapl]

Thanks, Meadd, for reminding me to read more from Dr. Sacks. Buffalopc7, did the citation mention the genders and ages of the children in the study?

[SB_UK]

...without diving into the article
- the best science can manage is a kinda' internally valid - logically consistent microcosm view upon *that* reality.
If we say cars are all either red or blue -
- and that the steering wheel is always the same colour as the car paint - - then in this microscientific puniverse'd ->- out'd pop the correct answer given the question ->-
'which colours are steering wheels in your world?'

...problem is that -
... is that who wanders about making statements akin to these
'cars are all red and blue'
'the steering wheel is always the same colour as the car paint'

-in current medical research - the thought process is (simplifying) ->-
if bad then disease
if disease then cure
if curing then administer chemical compund

->- my simplification parodies the unspoken simplification held as sacred within medical circles.

Often correct -><- but certainly not always correct -
- these basic ideas of as much harm (when applied inappropriately)
->- as good ... when applied correctly.

[Hyperion]

...without diving into the article
- the best science can manage is a kinda' internally valid - logically consistent microcosm view upon *that* reality.

Yep. I agree 100%. The scientific method is not perfect, but it is the best method available. Unless you have a better suggestion...

I mean, wwe tried the philosophical method, where we just think about something and pick the answer that feels right. This was how it was "discovered" that the Earth is the center of the universe around which the sun and planets and stars revolve, and it was how it was "discovered" that heavier objects will fall faster than slower ones.

Of course, then Gallileo came along with the idea that hey, maybe we ought to test these things out first. Let's make some observations under a number of different conditions, and try to find an answer that best fits objective, independently observed phenomena.

-in current medical research - the thought process is (simplifying) ->-
if bad then disease
if disease then cure
if curing then administer chemical compund

->- my simplification parodies the unspoken simplification held as sacred within medical circles.

Really? Because it sounds more like a parody of the anti-med wingnuts to me.

How about this:

If problems occur
observe behavior causing problems
if behavior causing problems matches x, y, and z, but not a, b, and c
diagnosis ADHD
choose chemical compound based on patient history and diagnosis
observe behavior
if behavior improved and if problem diminishes, continue treatment

If you're gonna make a simplified parody, at least make it somewhat realistic and not a straw man. Otherwise you are insulting the intelligence of everyone here.

Often correct -><- but certainly not always correct -

Ok, I'll agree that even the more accurate simplification I mention is not always correct. Do you have a proposal that is more often correct? Or that results in better outcomes? I mean, any idiot can criticize, but as we policy wonks are fond of saying "what would you do about it?"

[buffalopc7]

Originally Posted by Imnapl

Thanks, Meadd, for reminding me to read more from Dr. Sacks. Buffalopc7, did the citation mention the genders and ages of the children in the study?

Hi Imnapl,
Sorry, I hadn't realized you had posted to this thread. Here is the citation regarding the ages, etc. of the subjects:

Participants

A total of 135 children ranging from 7 to 11 years of age participated in the study. None of these children had participated in prior studies of television viewing by this research group. Three children (1 ADHD and 2 comparison) were excluded from the study because of a refusal to complete all tasks. The final numbers for the sample were 70 children diagnosed with ADHD (47 boys and 23 girls) and 64 comparison children (40 boys and 24 girls). As shown in Table 1, there were no significant group differences in gender, age, or ethnicity. The study was reviewed and approved by the university Institutional Review Board, and informed consent was obtained from the parents of all children. The session lasted approximately 2 hr and included measures additional to those reported here. Children were paid $10 for their participation.

I can certainly email the entire study in pdf to anyone who might be interested as well.

[buffalopc7]

Originally Posted by Hyperion

Yep. I agree 100%. The scientific method is not perfect, but it is the best method available. Unless you have a better suggestion...

You're absolutely right, the scientific method isn't perfect by any means, but it is all we have at the moment and it would be difficult to even suggest a way to take subjectivity out of research, since humans report it. My point wasn't a flaw in the study, but the fact that the researchers interjected their own inferences as to what the results meant. If you know anything about scientific research, you are supposed to present the data, not make assumptions or suggestions as to what it means. I am, however not naive enough to think this isn't common practice, and i've worked on enough studies to know that that practice is "encouraged" in order to perpetuate a belief or argument. The most disturbing thing is that the media garner information from research studies and often present the researcher's inferences as factual (or make up their own, which is also questionable practice). Do you see the implications of this? I merely used this particular study as an example, because I felt it was fairly blatant in its inference of the meaning of results.

[Hyperion]

Yeah, trust me, I'll rip apart a study if I think the authors have their heads up their asses (just look at my post on the craptacular ADHD-Lead study).

In this study, I don't see it as being that bad. I think that there might be other explanations for the behavior, but I don't think that their conclusions are completely unsupported. They had ADHD and regular kids watch a video and then asked them about events in the video, then they showed them another video with toys in the room and then asked the kids about the video. The room with toys drastically reduced the ADHD kids' ability to pay attention to the video.

My objection in the post you quote was aimed more at SB's criticism of the scientific method itself. I found it rather passe, given that everyone knows the limitations of scientific research, as many of those limitations are by design as they serve to delineate between that which is scientific and that which is not.

Often, people object to the scientific method as a means of introducing non-scientific evidence and attempting to give it the same weight.

[buffalopc7]

Originally Posted by Hyperion

Yeah, trust me, I'll rip apart a study if I think the authors have their heads up their asses (just look at my post on the craptacular ADHD-Lead study).

In this study, I don't see it as being that bad. I think that there might be other explanations for the behavior, but I don't think that their conclusions are completely unsupported. They had ADHD and regular kids watch a video and then asked them about events in the video, then they showed them another video with toys in the room and then asked the kids about the video. The room with toys drastically reduced the ADHD kids' ability to pay attention to the video.

My objection in the post you quote was aimed more at SB's criticism of the scientific method itself. I found it rather passe, given that everyone knows the limitations of scientific research, as many of those limitations are by design as they serve to delineate between that which is scientific and that which is not.

Often, people object to the scientific method as a means of introducing non-scientific evidence and attempting to give it the same weight.

I agree with you, their conclusion is definitely a possibility, however they chose a conclusion that I believe perpetuates a belief regarding children with ADHD which may not necessarily be accurate. To assume the children simply didn't comprehend the premise of the tv show is just one possible explanation. I know for myself, if you placed chocolate around a tv set while I was watching a show, i'd be hard pressed to tell you what the darn show was about. Am I therefore unable to comprehend? I think perhaps its more that I didn't pay attention, and therefore didn't have enough exposure (by my choice, admittedly) to the information to accurately understand the premise of the show. I'd have less of a problem if the researchers had suggested several possible reasons for the findings. In all honesty, not knowing these children, none of us really know why the children with ADHD were less accurate in answering the questions and that is (supposedly) the reason that we, as researchers, are not supposed to infer/suggest causality. Unfortunately, there is a common tendency to do so, which takes from the reader, their own ability to objectively review a study.