Ex-Gay Project Fails
September 28th, 2011
The National Organization “for” Marriage is trumpeting a new survey about changing your sexual orientation, based on “religiously-mediated” involvement in Christian conversion ministries:
Many professional voices proclaim that it is impossible to change homosexual orientation, and that the attempt to change is commonly and inherently harmful…
The results show change to be possible for some, and the attempt not harmful on average.
I don’t have access to the full survey, but the researchers have summarized the results. Here’s how it turned out after 6 to 7 years of tracking participants:
|98||The number of subjects when the study began: 72 men and 26 women.|
|37||The number of subjects who dropped out of the study and did not report their results. I don’t think we can count any of them as an ex-gay “win.”|
|12||The number who stopped trying to change, and embraced their gay identity.|
|18||The number who were reportedly chaste, “with substantive dis-identification with homosexual orientation.”
In other words, they’ve managed to stop having sex, and don’t think about getting same-sex down-and-dirty as much as they used to.
This isn’t a change in orientation, any more than a straight person is no longer straight because they’ve used prayer to become celibate and partially push some of their sexual feelings underground.
|17||The number who apparently stayed with the study, but whose outcomes are not described in the summary. Presumably, they’re not clear ex-gay “wins” either.|
|14||The number who reported “successful ‘conversion’ to heterosexual orientation and functioning.”|
That’s a meager 14% success rate.
Or is it? Let’s learn more about those 14 people.
First, it’s probably not 14. The study cautions us that the 14 conversions and 18 celibates represent “likely overly optimistic projections of anticipated success.”
In other words — less than 14 actual conversions.
But wait. Check out what “conversion” means:
Most of the individuals who reported that they were heterosexual at Time 3 did not report themselves to be without experience of homosexual arousal, and did not report heterosexual orientation to be unequivocal and uncomplicated.
You know what they call straight people who experience homosexual arousal, and whose orientation is at most equivocal? Bisexual. Most of the 14 heterosexual “conversions” seem to be bisexuals.
This leaves us with at most — at most — 6 individuals who went from gay to straight (as of now, at least; who knows where they’ll be in another 7 years).
And the authors aren’t willing to go even that far. Their single-sentence summary:
In short, the results do not prove that categorical change in sexual orientation is possible for everyone or anyone, but rather that meaningful shifts along a continuum that constitute real changes appear possible for some.
Wow. Out of 98 highly-motivated subjects, the authors found that a small, unspecified number can use prayer and counseling to shut down their sexual feelings or become a bit more bi. And possibly none who turned straight.
Frankly, I’m surprised they couldn’t find more. The authors claim their results:
…challenge the commonly expressed views of the mental health establishment that change of sexual orientation is impossible or very uncommon…
Actually, it looks more like the results confirm those views. If the antigay camp sees this as vindication and victory, they must be even more desperate than I thought.
UPDATE: I’m abashed to say that Timothy Kincaid has already covered this here at Box Turtle Bulletin, and in far more depth. You can check out his series of articles here.
Dr. Stanton Jones Replies
September 18th, 2007
Dr. Stanton Jones, co-author of Ex-Gays? A Longitudinal Study of Religiously Mediated Change in Sexual Orientation, responded to my preliminary review in the comments. I wanted to elevate his comments to another post for visibility and to start a new comment thread. I’d like to thank Dr. Jones for sharing his thoughts. I ask that all commenters treat the subject matter with similar civility. Thank you.
Dear Mr. Burroway and the readers of the Box Turtle Bulletin:
Some brief reflections on your review (with a quick added note that we will not be responding much in this type of format; the demands are simply too great):
1. Above all else, thank you for a substantive and thoughtful preliminary review that actually engages the factual, scientific and intellectual issues of the study instead of engaging in the types of character assassination and other negative tactics of some commentators. While I obviously disagree with your conclusion that “we’re still left waiting for that definitive breakthrough ex-gay study. I don’t think this one is it,” you have nevertheless made clear the grounds on which you have made this judgment, have discussed those grounds rationally, and those grounds are areas of legitimate debate.
2. There seems to be considerable confusion about the book not being available until October. The book is in fact available now; it was released the same day as the paper presentation in Nashville. Quite a number of your questions are answered in the full book presentation; the paper is a sampling of the findings, with an emphasis in the paper (admittedly) on the clearer findings that led us to our conclusions. Now to four substantive issues you raise:
3. Psychophysiological measurement: You ask about our failure to use MRI technology. First, we report in this paper and book on the data gathered up to two years ago, and the MRI technology was unavailable then. Second, even today I would not use it. The “No Lie MRI” appears to be just the latest hyped and unproven version of the hope for a foolproof lie detector. A recent New Yorker article by Margaret Talbot (“Duped: Can Brain Scans Uncover Lies?,” July 2, 2007) presents a readable and thorough discussion of just how overly-hyped and poorly validated this new technology is. There is inadequate scientific basis to believe the “No Lie MRI” would be a suitable measure for our subjects.
4. Retention: You put quite a bit of emphasis on our drop-outs and chided our efforts to contact and assess drop-outs without benefit of reading our account of this in the book. You particularly compare us unfavorably to the Add Health Study. Remember that the Add Health Study was a study of adolescents with parental approval, so those researchers were remarkably advantaged in terms of being able to track down missing participants through their families. We had no such advantage with an adult population. We went to extreme lengths to keep people in the study, involving multiple pleas and contacts including those through families and friends, and, when we had contact information at all, with personal calls and pleas from me. At some point, you must respect people’s wishes not to be contacted. We remain proud of our retention rate in the study.
5. Representativeness of sample: You say “I think at least one demographic variable they provided [i.e., age] is ample evidence that their sample is not representative.” To the best of my knowledge, there is only one study that can come close to the claim of getting a representative sample in this area, and that is the Bailey, Dunne, & Martin Australian twin study (Journal of Personality and Social Psychology, 2000) that concluded that genetic factors were not a statistically significant contributor to causation; this is a great sample because it was an exhaustive sample of every twin born in Australia! (Update: Dr. Jones has more on the Australian twin study here.) At some level, the ultimate representativeness of all other samples is debatable. The evidence you cite in your dismissal is a good example of our difficulty: You state your impression that Exodus conferences draw a younger crowd while our sample age is older, and that therefore that we have a bad sample. But you have provided no justification that your impressions of Exodus conference ages are valid, nor stated the bases for your inference that conference attenders are in turn representative of those who seek help in local Exodus ministries. We do acknowledge that we cannot prove the representativeness of our sample, but have numerous reasons, cited in the book, for why we think we got something like a representative sample. And one final note: samples have to be adequate to the task of the study, and we discuss in the book the reasons why the sample is adequate to test an absolute hypothesis (that change is impossible) and a strong hypothesis (that the change attempt is often and decisively harmful).
6. Truly prospective: We explicitly discuss in the book the implications of including the Phase 2 group. I would soften your criticism of the retrospective implications of this group, because in contrast to the Spitzer study where subjects could be looking back over many years or decades to remember their prior experience, our Phase 2 group were looking back into their immediate past. More importantly, however, it is vital to note that while the average changes noted were less strong for the Phase 1 group, Phase 1 subjects were proportionally represented in all of our success categories, so an easy dismissal of significant changes in this group is simply not possible.
Again, it was a delight to read a thoughtful review of our study. No study is perfect. To argue that ours is the strongest study yet done in this area is not the same as to argue that it is exemplary or perfect. It is a stronger study than any available, one particularly suited to the hypotheses investigated.
Stanton L. Jones
A Preliminary Review of Jones and Yarhouse’s “Ex-Gay? A Longitudinal Study”
September 17th, 2007
Last Thursday, Stanton Jones and Mark Yarhouse announced the results of their new ex-gay study at a press conference in Nashville, where the American Association of Christian Counselors was holding its annual conference. The study, Ex-Gays? A Longitudinal Study of Religiously Mediated Change in Sexual Orientation will be published by InterVarsity Press October 10. This review is based on a thirteen-page synopsis that was provided for the AACC.
This study was funded by Exodus, with Jones and Yarhouse promising that “we would be reporting publicly the results of our outcome study regardless of how encouraging or embarrassing Exodus might find those results.” Based on Exodus’ press release and Alan Chambers’ presence at the press conference, it appears that Exodus is quite pleased with the study. Exodus, by the way, chose that same weekend to host their regional conference in Nashville, making for a very well coordinated event.
The Study’s Design
According to Jones and Yarhouse, their study was intended to answer two questions: Is change in sexual orientation possible, and are attempts to change harmful? And to answer those questions, they set out to do something that hadn’t been done before. They constructed what’s called a longitudinal or prospective (i.e. forward-looking) study, where they followed a population of study participants as they were beginning their experience with ex-gay ministries and continued to follow them over a period of four years.
This is an important feature of the study. One of the many criticisms for Robert Spitzer’s 2003 ex-gay study was that it was a retrospective (i.e., backwards-looking) study. In other words, study participants were asked to remember back to before they began their attempts to change and report from memory their sexual orientation and attractions. Jones and Yarhouse chose to conduct a prospective study instead:
In contrast to retrospective methods that ask participants to remember change experiences that happened in their pasts, a prospective methodology begins assessment when individuals are starting the change process and assesses them as the results unfold.
Because it’s longitudinal, beginning as the ex-gay participants begin their own journey, there’s no reliance on possibly faulty memories when asked, “What were your attractions several years ago when you started?”
When the study began, participants undertook a number of interviews face-to-face. This was at Time 1. There were two more interviews, Time 2 and Time 3, with the span between Time 1 and Time 3 being between thirty months to four years. Most of the Time 2 interviews were conducted face to face (15% were over the phone) and all of the Time 3 interviews were done over the phone. For all three, Jones and Yarhouse used a number of recognized, standardized measures for sexuality and mental health, with the crucial self-reports of sexuality being conducted via mail-in questionnaires.
Another weakness with Robert Spitzer’s study was that he didn’t use any standard measures for sexual orientation. He didn’t use the Kinsey scale (where 0= completely heterosexual and 6 = completely homosexual), nor did he use the Shively-DeCecco scales (which separates the intensity of homosexual and heterosexual attractions on two separate scales for independent measurement, with the zero axis representing perfect asexuality). Jones and Yarhouse used both sets of scales for their analysis, using commonly recognized standardized questionnaires to determine ratings at Time 1, Time 2 and Time 3.
Probably the weakest link in the design was in relying on self-reports for assessing sexual orientation instead of physiological measures of arousal. They addressed those criticisms this way:
Psychophysiological measures assess sexual arousal and orientation by attaching sensors to the genitals of subjects and measuring sexual arousal while the subjects watch pornography. We judged these methods as pragmatically impossible given the dispersed nature of our sample and the limitations of our funding, as morally unacceptable to the bulk of our research participants, and as not justified in light of current research challenging the reliability and validity of the methods themselves.
These are all legitimate objections as far as penile and vaginal plethysmography are concerned. There are however new emerging technologies involving MRI’s which may be useful for future studies.
Difficulty In Recruiting Participants
While Jones and Yarhouse’s study appears to be very well designed, it quickly falls apart on execution. The sample size was disappointingly small, too small for an effective retrospective study. They told a reporter from Christianity Today that they had hoped to recruit some three hundred participants, but they found “many Exodus ministries mysteriously uncooperative.” They only wound up with 98 at the beginning of the study (72 men and 26 women), a population they describe as “respectably large.” Yet it is half the size of Spitzer’s 2003 study.
Jones and Yarhouse wanted to limit their study’s participants to those who were in their first year of ex-gay ministry. But when they found that they were having trouble getting enough people to participate (they only found 57 subject who met this criteria), they expanded their study to include 41 subjects who had been involved in ex-gay ministries for between one to three years. The participants who had been in ex-gay ministries for less than a year are referred to as “Phase 1” subpopulation, and the 41 who were added to increase the sample size were labeled the “Phase 2” subpopulation.
This poses two critically important problems. First, we just saw Jones and Yarhouse explain that the whole reason they did a prospective study was to reduce the faulty memories of “change experiences that happened in their pasts” — errors which can occur when asking people to go back as far as three years to assess their beginning points on the Kinsey and Shively-DeCecco scales. This was the very problem that Jones and Yarhouse hoped to avoid in designing a prospective longitudinal study, but in the end nearly half of their results ended up being based on retrospective responses.
This diluted the very purpose of doing a longitudinal study, and as Jones and Yarhouse describe it, this also clearly affected the results:
We expected that the results of change would be somewhat less positive in this group (phase 1), as individuals experiencing difficulty with change would likely be somewhat less positive in this group, as individuals experiencing difficulty with change would be likely to get frustrated or discouraged early on and drop out of the change process. We were able to retain these Phase 1 subjects in our study at the same rate as the whole population, and indeed found that change results from them were a bit less positive.
Left unsaid but clearly implied is the second problem with adding Phase 2 participants. Since they had already hung in there for between one and three years, that subpopulation would not have included those who entered ex-gay ministries at the same time they did but who were discouraged early on and dropped out. It’s no wonder the change results for Phase 1 were less positive than Phase 2. There’s no indication how “less positive” those results were, not in this synopsis anyway. Hopefully the book will break these numbers out.
But in the synopsis at least, the study’s results appear to combine Phase 2 and Phase 1 participants, which represents an unacceptable mixing of prospective (Phase 1) and retrospective (Phase 2) participants. And since the Phase 2 participants make up nearly half the total sample, this ruins any chance of this being a truly prospective study.
Whenever a longitudinal study is being conducted over a period of several years, there are always dropouts along the way. This is common and to be expected. That makes it all the more important to begin the study with a large population. Unfortunately, this one wasn’t terribly large to begin with; it started out at less than half the size of Spitzer’s 2003 study. Jones and Yarhouse report that:
Over time, our sample eroded from 98 subjects at our initial Time 1 assessment to 85 at Time 2 and 73 at Time 3, which is a Time 1 to Time 3 retention rate of 74.5%. This retention rate compares favorable to that of the best “gold standard” longitudinal studies. For example, the widely respected and amply funded National Longitudinal Study of Adolescent Health (or Add Health study reported a retention rate from Time 1 to Time 3 of 73% for their enormous sample.
The Add Health Study Jones and Yarhouse cite began with 20,745 in 1996, ending with 15,170 during Wave 3 in 2001-2002. But this retention rate of 73% was spread over some 5-6 years, not the three to four years of Jones and Yarhouse’s study.
What’s more, the Add Health study undertook a rigorous investigation of their dropouts (PDF: 228KB/17 pages) and concluded that the dropouts affected their results by less than 1 percent. Jones and Yarhouse didn’t assess the impact of their dropouts, but they did say this:
We know from direct conversation that a few subjects decided to accept gay identity and did not believe that we would honestly report data on their experience. On the other hand, we know from direct conversations that we lost other subjects who believed themselves healed of all homosexual inclinations and who withdrew from the study because continued participation reminded them of the very negative experiences they had had as homosexuals. Generally speaking, as is typical, we lost subjects for unknown reasons.
Remember, Jones and Yarhouse described those “experiencing difficulty with change would be likely to get frustrated or discouraged early on and drop out of the change process.” And so assessing the dropouts becomes critically important, because unlike the Add Health study, the very reason for dropping out of this study may have direct bearing on both questions the study was designed to address: Do people change, and are they harmed by the process? With as much as a quarter of the initial population dropping out potentially for reasons directly related to the study’s questions, this missing analysis represents a likely critical failure, one which could potentially invalidate the study’s conclusions.
Representativeness of Study Participants
Jones and Yarhouse describe their sample’s representativeness in contradictory and confusing terms:
Our study examines a representative sample of the population of those in Exodus seeking sexual orientation change. We cannot be absolutely certain of perfect representativeness, since no scientific evidence exists for describing the parameters of such representativeness. Still, we are confident that our participant pool is a good snapshot of those seeking help from Exodus.
Did you get that? It’s representative, but they can’t prove it. But they’re confident anyway.
When researchers make a sweeping statement, especially one as important as representativeness, they bear the burden of providing evidence to support their claim. If they can’t do that, then they must instead caution that their sample may not be representative and list the reasons why. I don’t think a respected peer-reviewed journal would let Jones and Yarhouse get by with claiming representativeness with nothing to substantiate that claim.
Their synopsis doesn’t describe how members were recruited into the study, so we can’t judge what selection biases may occur during recruitment. (I’m sure the book addresses some of this — I’d be shocked if it didn’t.) Nor do they discuss how their demographics might compare with other measures for ex-gay ministry participant populations. There are conferences, rosters, or simple surveys of ex-gay ministry leaders that they could have culled and compared their demographic data with.
But they didn’t appear to have done this, not according to the synopsis anyway.
But I think at least one demographic variable they provided is ample evidence that their sample is not representative. For example, they said that the average age of their sample was 37.50 years. Having asserted that their sample was “fairly representative,” they extrapolated that to the entire Exodus population this way:
The average age was older than we had expected, and its significance should be underscored. There is an unflattering caricature that Exodus groups appeal primarily to young, naïve, confused and sexually inexperienced individuals.
In this statement, Jones and Yarhouse appear to be more interested in defending Exodus’ reputation than in defending their own sample. But when I attended the Exodus Freedom Conference in Irvine California in June 2007, I got the distinct impression that the average age of the 800 participants was well under 37 years — perhaps even under thirty. The median age of “strugglers” was certainly close to thirty. The conference audience definitely skewed quite young.
We should keep in mind that Jones and Yarhouse limited their study sample to those over eighteen; their youngest participant ended up being twenty-one. Exodus, on the other hand, allowed registrants for their annual conference to be as young as thirteen, although I don’t think I saw anybody that young there. I did see a large number of teenagers, and an extraordinary number of young people, largely under thirty. Exodus had special programs set aside at that conference for younger people which old fogies like me weren’t allowed to attend. Exodus even operates an entire ministry called Exodus Youth, headed by Scott Davis, which specifically targets young people of high school and junior high ages. The Love Won Out ex-gay conferences also conduct several workshops for youth. And again, some of these workshops are closed to older adults.
Jones and Yarhouse may have had good ethical and methodological reasons for limiting their study to those above the age of eighteen. There are issues of informed consent, and questions would undoubtedly arise as to whether youth who are still under their parents direction would feel free to answer questions truthfully. But by limiting the study to those above the age of eighteen, Jones and Yarhouse guaranteed that their study would not be representative of Exodus participants overall.
Results – Is Change Possible?
As Timothy Kincaid already reported, the breakdown of the quantitative results went this way:
- 33 people reported change (moving from homosexual, bisexual or other at Time 1 to heterosexual at time 3; or homosexual at Time 1 to bisexual or other at Time 3)
- 29 reported no change
- 8 reported “negative change” (moving from heterosexual, bisexual or other at Time 1 to homosexual at Time 3; or from heterosexual at Time 1 to bisexual or other at Time 3).
- 3 reported uncertain change (moving from bisexual to other, or the reverse)
Keep in mind however that these results mix the truly prospective participants (Phase 1 participants who who began the study during their first year in Exodus ministries) and the retrospective participants (Phase 2 participants who had been in ministries for between one and three years). We don’t know what the mix of these two subpopulations are in the results. Since Jones and Yarhouse already stated that reported change from the prospective phase 1 group were “a bit less positive,” we know the results aren’t the same. But unless we understand how Phase 1 fared, we don’t know how mixing in people who were asked what their beginning orientation was retrospectively affected the results.
These results were derived using standardized measures using Kinsey and Shively-DeCecco scales. And the the Shively-DeCecco scales (remember, this separates homosexual attraction and heterosexual attractions on two separate axis), revealed something particularly interesting:
Changes on the Shively and DeCecco ratings for all three of our analysis followed a stable pattern… We see that change away from homosexual orientation are consistently about twice the magnitude of changes toward heterosexual orientation. It would appear, then, that while change away from homosexual orientation is related to change toward heterosexual orientation, the two are not identical processes. The subjects appear to more easily decrease homosexual attraction than they increase heterosexual attraction. [Emphasis in the original]
In many ways this confirms what many opponents of ex-gay therapy have noted, that attempting to change sexual orientation does not necessarily make someone straight. In fact, this particular finding makes it all the more unlikely, and puts into context Jones and Yarhouse’s characterization of success as “satisfactory, if not uncomplicated, heterosexual adjustment.”
This also, I think, goes a long way toward describing something else. It is often assumed that those who reported the most change were probably bisexual to some degree when starting the change process. To test that theory, Jones and Yarhouse created a subpopulation from their sample that, for want of a better term, they dubbed “The Truly Gay”:
… [T]o be classed as truly gay, subjects must have reported above average homosexual attraction and reported homosexual behavior and reported past embraced of a gay identity. We would emphasize that these were much more rigorous standards than are typically employed in empirical studies to classify research subjects as homosexual. Using this method, 45 out of our total 98 subjects were classed as “Truly Gay,” just less than half the population sample. We expected that the results of change for the Truly Gay subpopulation would be less positive, as they individuals would be those more set and stable in their sexual orientation. This is not what we found. Rather the change reported by the Truly Gay subpopulation was consistently stronger than that reported by others.
It’s unclear to me what they meant by “above average homosexual attraction” in their definition for the “Truly Gay.” Most researchers consider only Kinsey 5’s and 6’s to be “truly gay.” It’s not clear that this is what Jones and Yarhouse did here. By saying “above average homosexual attraction,” do they mean above average for this sample? If so, what was that cut-off? Maybe the book will clear things up. We’ll see.
But let’s assume for a moment that their criteria is valid, and let’s look at this in light of what they noticed about change to begin with: A change away from homosexual attractions at a rate that is about twice the rate of change toward heterosexual attraction. When looked at it this way, it is possible that the “stronger change” for the “Truly Gay” subpopulation was possible simply because there was a greater potential travel along the Kinsey or Sively-DeCecco scales to begin with; many bisexuals would have begun their attempts to change already partway down those paths. And since overall, the best functioning was “satisfactory, if not uncomplicated, heterosexuality,” it appears that for both groups, there was a finite limit short of Kinsey 0 or 1 that few in either group approached.
Qualitative Analysis of Change
So far, we’ve talked about statistical measures of change based on Kinsey and Sively-DeCecco scales. Jones and Yarhouse also described some qualitative analysis, based on open-ended questions about participants’ attractions, experiences and identity. Those results were:
- “Success: Conversion”: There were subjects who reported that they felt their change to be successful and reported substantial reduction in homosexual desire and addition of heterosexual attraction and functioning. 15% (11 of 72) at Time 3 met this standard.
- “Success: Chastity”: These were subjects who reported that their change was successful, and who reported homosexual attraction to be present only incidentally or in a way that does not seem to bring about distress, allowing them to live happily without overt sexual activity. 23% (17 of 72) at Time 3 met this standard.
- “Continuing”: These persons may have experienced modest decreases in homosexual attraction, but were not satisfied with their degree of change and remained committed to the change process. 29% (21 of 72) at Time 3 met this standard.
- “Non-response”: These people experienced no significant change. They had not given up on the change process, but may be confused or conflicted about which direction to turn next. 15% (11 of 72) at Time 3 were in this category.
- “Failure: Confused”: These persons had experienced no significant change and had given up on the change process but without yet embracing gay identity. No change reported and had given up but did not label themselves gay. 4% (3 of 72) at Time 3 were in this category
- “Failure: Gay identity”: These persons had clearly given up on the change process and embraced a gay identity. 8% (6 of 72) of the sample at Time 3 were in this category.
To further understand what all this means, it would be important to know how the dropouts might have affected these results. As we mentioned earlier, the Adolescent Health study (which Jones and Yarhouse upheld as a “gold standard”) made a concerted effort to understand how their dropouts might have affected the results. In doing so, they discovered that fewer than 5% dropped out because of refusal to continue. With that and other information at hand, they were able to determine that their dropouts affected the results by no more than a single percentage point.
Jones and Yarhouse appear to show no similar curiosity, and this represents a very significant failing of their study. In fact, the dropouts might have contributed very significantly towards higher “failure” numbers. But since Jones and Yarhouse appear to be incurious to find out more about this group, we are left in the dark.
Outcomes for Harm
Jones and Yarhouse administered the System Check List-90-Revised (SCL), which they describe as “a respected measure of psychological distress that is often used to measure the effects of psychotherapy.” They report no difference in the SCL scores from Time 1 to Time 3 when compared to others who are undergoing outpatient counseling.
But again, what about the dropouts? Did they report higher SCL scores at Time 1 or Time 2 before dropping out? We don’t know, not from the synopsis anyway. Again, maybe the full book will provide more details. But without this critical information to understand how the dropouts might have affected the results, Jones and Yarhouse cannot confidently conclude that attempting to change produced no harm. At best, they can only conclude that there was no greater degree of distress among those who continued ex-gay therapy when compared to mentally distressed persons undergoing psychological counseling for other issues — and by the way, is that really a legitimate comparison? I think it’s debatable. At any rate, where they chose to look, there was no problem. Where they chose not to look, who knows?
From Jones and Yarhouse’s synopsis of their study, I have a few more questions than answers. Hopefully I’ll get a copy of the full report in a few days. If so, I’ll post a more complete review as time permits. Until then, consider this review a preliminary one.
I’d have to say that I was very impressed with the study’s design, and very disappointed in its execution. Seventy-two participants out of Alan Chambers much-repeated “thousands” or “tens of thousands” doesn’t impress me much. I’m especially disappointed with these particular weaknesses:
- Jones and Yarhouse’s insistence that the study is representative of Exodus participants is completely without merit. If Jones and Yarhouse feel free to make such a sweeping claim with no data to support it, one wonders what other sweeping claims they may have made.
- Jones and Yarhouse’s apparent incuriosity towards those who dropped out borders on willful ignorance. Maybe the full book will provide better information in this area, but the synopsis leaves the impression that unlike the Add Health study that they admired, they didn’t try to learn what those dropouts might mean for their results.
- Jones and Yarhouse’s inclusion of those who had been in Exodus member ministries for between one and three years — and having that group making up nearly half of the study — makes a significant chunk of what was supposed to be a prospective study a retrospective one instead. And it misses those who “failed” out of that Phase 2 group before they had a chance to join the study. This is a particularly sloppy failing that most certainly biased the results in favor of more “successes” and fewer “failures.”
We’ve waited quite a long time for a better study than Robert Spitzer’s 2003 effort. This study held great promise based on its initial design, but its conduct left much to be desired. Its rigorous design was not matched by similar rigor in execution. And so we’re still left waiting for that definitive breakthrough ex-gay study. I don’t think this one is it.
Update: Stanton Jones Responds
Exodus Press Release Spins Study
September 15th, 2007
Yesterday, I made the mistake of promising a detailed review of the synopsis from Stanton Jones and Mark Yarhouse of their new ex-gay study. Little did I know that unexpected company would be dropping in this weekend. So watch this space for my review sometime Monday.
Meanwhile, Exodus is pleased as punch about Jones’ and Yarhouse’s ex-gay study:
Alan Chambers, a former homosexual and President of Exodus International, responded to the study findings at today’s press conference, “Finally, there is now scientific evidence to prove what we as former homosexuals have known all along – that those who struggle with unwanted same-sex attraction can experience freedom from it.”
Alan Chambers’ sweeping generalization “that those who struggle.. can experience freedom” isn’t supported by the study. At best, a few (namely, eleven out of this sample of seventy-three) found freedom — that is, if freedom means “satisfactory, if not uncomplicated, heterosexual adjustment” as Jones and Yarhouse so carefully put it. Another seventeen decided that celibacy was good enough. Not what I’d call freedom, but hey — different strokes, right?
Update: Here’s the review I promised: A Preliminary Review of Jones and Yarhouse’s “Ex-Gay? A Longitudinal Study”
Limited Available Information on Study has Confusing Numbers
September 14th, 2007
Dr. Warren Throckmorton is reporting some of the findings from the Yarhouse and Jones book. It appears that the study was over four years and included 98 people who were referred by various Exodus ministries.
- 33 people reported change in the desired manner (from gay at time 1 in the heterosexual direction at time 3)
- 29 reported no change
- 8 reported change in the undesired direction
- 3 were unsure how to describe their experience of change
and 25 people discontinued participation in the study during that time. The study also reports:
- Success: Conversion – There were subjects who reported that they felt their change to be successful and reported substantial reduction in homosexual desire and addition of heterosexual attraction and functioning at Time 3. 15% met these criteria.
- Success: Chastity – These people experienced satisfactory reductions in homosexual desire and were living chaste lives. 23% were in this category.
- Continuing – These persons experienced only modest change in the desired direction but expressed commitment to continue. 29% were in this category.
- No-response – These people experienced no change and were conflicted about the future even though they had not given up. 15% were here.
- Failure (from their perspective): Confused – No change reported and had given up but did not label themselves gay. 4% were in this group
- Failure: Gay identity – No change, no pursuit and had come as gay. 8% were in this category.
Assuming that these are percentages of the 73 participants who made it to the fourth year, this would break out as follows:
- Success: Conversion – 11
- Success: Chastity – 17
- Continuing – 21
- No-response – 11
- Failure: Confused – 3
- Failure: Gay identity – 6
With four people left unaccounted for.
Try as I might, I can’t get these two findings to reconcile. Did 33 people report a change in the positive direction, or did 28? Did 8 people identify as gay or did 6?
We will have to wait for Jim’s analysis of the book for better answers. At present, we can only conclude that, at best:
Perhaps eleven percent of an nonrepresentative sample of 98 highly motivated gay people who went through Exodus programs reported that after four years there was “substantial reduction in homosexual desire and addition of heterosexual attraction and functioning”.
Christianity Today provides further clarification on those eleven successes.
Most of the individuals who reported that they were heterosexual at Time 3 did not report themselves to be without experience of homosexual arousal, and did not report heterosexual orientation to be unequivocal and uncomplicated. … We believe the individuals who presented themselves as heterosexual success stories at Time 3 are heterosexual in some meaningful but complicated sense of the term.
These sound less like Mom and Dad heterosexuals and more like Larry Craig heterosexuals. In other words, the number of individuals who went from plain old gay to plain old straight: zero. Not an overwhelming success story, I’m inclined to think.
Jones’ and Yarhouse’s Ex-Gay Study Released
September 14th, 2007
The results of Stanton Jones’ and Mark Yarhouse’s Exodus study were released yesterday. I have a synopsis of that study that I’m reviewing now. Short take: The methodology seems to be pretty good, (I’m especially happy to see them use standardized measures for sexuality, something that Spitzer didn’t do).
The greatest weakness is its small sample size, and I’m concerned about the cohorts that they added to the study to try to beef it up. They originally wanted to study Exodus participants who were in their first year of attending an Exodus member ministry, but when they couldn’t find enough subjects they added a cohort of strugglers who had been in the ministry for one to three years. I don’t think there’s anything nefarious going on there, but so far I can’t sort out how those cohorts affected the results. I think that for what’s supposed to be a longitudinal study, this is critically important. I also question their claims that their sample is representative of Exodus participants. So far as I can tell, they fail to justify that claim.
These are just first quick impressions from the synopsis. I’ll have more for
tomorrow morning late tomorrow afternoon. The study itself is a 375-page book. I’m trying to obtain an advance copy, and when I do it will obviously take a great deal of time to go through it.
Jones and Yarhouse: “The Use, Misuse and Abuse of Science in the Ecclesiastical Homosexuality Debates”
September 13th, 2007
So today is when Stanton L. Jones and Mark Yarhouse are set to release the results of their ex-gay study in Nashville at the American Association of Christian Counselors World Conference. The study itself, Ex-Gays? A Longitudinal Study of Religiously Mediated Change in Sexual Orientation, will be published in book form by InterVarsity Press, but won’t be publicly available until October 10. This study will reportedly will show whether a significant percentage of gays can change their sexual orientation through religious-based counseling.
Since this study went to a commercial Christian publisher instead of a peer-reviewed journal, we have no a priori assurance that it has been vetted by qualified professionals. We can’t rely on IVP Press to to that vetting; they’ve also agreed to publish Richard Cohen’s forthcoming book despite his many embarrassments to the ex-gay movement. And we can’t rely on George Rekers, who praised the study in the IVP press release. He was similarly impressed with Paul Cameron’s Research.
So until we’re able to obtain a copy to evaluate for ourselves, it’ll be difficult to judge its methodology or conclusions. But a glimpse into Jones’ and Yarhouse’s previous collaborations might give us an idea of what to look for.
Jones and Yarhouse have collaborated at least three times before. They wrote “The Use, Misuse and Abuse of Science in the Ecclesiastical Homosexuality Debates,” which appeared in the 2000 anthology Homosexuality, Science, and the “Plain Sense” of Scripture (edited by David L Balch and published by Eerdmans). That chapter was based largely on an earlier article they wrote for the Christian Scholar’s Review in 1997 titled “Science and the Ecclesiastical Homosexuality Debates.” They also contributed a chapter titled “The Homosexual Client” in the 1997 anthology Christian Counseling Ethics (edited by R.K. Sanders and published by InterVarsity).
In “The Use, Misuse and Abuse of Science,” Jones and Yarhouse avoid most of the pitfalls which characterize much of the anti-gay cannon. The tone is considerably more moderate than most of the literature. There’s virtually no demonizing or name-calling, they stick to the more reputable researchers and their findings (Paul Cameron need not apply), they portray pro-gay arguments with reasonable accuracy and avoid the typical strawman arguments that are all too common elsewhere.
But “The Use, Misuse and Abuse of Science” is not an unbiased chapter, and I don’t think Jones and Yarhouse would pretend otherwise. When Homosexuality, Science, and the “Plain Sense” of Scripture” was released in 2000, they commented:
We (Stan and Mark) have watched for years as the supposed “scientific evidence” has been used in the ethical/moral debates of the various Christian denominations over the divisive topic of homosexuality. The majority of the time, the “evidence” has been used against the traditional moral position that sees homosexual behavior as sin.
Their examples of “misuse and abuse” of science only extended to the pro-gay side of the debates. They were strangely silent on the many ways silence on the many countless ways in which science is widely abused by the anti-gay side — abuses so widespread that debunking them can take up several entire web sites. (Ahem!)
Prevalence of Homosexuality
In “The Use, Misuse and Abuse of Science,” Jones and Yarhouse undertook four specific areas for review. The first topic for review was of studies determining the the prevalence of homosexuality. This is an important point in anti-gay circles, with anti-gay advocates prefering much lower prevalence figures for many reasons. In simple terms, the lower the number, the less “normal” homosexuality becomes. They especially don’t like Kinsey’s famous ten-percent statistic — a statistic which most of the more prominent pro-gay advocates have abandoned quite some time ago.
Jones and Yarhouse perform a thorough debinking of Kinsey’s statistic, and in truth most of those criticisms are perfectly valid. For one, Kinsey didn’t use probability sampling techniques. He also was doing something that had never been done before: undertaking a wide-ranging survey of American attitudes about sexuality. Begun in the 1930’s, his research was groundbreaking. But as is the case with most pioneers with no prior precedence to guide them, “mistakes were made.” Kinsey was certainly no exception.
Jones and Yarhouse then reviewed a half dozen surveys which were published before 1995, all with low prevalence figures. But they didn’t discuss the many factors which contribute to low numbers. Homosexuality then, even moreso than now, was highly stigmatized (AIDS was in full rage, “Don’t Ask Don’t Tell” had been written into law by a Democratically-controlled Congress in 1993, DOMA passed both houses of Congress with wide bipartisan support in 1996.). Also, because of small sample sizes, margins of error were often wide, and refusal rates — the percentage of people who refused to participate for whatever reason — hovered in the 20-35% range. That’s why responsible researchers routinely caution that results reflect the low range for homosexuality.
But not Jones and Yarhouse. True to form, they chose a low-ball figure:
So when the genders are combined, homosexuality almost certainly characterizes less than 3 percent of the population, and the correct percentage combining men and women might be lower even than 2 percent.
What Causes Homosexual Orientation?
The second area Jones and Yarhouse reviewed was the several biological and environmental theories for the causes of homosexuality. Again, no surprises here. The biological theories were dissected in great detail and largely dismissed for their obvious weaknesses. And in truth, the state of research as of 2000 was indeed not terribly strong — especially if one were to assume that there will ever be a single cause for something as complex as human sexuality, an assumption that is essential in order for their grounds for dismissal to be taken seriously. (Ironically, it is also an assumption they disavow just a little later in the chapter.)
In contrast, Jones and Yarhouse were rather protective of psychological theories — distant father, absent parent, molestation, you get the idea. While they expended a great deal of effort to debunk each of the major biological theories, their review of environmental theories was cursory and circumspect at best. They acknowledged that there has been very little recent study on environmental causes, but chalked that up for “enthusiasm for biological explanations,” concluding:
Though the substantive body of psychological causation research is aging and not being regularly renewed, it has never been refuted and still holds promise for understanding part of the causal puzzle of homosexuality.
They do however note that there may be an interplay between biological and experiential factors, and that “the complex factors which results in the orientation toward homosexuality probably differs from person to person.” But even if biology plays a role, they conclude:
There appears to be a variety of factors which can provide a push in the direction of homosexuality for some persons, but there is no evidence that this “push” renders human choice utterly irrelevant. …
One study of the correlations between the television viewing of adopted children and their adoptive and biological parents produce evidence of “significant genetic influence on individual differences in children’s television viewing.” … All of us would reject the notion that our genes make us sit for a certain number of hours in front of a television screen, but we may have a predisposition (sedentary tendencies?) which would make the choice to view television appealing to varying degrees.
Is Homosexuality a Pathology?
From there, they moved on to the question, “Is homosexuality a pathology?” Here they focused on personal distress (“Contemporary research continues to suggest high levels of distress in the homosexual population even if that conclusion is usually not stated.”) and maladaptiveness (citing several non-representative studies for promiscuity statistics to suggest gay men and women cannot form monogamous relationships, “an essential adaptive capacity.”) They conclude:
The evidence cited above fails far short of a convincing case that homosexuality in itself constitutes a psychopathological condition. The evidence also suggests that one would be on shaky grounds in proclaiming that there is no evidence that homosexuality is anything other than an healthy, normal lifestyle variant.
Is Change Impossible?
And finally — and the point that is most relevant for their newest ex-gay study — they ask “is change to heterosexuality impossible for the homosexual?” Remember, this was before Robert Spitzer’s ex-gay study, “Can some gay men and lesbians change their sexual orientation?” appeared in the October 2003 issue of the Archives of Sexual Behavior. So at this point, they had little to go on. But I did notice a very interesting footnote:
111. For a reasonably complete review of existing “conversion therapy” studies, see J. Nicolosi, Reparative Therapy of Male Homosexuality (New York: Jason Aronson, 1991). Critics are right to note that many of these studies lack methodological rigor and are basically compilations of independent clinical interventions. Reported success rates have hovered in the 33 to 50 percent range. [Emphasis mine]
There it is. The 30-to-50 percent range we’ve been hearing about. Or, according to the much bigger picture, the 50-to-70 percent failure rate.
Jones and Yarhouse however did close their discussion of change by noting:
It is troubling that the many Christian ministries which attempt to provide opportunities for growth and healing for the homosexual person rarely if ever study and report their success rates.
Now with their newly announced ex-gay study, it looks like they are trying to address that deficiency. But as I said, until we get a copy of the study and can evaluate it ourselves, I think skepticism is in order based on Jones and Yarhouse’s previous collaborations. While “The Use, Misuse and Abuse of Science” certainly doesn’t fall into the usual traps of egregious exploitation of scientific research, neither does it rise to being a dispassionate review of the scientific literature.
“The Use, Misuse and Abuse of Science” has become an influential resource among anti-gay activists. Several speakers at Love Won Out and the Exodus Freedom Conference referred to this chapter specifically when they want to debunk studies on the prevalence of homosexuality or the possible biological contributions to homosexuality. The book was also sold at those conferences and was highly recommended by several speakers and participants.
The results from Ex-Gays? A Longitudinal Study of Religiously Mediated Change in Sexual Orientation will be released today during a press conference at the American Association of Christian Counselors World Conference in Nashville, Tennessee. Exodus will be holding their regional conference in Nashville that same weekend. That’s quite an audience clamoring for this book. But whenever a researcher bypasses the peer-review process and has a book to sell to an eagerly waiting customer base, skepticism is warranted.
InterVarsity Press Announces Release of Exodus Ex-Gay Study
September 9th, 2007
InterVarsity Press issued this press release, announcing the Sept 13 release of a study by Stanton L. Jones (Wheaton College) and Mark A. Yarhouse (Regent University). Titled “Ex-Gays? A Longitudinal Study of Religiously Mediated Change in Sexual Orientation,“ the study is intended to address two questions: can people change, and is trying to change harmful?
When Robert Spitzer’s ex-gay study, “Can some gay men and lesbians change their sexual orientation?” appeared in the October 2003 issue of the Archives of Sexual Behavior, the journal took the highly unusual step of publishing twenty-six peer review commentaries, both critical and laudatory. Among the commentaries was Mark Yarhouse’s “How Spitzer’s study gives a voice to the disenfranchised within a minority group.” He notes a few of the limitations of Spitzer’s study and muses on what a better constructed study might look like:
There is a need for studies with improved methodology. This would include a prospective longitudinal design in which participants provide information on sexual behavior, attractions, fantasy, and so on, prior to or in the early stages of therapy, and then tracked over tie, so that something as potentially unreliable as memory recall would not play so prominent a role in studies that touch on such a controversial topic.
I would also suggest another set of data: follow-up on those who drop out of ex-gay ministries, and their post-therapeutic experiences and perspectives. Yarhouse wrote another paper titled, “An Inclusive Response to LGB and Conservative Religious Persons: The Case of Same-Sex Attraction and Behavior”, which appeared in the June 2002 issue of Professional Psychology: Research and Practice. In that paper, he recognized the existence of “ex-ex-gays”:
A similar tension exists when we consider “ex-ex-gays.” They are individuals who once lived an LGB lifestyle, later attempted to change their behavior or attractions, and still later returned to living an LGB lifestyle. From a conservative religious perspective, ex-ex-gays may be the result of poor therapeutic technique, insufficient client commitment or motivation, moral or spiritual failure, or failure of ministries to offer realistic expectations of change. This last consideration is particularly important. It might be that conservative religious persons hold out expectations for change that are too high (i.e., that a person would be free from every vestige of same-sex desire and would be happy and fulfilled in marriage). From this perspective, ex-ex-gays are discouraged, sometimes angry, about their experiences within religion-based ministries.
Gay-affirmative theorists tend to see ex-ex-gays as casualties of professional interventions and religious ministries (Haldeman, 1994). Gay-affirmative theorists propose that ex-ex-gays are the result of the predictable failure of sexual reorientation therapy and religion-based ministries to accomplish what they purport to accomplish. According to Haldeman, some question whether these proponents of reorientation and reparative therapies are not disturbed themselves, preying on vulnerable persons who are hoping against hope to experience change.
Again, is there merit to both accounts? Is it possible that some people are misled about what reorientation and reparative therapy can offer? It is possible that some people do hold expectations for change that are too high. Whether the individual is freely seeking changes for personally felt reasons or is being taken to a program for change by a third party is also a factor in relation to this topic. All of this depends on several factors, including what sexual orientation is, whether it is immutable, and what evidence exists for the effectiveness of reorientation and reparative therapies.
Having talked to a number of ex-ex-gays myself, I think this discussion of their perspectives are highly over-simplistic. While some may have held a number of unrealistic expectations — expectations that are often promoted by ex-gay ministries themselves — most whom I talked to had more realistic expectations. They just found the prospect of living with those expectations unrealistic.
When discussing possible harms of attempting to change, it will be essential for this study to also follow the experiences of those who drop out of ex-gay ministries, and to talk to ex-gay survivors directly.
Jones and Yarhouse have collaborated before. They wrote “The Use, Misuse and Abuse of Science in the Ecclesiastical Homosexuality Debates,” which appeared in the 2000 anthology Homosexuality, Science, and the “Plain Sense” of Scripture” (edited by David L Balch and published by Eerdmans). This article, which largely limits its focus on the “misuse and abuse” by the gay-affirming side rather than the anti-gay side, clearly shows their biases in discussing whether homosexuality is a pathology and in dealing with the possible biological and environmental causes of homosexuality. I’ll have more on this later.
This latest study will be released during a press conference at the American Association of Christian Counselors World Conference on September 13 in Nashville, Tennessee. And what a coincidence. Exodus will be holding their regional conference in Nashville that same weekend. Gee, what are the odds?
Cameron Promoter George Rekers: New Ex-Gay Study “Meets High Research Standards”
September 9th, 2007
When InterVarsity Press issued this press release announcing upcoming release of “Ex-Gays? A Longitudinal Study of Religiously Mediated Change in Sexual Orientation,“ by Stanton L. Jones and Mark A. Yarhouse, they included this endorsement:
George A. Rekers, Professor of Neuropsychiatry and Behavioral Science Emeritus at the University of South Carolina School of Medicine, states that the study “meets the high research standards set by the American Psychological Association that individuals be validly assessed, followed and reported over time with a prospective, longitudinal outcome research design.”
I would question Rekers’ ability to recognize studies which “meets the high research standards set by the APA.” In 2002, he held one of Paul Cameron’s studies in pretty high regard:
In the study of homosexual parenting with the best research methodology to date, Cameron and Cameron obtained a random sample by a one-wave, systematic cluster sampling of six U.S. metropolitan areas… [emphasis mine]
I reviewed that study that impressed Rekers so much here.
Notes On the NARTH Conference Agenda
August 20th, 2007
A few weeks ago, the National Association For the Research and Therapy of Homosexuality (NARTH) posted its 2007 conference agenda (PDF: 108KB/3 pages). Through considerable detective work, Ex-Gay Watch’s David Roberts suggests that one item may be a talk by Stanton Jones on the results of a secret five-year study on the outcomes of religiously-based change therapies.
Another item I noticed is a talk by Christopher Rosik called “A Proposal For the Development of Treatment Guidelines For Unwanted Same-Sex Attractions.” This talk appears to be a response to Warren Throckmorton’s and Mark Yarhouse’s Sexual Identity Therapy Framework, which outgoing NARTH president Joseph Nicolosi has already rejected.
Rosik’s previous stab at this subject is nothing more than a common Cameron-esque anti-gay tract which pulls medical science out of context to re-pathologize gays and lesbians. In fact, it is so Cameron-esque that he cites three separate papers by Paul Cameron, two to claim that gays and lesbians have a significantly shorter lifespan than anyone else, and another to claim that child sexual abuse “causes” homosexuality.” I wonder if Rosik’s talk at NARTH will be significantly different?