Editor of “Gay Parenting” paper responds to BTB reader
June 15th, 2012
Andrew, a reader and regular commenter at Box Turtle Bulletin, emailed James Wright, the editor of Social Science Research. Citing his own experience as a published author of scientific articles, Andrew expressed concern about Mark Regnerus’ article, its failure to address its objectives, and the careless way in which it lends itself to political abuse (snippet):
In short, the author ultimately fails to address the question he seeks to any reasonable degree. In the past days, Regnerus himself has publicly acknowledged that acquiring some of the data necessary to arrive at the conclusions he does is a “methodological impossibility” at present, and that there’s a “low ceiling to what’s possible” with this information. Given the critical impact of frankly inadequate work funded by an ultraconservative think tank, I question the ethics of publishing such incomplete (some might argue shoddy) work. This will have significant real-world impact, given the political salience of the issue.
James Wright responded (in full):
The paper to which you refer was vetted, reviewed and revised following exactly the same processes that all SSR submissions go through. None of the three external reviewers of the paper, nor any of the three formal commenters, raised any prohibitive concerns about any aspect of the study. Some suggestions for revisions were made in the first round of reviews and those suggestions were followed in the published version.
For the record, Dr. Regnerus is a well-known, respected, and widely cited member of the social science community. A check this morning of the Publish or Perish data base shows 2,415 scientific citations to his papers, which generally appear in high-quality social science journals.
I am told that Professor Regnerus intends to release the raw data for reanalysis sometime in the fall, and I expect those data to be pored over very carefully by a large number of investigators. Any possibly erroneous conclusions that come to light as a result of this process are certain to be reported. This is what makes science self-correcting.
Since we followed the same procedures that are followed for all submissions (some three-quarters of which are rejected, by the way), I would not call the decision to publish the paper “an editorial oversight,” although I am quick to admit that peer review is not a perfect process. As for seeking publicity, be assured that I much prefer the relative anonymity within which my journal editing normally takes place.
I will indulge myself in one final point. The children studied in this survey were raised in an era when it was legally impossible for their parents to form normal marital unions, when gay people were subjected to hostilities and prejudices of the worst imaginable sort, and where their children would have been stereotyped and vilified by their peers and others. The hypothesis that these children would not suffer lasting effects from this sort of social environment seems implausible in the extreme. I do not see that is damaging either to the parents or the children to call attention to the formidable difficulties gay parents must have faced (and still face) in trying to raise their children, or to the consequences for these children that are still detectable years and even decades later. To the contrary, these strike me as precisely the realities that must be acknowledged and faced if we are ever to progress beyond our current heteronormative bigotries.
With best wishes,
Perhaps Mr. Wright simply doesn’t understand the objections to the article. Hearing complaints about how the conclusions are not supported, it seems that he thinks that it is the conclusions themselves that are causing concern – that if they had been glowing then the report would be accepted. And besides, the conclusions confirm what his presumptions already know.
Wright’s discussion about the likelihood of children of gay parents growing up in the 80′s and 90′s having detectable negative social consequences is not a bad hypothesis. It would not surprise me to find that there were some differences between those children raised by intact same-sex families and intact opposite-sex families, especially in socially hostile locations. But we don’t know that to be true and, despite its pretensions and Regnerus’ media claims, his study tells us nothing whatsoever about that possibility. Besides, difference does not necessitate negative consequences (for example, children of Orthodox Jews tend to have much lower drop-out rates).
While Wright sees logical and predictable results about “the children studied in this survey”, he misses the point that there weren’t any. Of all the children in the survey, only two were raised for their entire childhood by a female same-sex couple and none were raised for their entire childhood by a male same-sex couple.
We do know that Dr. Wright is not unaware of proper sampling techniques or the limitations of inadequate data. In 2010, he co-edited The Handbook of Survey Research, Second Edition, a synopsis of which describes the book thusly:
Detailed chapters include: sampling; measurement; questionnaire construction and question writing; survey implementation and management; survey data analysis; special types of surveys; and integrating surveys with other data collection methods. This handbook is distinguished from other texts by its greater comprehensiveness and depth of coverage including topics such as measurement models, the role of cognitive psychology, surveying networks, and cross-national/cross-cultural surveys. Timely and relevant it includes materials that are only now becoming highly influential topics.
So it’s hard to say at this point exactly why Wright rushed to publish this obviously flawed study. Perhaps he was impressed by the scientific citations that Regnerus has generated (Dr. Wright is not, himself, affiliated with a prestigious social science program and his publication, while carrying the Elsevier name, is not sufficiently important, for example, to be carried by the University of California).
It may be that he was blinded by his presumptions and green-lighted a paper that confirmed what ‘everyone knows’. Or maybe his desire that we progress beyond our heteronormative bigotries doesn’t extend to full social and legal equality and he felt it important that Regnerus’ “findings” be presented with politically expedient timing.
I will resist guessing as to his motives. But I do hope that in his continuing conversation with Andrew he will give serious thought to the matter and will be open to rethinking the wisdom of his decision.
Express Delivery: The Regnerus “Gay Parenting” Paper Took The Fast Lane To Publication
June 15th, 2012
One of the things that a few researchers have commented privately to me about concerning Mark Regnerus’s much-dissected study on so-called gay and lesbian parenting (you can read my dissection here), is the amazing speed with which the paper was submitted, peer reviewed and accepted for publication by the journal Social Science Research. The typical process is measured in months, if not years in some cases. But for Regnerus’s paper, only 41 days passed from the time the manuscript was submitted to the day the editors accepted it for publication. This accomplishment is notably rare for journal publication, and it has left quite a few researchers I’ve talked to scratching their heads.
To be sure, not everything about the paper could be measured in days. Regnerus reported in his paper that the survey firm he hired, Knowledge Networks, had a hard time finding adult children with parents who fit his peculiar definition of being lesbian or gay. One solution was spend more time to keep looking:
Thus in order to boost the number of respondents who reported being adopted or whose parent had a same-sex romantic relationship, the screener survey (which distinguished such respondents) was left in the field for several months between July 2011 and February 2012, enabling existing panelists more time to be screened and new panelists to be added.
Altogether, those extended efforts combined with the originally drawn sample yielded 236 adult children whose mother or father “ever [had] a romantic relationship with someone of the same sex.” Regnerus has, when asked directly, admitted that those numbers are much too small to make adequate comparisons to determine their fitness as parents — even though he nevertheless made those comparisons in his paper. When BTB reader StraightGrandmother pressed him in an email exchange on this point, he responded:
I maxed what Knowledge Networks could do with their panel, and no research firm out there is in a position to generate a larger N. Perhaps I could’ve left it in the field for another year, but that is quite awhile, and wouldn’t have doubled the sample size of LMs or GFs.
I’m not following why he doesn’t believe he couldn’t have significantly increased his sample size if the question had been held open for another year. If it is true that, as he acknowledged, that he was hampered by the difficulty of finding enough people to fit his definitions, then the only statistically relevant solution is to keep looking, and not to construct an arbitrarily heterogeneous category to contrast against a homogeneous one and falsely claim that the comparison is legitimate.
And so why not wait? Why not do the only statistically proper thing to do and hold the question open for a year? We’re talking about a paper that was intended to either validate or destroy 30 years of social science research on gay and lesbian parenting. What’s another year more?
What’s the rush?
And rush is clearly the right word. Just look a this the timeline:
July 2011-February 2012: As Regnerus recounted in his paper and I quoted above, Knowledge Networks, the company which conducted the survey and gathered the raw data, held the data collection process open during this period in order to find enough adult children of “Lesbian Mothers” or “Gay Fathers” (terminology that I always find difficult writing out when one considers how loosely he defined those categories)
February 1, 2012: The article is submitted to Social Science Research for consideration. This date is given on the paper’s front page. If the previous time period he gave for collecting his samples is correct, then he had only days to get the last of his data in, the numbers crunched, and the nineteen-page article completed. That alone is pretty remarkable.
February 29, 2012: The article is revised for publication. This typically occurs due to feedback from peer reviews. Taking only one month for others to review the study, get the feedback back to him and for him to incorporate revisions, again, is a remarkable turnaround.
March 12, 2012: The article is accepted for publication.
But then, from the time the article was accepted for publication until mention of it suddenly appeared in the Deseret News and The Washington Times, the article was never posted on Social Science Research’s Articles In Press section. After the incredible speed with which the data was gathered, article written, reviewed, revised, and accepted for publication, the article just sat there, hidden, available only to the select few that Regnerus chose to make it available to. In fact, one potential critic of that Deseret News contacted to comment on the study couldn’t because he had not seen it and Deseret News refused to provide him with a copy.
As of today, Social Science Research lists 33 articles as being in press. Three of those articles have been in the queue since before March 12 when Regnerus’s paper was accepted. And as you can see, none of the other articles were prepared with the lightning speed which Regnerus’s paper enjoyed.
|York||Feb 1, 2012||May 4, 2012||May 31, 2012||Jun 13, 2012|
|Krumpal||Jul 13, 2011||May 24, 2012||May 31, 2012||Jun 13, 2012|
|Price & Collett||Sep 5, 2011||May 15, 2012||May 31, 2012||Jun 13, 2012|
|Alvarado & Turley||Nov 11, 2011||May 10, 2012||May 31, 2012||Jun 13, 2012|
|Schmidt & Danziger||Jul 18, 2011||May 31, 2012||Jun 4, 2012||Jun 12, 2012|
|Warner & Adams||Jun 10, 2011||May 23, 2012||May 31, 2012||Jun 11, 2012|
|Kreisman||Oct 22, 2011||Feb 23, 2012||May 10, 2012||May 31, 2012|
|Högnäs & Carlson||Jul 25, 2011||Mar 7, 2011||May 10, 2012||May 30, 2012|
|Arnio, et al.||Jul 10, 2011||Jan 18, 2012||May 10, 2012||May 22, 2012|
|Kye & Mare||Apr 5, 2011||Feb 20, 2012||May 10, 2012||May 18, 2012|
|Cor, et al.||Aug 10, 2011||Apr 26, 2012||May 2, 2012||May 16, 2012|
|Daw & Hardie||Feb 5, 2011||Apr 20, 2012||May 2, 2012||May 14, 2012|
|Yan, et al.||Jul 7, 2011||Dec 15, 2011||May 2, 2012||May 11, 2012|
|Shu & Zhu||Feb 23, 2011||Jan 4, 2012||May 1, 2012||May 11, 2012|
|Seltzer, et al.||Aug 23, 2011||Apr 24, 2012||May 2, 2012||May 11, 2012|
|Klugman, et al.||Jul 1, 2011||Apr 25, 2012||May 2, 2012||May 10, 2012|
|Felson & Painter-Davis||Sep 19, 2011||Apr 15, 2012||Apr 23, 2012||May 2, 3012|
|Lancee & Van de Werfhorst||Jun 22, 2011||Mar 25, 2012||Apr 16, 2012||Apr 26, 2012|
|Marquart-Pyatt||Jan 24, 2011||Mar 27, 2012||Apr 2, 2012||Apr 17, 2012|
|Brauner-Otto, et al.||Jun 10, 2011||Mar 30, 2012||Apr 2, 2012||Apr 16, 2012|
|Mollborn, et al.||Sep 1, 2011||Mar 23, 2012||Apr 2, 2012||Apr 16, 2012|
|Xie, et al.||Sep 4, 2009||Feb 12, 2012||Apr 1, 2012||Apr 10, 2012|
|Cheadle & Sittner Hartshorn||Jul 8, 2011||Mar 18, 2012||Mar 27, 2012||Apr 3, 2012|
|Logan & Zhang||Oct 11, 2011||Feb 13, 2912||Mar 14, 2012||Apr 3, 2012|
|Cheadle & Schwadel||Sep 23, 2011||Mar 18, 2012||Mar 27, 2012||Apr 1, 2012|
|Sonnenberg, et al.||Jul 7, 2011||Nov 2, 2011||Mar 19, 2012||Mar 27, 2012|
|Zhou||Apr 15, 2011||Jan 15, 2012||Jan 2, 1900||Mar 27, 2012|
|Teney & Hanquinet||Jul 31, 2011||Mar 7, 2012||Mar 15, 2012||Mar 23, 2012|
|Thornton, et al.||Aug 24, 2011||Jan 18, 2012||Mar 12, 2012||Mar 21, 2012|
|Guzzo & Hayford||Mar 9, 2011||Dec 22, 2011||Mar 12, 2012||Mar 21, 2012|
|Villareal & Hamilton||Jan 6, 2011||Feb 15, 2012||Feb 21, 2012||Mar 7, 2012|
|Smith, et al.||May 17, 2011||Feb 13, 2012||Feb 14, 2012||Feb 27, 2012|
|Logan & Shin||Apr 8, 2011||Jan 29, 2012||Jan 31, 2012||Feb 7, 2012|
And how does the timeline for Regnerus’s paper stack up with the others appearing in the July 2012 issue? Have a look:
|Marks||Oct 3, 2011||Mar 8, 2012||Mar 12, 2012||—|
|Regnerus||Feb 1, 2012||Feb 29, 2012||Mar 12, 2012||—|
|Zuberi||Jan 28, 2010||Jan 13, 2012||Jan 17, 2012||Jan 29, 2012|
|Casciano & Massey||Apr 5, 2011||Feb 11, 2012||Feb 14, 2012||Mar 3, 2012|
|Schmeer||Oct 28, 2010||Jan 20, 2012||Jan 23, 2012||Feb 2, 2012|
|Treas & Tai||May 31, 2011||Jan 26, 2012||Jan 30, 2012||Feb 13, 2012|
|Ehlert||Nov 10, 2010||Jan 10, 2012||Feb 7, 2012||Feb 17, 2012|
|Manlove||Jan 21, 2010||Feb 2, 2012||Feb 7, 2012||Feb 21, 2012|
|Henretta, et al.||Feb 10, 2011||Dec 15, 2011||Feb 21, 2012||Mar 2, 2012|
|Flashman||Feb 5, 2011||Jan 15, 2012||Mar 1, 2012||Mar 13, 2012|
|Béteille, et al.||Jul 18, 2011||Mar 1, 2012||Mar 12, 2012||Mar 27, 2012|
|Pharris-Ciurej, et al.||May 21, 2009||Mar 1, 2012||Mar 12, 2012||Mar 21, 3012|
|Skaggs, et al.||Aug 4, 2010||Jan 17, 2012||Jan 17, 2012||Jan 26, 2012|
|Foschi & Valenzuela||Jul 25, 2011||Feb 1, 2012||Feb 2, 2012||Feb 11, 2012|
|Ergas & York||Nov 17, 2011||Feb 2, 2012||Mar 12, 2012||Mar 17, 2012|
|Maimon & Browning||Apr 16, 2011||Jan 25, 2012||Jan 30, 2012||Feb 8, 2012|
|Leopold, et al.||Jul 16, 2011||Feb 20, 2012||Mar 12, 2012||Mar 21, 2012|
You will notice that the only other article which didn’t appear in press before showing up in the July issue is a paper by Loren Marks which criticizes virtually every other study about gay and lesbian parenting from the past 30 years. (I hope to review that article over the weekend.) It too, along with Regnerus’s study, has been hailed by anti-gay activists and discussions of it appeared alongside Regnerus’s study in the Deseret News and The Washington Times.
By the standards of Social Science Research, it looks like the skids were well greased to get Regnerus’s article through the process as quickly as possible, and then to carefully control its release afterward. With these kinds of dates, it’s very difficult to believe that Regnerus’s article did not receive special treatment throughout the processes.
And so this brings up the obvious question: Why?
There are two major sets of events this year which might provide an answer. By the time this year is over, it looks like there will have been a total of five ballot initiatives on same-sex marriage. We had one constitutional amendment passed in North Carolina, and there will be another on the ballot in Minnesota. Maine and Washington will vote on whether to allow same-sex marriages in those states, and it looks all but certain that a similar question will be put to voters in Maryland. And if history is any guide, one important topic in these debates will be whether gay and lesbian couples are fit to raise children.
And that’s in addition to the four federal court cases challenging the Defense of Marriage Act — all of them successful so far. At least two of them, along with the pending Prop 8 case, are likely to go to the U.S. Supreme Court very soon. Arguments over whether gays and lesbians can be good parents have figured in many of those cases as well. You can count on this paper showing up in briefs and filings as these cases move forward.
The conservative Whiterspoon Institute and Bradly Foundations together threw $785,000 at this study. They purchased a potent weapon, notwithstanding Regnerus’s protestations that the study says nothing about the fitness of gay or lesbian couples to raise children. And Social Science Review has been co-opted into that fight. Which goes to show that money doesn’t but good research, but it does buy politically useful research just when you need it.
Regnerus Admits He Lacks the Data to Critique Same-Sex Parenting (*so why is he doing it?)
June 13th, 2012
This is getting uglier.
Prof. Mark Regnerus has been giving interviews about his study on parents who’ve had same-sex relationships, saying things like this:
Well, in the generation that are adults now, kids raised in a same-sex household were more likely to experience instability and shifting household arrangements. For example, 14 percent of kids whose moms had a lesbian relationship reported spending more time in foster care, well above the average of 2 percent among all respondents.
That leapt out at me because the error is obvious: The second sentence in no way supports the first. Children whose “moms had a lesbian relationship” weren’t necessarily “raised in a same-sex household” — the children might have never even met their mother’s lesbian partner, much less have been raised by her. Jim Burroway has done some great work pointing this out, and I’d like to extend it. In fact, I’d like to go so far as to show that Regnerus himself admits that he has, well, nothing.
Regnerus’s team interviewed 15,058 people. Few of them had a gay parent; even fewer lived with their gay parent’s partner for a significant time; and fewer still came from what Regnerus calls a “‘planned’ gay family.”
|Respondents who:||“Lesbian” mother||“Gay” father|
|Had a parent in a same-sex relationship||175||73|
|Lived with parent’s same-sex partner more than 3 years||40||1|
|Came from “planned” gay families (estimated)||30 – 45||less than 1|
A couple points:
- Regnerus is fond of talking about “lesbian mothers” and “gay fathers,” but he defines them as adults who have ever had a same-sex romantic relationship, even if it only happened once, even if it only lasted a few days.
- Regnerus has no data on “planned gay families.” He derived those numbers from looking at “respondents who claimed that (1) their biological parents were never married or lived together, and that (2) they never lived with a parental opposite-sex partner or with their biological father.” The numbers are a guess.
Back to those numbers, though. Regnerus obviously can’t draw any conclusions male same-sex parenting based on a sample of less than 1. How about lesbian same-sex parenting? Is his sample of 30-45 respondents enough to significantly describe the broader population?
Here’s the kicker: Regnerus agrees with me. His article bemoans the low sample sizes of studies that offered up good results for same-sex parenting:
It is not surprising that statistically-significant differences would not emerge in studies employing as few as 18 or 33 or 44 cases of respondents with same-sex parents, respectively…Even analyzing matched samples, as a variety of studies have done, fails to mitigate the challenge of locating statistically-significant differences when the sample size is small.
Look at the numbers in that quote. Now look back at the numbers in the table. This is Regnerus telling us he’s got, as I said, nothing.
Now here’s why this is so ugly.
- In the study’s introduction, Regnerus frames it as an examination of same-parenting and a corrective to flaws in earlier, positive studies on same-sex parenting.
- But Regnerus’s data on same-sex parenting contains the same sample-size flaws for which he which criticized those other studies.
- So once he leaves his introduction and enters analysis, he abandons all pretense of studying same-sex parenting and focuses instead on parents who have ever had a same-sex romantic relationship, regardless of whether they raised a child with that same-sex partner.
- Nevertheless, he does not correct his introduction in order to frame the issue properly.
- And finally, he grants interviews to conservative outlets, claiming that his study shows the harm of same-sex parenting, even though his own words, in his own study, demonstrate that he knows his sample size is just too damn small to say anything with confidence.
Am I wrong to call this ugly? Prof. Regnerus could well be following this blog, given Jim’s excellent and well-publicized work. I hope the professor provides us an explanation and justification for what he’s telling the press.
Why We Talk about Anti-Gay Bias
June 12th, 2012
The Washington Times has (indirectly) declared that not being “entirely heterosexual” is a negative outcome. And it did this in a news article, not an editorial.
In a widely-quoted piece on Mark Regnerus’ gay-parenting study, the Washington Times wrote:
He found that, when compared with adults raised in married, mother-father families, adults raised by lesbian mothers had negative outcomes in 24 of 40 categories…
Actually, no. Regnerus didn’t find that at all. Rather he found 24 statistically-significant differences, and of those differences he wrote:
…in the vast majority of cases the optimal outcome—where one can be readily discerned—favors [intact biological families].
Emphasis added. See what the Times did? It put words into Regnerus’s mouth. Regnerus did not say all these outcomes were negative. That was invented by the Times.
Here’s the kicker. One of the those differences is whether someone “identifies as entirely heterosexual.” Regneros didn’t classify this was positive or negative. He gives a few examples of outcomes that are “obviously suboptimal (such as education, depression, employment status, or marijuana use),” but sexual orientation isn’t on that list.
By now, though, this Washington Times falsehood has been widely reported as truth when it’s nothing of the sort.
What makes this so insidious is that it’s unstated. In one place the reader hears that all the differences were negative. In another, that one of those differences was sexual orientation. The association sinks in without ever being explicitly connected.
But it’s still there, a little splash of anti-gay bias that lands in the article almost unnoticed, one more drop of rain added to the river of homophobia.
My take on the “Children of Gay Parents” study
June 12th, 2012
The key to understanding Mark Regnerus’ study – and to understanding it’s failure – is understanding the motivations of the author and his funders. And, sadly, this is something that I think we fail to grasp with subtlety.
We tend to look at individuals and organizations who oppose our equality as being “anti-gay” and, as that is important to us, we elevate it’s importance to them. We see them as primarily and “anti-gay organization” and attribute motives and malices to them. This may not always be accurate. Anti-gay malice may simply be but a small – incidental even – part of their motivation.
Let me give an illustration: If one is Jewish, then it can be easy to see the Ku Klux Klan’s efforts over the years through the lens of how it impacts you. If one is insufficiently aware of the totality of their endeavors, one might confuse them of being an “anti-Jewish organization”. If one is black, that is not at all how they are perceived.
We need to understand that Regnerus’ study was not necessarily to prove that gay families are inferior. Rather, his goal was to prove that married heterosexual biological parents (intact biological families – “IBFs”) are superior. Not just superior to gays, but superior to everyone else. Gays are just one group, joining divorced parents, single parents, widows, adoptive parents, and all who aren’t IBF.
Children appear most apt to succeed well as adults when they spend their entire childhood with their married mother and father, and especially when the parents remain married to the present day.
Regnerus did not set out to say anything about orientation, he simply set out to prove that a certain family structure is superior. And that’s where he failed.
When discussing heterosexual parents, he did compare family structures. The distinctions and differences between the groups were determined by marital status, divorce, step-parentage and the like, all of which address the structure of the families. However when it came time to discuss children of parents in which one was same-sex attracted, Regnerus played a sleight of hand. He redefined his terms such that ‘having a gay parent’ became in and of itself a family structure.
Regnerus did the same thing for adopted children. The stability of the family, divorce, age of adoption, prior trauma, nothing at all was important other than the way in which they differed from IBF, and as they violated the “B” (biological) then that is the only measure that was important. Regnerus’ “family structures” became defined not by what they were, but by what they were not. There were the Not-I’s: divorce, step-family, single; and the Not-B’s: adopted by ‘strangers’, gay fathers, lesbian mothers (and especially violates the unstated but underlying requirement that the IBF be heterosexual).
Oddly enough, while claiming that he didn’t “go into orientation of parents in this study”, that is precisely what he did. Should one parent have had a same-sex relationship of any sort, that was the determinant that pulled them out of whatever family structure they might have been included in and placed them, de facto, into a non-IBF family structure.
(Imagine if he had done a study in which some other situation were used to create a new family structure: “Families in which the parents are married fare better than ones in which one parent abuses drugs.” Or perhaps “Families in which the parents are married fare better than ones in which both parents work.” It sounds meaningful until you try find the meaning.)
So what is Regnerus to do with this data? He didn’t get the data he hoped for. He didn’t get meaningful data to address the premise he wished to support. He can’t break up the same-sex attracted parents into statistically meaningful family structure groups; he doesn’t have sufficient sample size.
So he has two choices:
He can eliminate the same-sex attracted parents from the study (or put them in comparable family structure groups) or even report that sample sizes disallowed any meaningful conclusions about the comparisons between IBFs and gay families. But then he’s left with a study that says “married parents do better than divorced parents” and that wouldn’t generate headlines in his mother’s Christmas letter, much less in mainstream press. And his funders would object to three quarters of a million bucks being spent on something that has been shown to be true in many studies before this one.
They know that they are superior (and just a bit more special) than divorced parents or those slutty single mothers (who surely are all on welfare). They have studies to prove it. But so far they didn’t have anything to point to which would prove them to be superior to same-sex families.
So instead he chose to play word games. He decided to claim that “one parent is same-sex attracted” is a family structure in the same way that married or separated is a family structure. And, of course, this could be presented (with lots of “oh, no, really”) as implied evidence that IBF is superior to same-sex married couples. Which is precisely what Regnerus did when he said:
In fact, the most significant story in this study is arguably not about the differences among young–adult children whose parents who have had same-sex relationships and those whose parents are married biological mothers and fathers, but between the latter and nearly everyone else.
But, ultimately, once one says “oh, but being gay is not a family structure” then his study becomes meaningless. As Mark Regnerus is discovering.
“Children of Gay Parents” Author Responds To BTB Reader
June 12th, 2012
Whoever invented the term “crowd-sourcing” was onto something, and the BTB crowd is among the best. Regular commenter StraightGrandmother had an interesting email exchange which she posted in the comments section. I want to bring up those email responses here.
Her first email began: “You can’t just force all the Children of Parents who had a same sex relationship into one bucket. it is not representative of what that respondent grew up in for a home life,” She then peppered him with several questions: How many LM/GF had biological parents who were married and stay married, how many were married then divorced, were never married, had a “fling”, etc. In other words, how many LM/GF people fit the more homogenous categories that he created for adult children of heterosexual couples. You can read her full email here. Regnerus replied:
Dear Ms. _______,
I will do my best to get answers to most if not all of your questions, hopefully in the next few days. However, there is not data on “flings,” only the presence or absence of relationships, and whether the respondent lived with the parent and their same-sex partner, and at what age (plumbing the calendar data is time-consuming work, however.)
I believe the article should be publicly available for free on Monday, from the publisher’s website. That is my understanding. I’m sorry you paid for it. I could’ve sent you a copy upon request.
People of good will (and some without) have and will continue to have lots of comments on measurement decisions, etc. Is understandable. Your comments are well-taken. A key priority, however, was always sample size. Curb it too much by slicing groups (wisely, even) into different categories and statistical power drops precipitously. With a much larger sample size, I would’ve done that. Was a judgment call with which some disagree. I maxed what Knowledge Networks could do with their panel, and no research firm out there is in a position to generate a larger N. Perhaps I could’ve left it in the field for another year, but that is quite awhile, and wouldn’t have doubled the sample size of LMs or GFs.
The study was reviewed the regular way, with multiple blind reviews to which I was required to respond.
I added some commentary about the study background, context, at the place where I blog once a week:http://www.patheos.com/blogs/blackwhiteandgray/2012/06/q-a-with-mark-regnerus-about-the-background-of-his-new-study/.
You can cut, paste, and post whatever you wish…
StraightGrandmother thanked him and clarified why she asked about “flings” :
There is something much much different about a home environment where the parents are in a Mixed Orientation Marriage and one spouse has an extra marital affair with someone of their same sex. The troubling part is the extra marital affair not just the fact that it was with a person of the same sex.
This situation is much different than a sole lesbian who establishes a same sex relationship. See the difference between an extra marital fling and a normal two person relationship?
By the way, one of the key methodological criticisms circulating is that–basically–in a population-based sample, I haven’t really evaluated how the adult children of stably-intact coupled self-identified lesbians have fared. Right? Right. And I’m telling you that it cannot be feasibly accomplished. It is a methodological (practical) impossibility at present, for reasons I describe: they really didn’t exist in numbers that could be amply obtained *randomly*. It may well be a flaw–limitation, I think–but it is unavoidable. We maxxed Knowledge Networks’ ability, and no firm is positioned to do better. It would have cost untold millions of dollars, and still may not generate the number of cases needed for statistical analyses. If randomness wasn’t the key priority, then we could’ve done it. And we’d have had a nonrandom sample that was no better than anything before it. So, while critics are taking potshots, they should remember that there’s a (low) ceiling to what’s possible here. My team of consultants elected to go with the screener questions (including the one about same-sex relationships) that we did, anticipating–accurately, too–that there would be no way of generating ample sample size if we narrowed the criteria (for who counts as a lesbian parent) to the sort that critics are calling for. We figured that, with the household roster/calendar offering the opportunity to identify who you lived with, we’d comfortably get enough cases wherein the respondent reported living with mom and her partner for many consecutive years. But few did.
Ergo, in contrast to some impressions, I didn’t construct the study to tell the sort of story I wanted.
I hope that people read the three comments that were also published with the study. They voice confidence in the data, while asserting ample concerns about its use.
You can read an overview of those three commentaries published in Social Science Review here.
In the third email, StraightGrandmother emphasized her concerns about mixed orientation marriages being included in the LM and GF categories:
If your gay/lesbian population primarily produced children in a Mixed Orientation marriage then I feel you should have clearly said that. Because then your research is showing what I believe to be true, Mixed Orientation Marriages are very hard on children. We know that only 1/3 of Mixed Orientation Marriages attempt to stay together after disclosure and of that 1/3 only half manage to stay together for 3 years or more (and it goes really down hill after 7 years).
IF your gay/lesbian population primarily produced children in a Mixed Orientation marriage then I feel you should have clearly said that. Because then your research is showing what I believe to be true, Mixed Orientation Marriages are very hard on children. We know that only 1/3 of Mixed Orientation Marriages attempt to stay together after disclosure and of that 1/3 only half manage to stay together for 3 years or more (and it goes really down hill after 7 years).
She mentioned Zach Wahls, the young man who was raised by two mothers as the only family he ever knew and who testified about his family before an Iowa statehouse hearing to consider rescinding same-sex marriages in that state, as an example of someone who was really the product of Lesbian Mothers. StraightGrandmother castigated Regnerus for what she saw as carelessness with how he described his “lesbian mothers” category. “You were not up front saying, ’99.5% of the respondents with a gay or lesbian parent were raised in a home where the parents had a Mixed Orientation Marriage.’ Just that simple sentence would have made everything right. …Because of that one missing sentence clearly stating the Mixed Orientation Status you are being attacked instead of being praised.”
Well, it was quite unlikely that I would be praised in the scholarly community. Nor did I expect to be. Trust me, I’m not surprised by the antagonism, but nor does it feel like a badge of honor to me.
The Daddy donor study, with which I am familiar, is not a population-based random sample. It’s an opt-in sample. I even inquired about this study with Abt SRBI and it was apparent that they couldn’t handle it like KN could. They didn’t randomly survey 1,000,000 people. Only the Census–or some precious few deep-pocketed federal studies–contacts that many. The research team discussed an opt-in sample supplement to boost N, and advised against it because the probability of respondents’ inclusion could not be ascertained. Been there, discussed that, elected not to go there in order to preserve the quality of the sample.
I don’t go into orientation of parents in this study. That is a deeper well, and in the cohort that I’m studying, I don’t presume that either they or their parents would confidently call their parent gay or lesbian (or something else). Ergo, we made it about behavior–and not discreet behavior but something their child would be aware of.
Moreover, plenty of scholars, like Lisa Diamond, assert that women’s sexuality is more fluid than men’s. (I make no claim or assertion either way.) If she’s right, your claim about mixed-orientation marriages seems more fixed, especially for women, than it need be in social and marital reality. But I elected NOT to make this about orientation or self-identity. You suggest more ominous motivation, but I assure you that was not true.
Your accusations are getting more heated, and I’m afraid unless we can correspond civilly, I may have to call a conclusion to this.
Again, please post.
How many times are they going to do this?
June 11th, 2012
Two days after leaving the AIDS/LifeCycle bubble of love, I force myself to click on NOM’s blog page, and you know what I find?
Same sh*t, different day.
NOM (and the rest of the anti-gay world) is crowing about a study on the awfulness of gay parenting. You can read more about that here and here and here. Let me focus on one distinct and familiar flaw: The study compares the children of married biological parents with those from broken homes — and the study’s “lesbian mothers” that our opponents are vilifying generally weren’t married to each other; nor were the gay fathers. No, they were often in opposite-sex relationships that broke down.
Our opponents’ reporting hasn’t focused on that. But I can easily imagine how how an unstable, dishonest mom/dad relationship would (a) be harmful to the kids, and (b) have nothing to do with same-sex parenting!
Now, you can combat this sort of thing with careful analysis. But the error is so common, so recurring, that I’ve put together a cartoon I now can just whip out when the need arises. Who knows, maybe this is the best way to make such an obvious and elementary point:
Feel free to post it where needed.
Overview: Three Responses to Mark Regnerus’s Study of Children of Parents In Same-Sex Relationships
June 11th, 2012
For those of you returning from the weekend, you may have missed the release of a new study published in the July issue of the journal Social Science Research. That study claims to show “numerous, consistent differences, especially between children of women who have had a lesbian relationship and those with still-married (heterosexual) biological parents.” If true, these results would up-end some thirty years of established scientific research which had previously shown that gay and lesbian parents are, on the whole, just as good parents as their straight counterparts. The study received one-sided coverage in the Deseret News and the Washington Times.
If you haven’t already done so, I would recommend you read my analysis of Mark Regnerus’s study, “How Different are the Adult Children of Parents Who Have Same-Sex Relationships? Findings from the New Family Structures Study” (Social Science Review 41, no. 4 (July 2012): 752-770.)
Later yesterday morning, I learned that the July 2012 issue of Social Science Review also includes three commentaries on Regnerus’s paper and a rejoinder by Regnerus. That same issue also published a related paper by Loren Marks reviewing and identifying several weaknesses of the past thirty years of social science research about gay parents. I probably won’t be able to review Marks’s paper until later in the week. The three commentaries also remarked on Marks’s paper, but I will concentrate this overview on their observations about the Regnerus paper.
Commentary #1: Paul R. Amato, of Penn State’s Department of Sociology. Amato, like the other two commenters who we will review shortly, commended the Regnerus paper for its strongest feature: the use of a large, national probability sample. As I said yesterday, this is an extraordinarily rare achievement in social science research, and it is why Regnerus’s paper is so important. Amato reviewed Regnerus’s data and for the most part verified Regnerus’s finding that Regnerus’s sample of children of Lesbian Mothers (with all of the issues surrounding the construction of that sample I noted yesterday), did indeed depart from the Intact Biological Family sample.
Amato used a different statistical method to come to that conclusion, a method known as “effect size.” A very brief description is in order. Effect size is, according to this paper, “a simple way of quantifying the difference between two groups that has many advantages over the use of tests of statistical significance alone.” Traditional statistical tests, like those Rengnerus used, identify the statistical significance of two measures, meaning that they measure whether the similarity or difference between two measures can be explained by random chance — sort of like two numbers that fall within the same margin of error. If they can’t, then they are said to be statistically significant. Effect size, on the other hand, estimates the strength of an apparent relationship between two populations based on the standard deviation of results of the control population. This is important because if the control population has a high degree of variability, then even a large differences in the averages between the two groups might not mean anything since there would still be a large overlap.
I know that’s not enough of an explanation to make you (or me) an expert on statistical significance and effect size. But the important thing to keep in mind is that in social science and medical research, it is increasingly recognized that both calculations should be performed. Traditionally, only statistical significance is calculated (as Regnerus did in his paper) and many authors today still only rely on statistical significance tests to evaluate their data. But some journals are beginning to require effect size calculations alongside statistical significance measures, and statistical software packages are beginning to include effect size in their libraries. But calculating effect size has not yet become a standard standard practice.
So, getting back to Amato’s paper, he ran some effect size calculations and, as I said, he confirmed Regnerus’s finding that children growing up in lesbian households (as aggregated in Regnerus’s sample with all of its problems — and I want to keep reiterating that) differed from children growing up in intact biological families with “a moderately large effect size.” However:
The choice of comparison group makes a difference, however. Comparisons of offspring with lesbian mothers and offspring from heterosexual stepfamilies revealed a mean effect size of only .15. When children with divorced or continuously single mothers served as the comparison group, the mean effect size was only .19. I would describe these effect sizes as weak.
What is the most appropriate comparison group? This is a difficult question, given the heterogeneity of gay and lesbian families with children. Consider lesbian couples who have children through sperm donation, or gay couples who have children through surrogacy. Is it reasonable to compare these children with the children of continuously married heterosexual parents? Or should children in the heterosexual comparison group be limited to those born via sperm donation or surrogacy? What about lesbian mothers or gay fathers with children from former marriages or unions? Should these children be compared with those of heterosexual parents who are married, cohabiting, remarried, divorced, or never married? The fact that same-sex marriage is now allowed in several states adds another level of complexity to the problem. Perhaps in future studies, married same-sex parents should be matched with married heterosexual parents.
Which pretty much echoed my concerns yesterday.
Commentary #2: David J. Eggebeen of Penn State’s Department of Human Development and Family Studies. Eggebeen echoed many of Amato’s sentiments. First, he praised the sophistication of Regnerus’s raw data set, which represents a significant advance over previous studies. However:
Nevertheless these data are far from ideal. A larger sample does not translate into a sufficiently large sample. Given some of the complexity of the family lives detailed above, a sample of 163 young adults who report a lesbian mother and 73 who report a gay father are frustratingly inadequate for doing anything but broad comparisons across family characteristics.
Eggebeen also cautions against jumping to quick conclusions about what the data means:
Finding a significant number of negative correlates of well-being for children with gay or lesbian parents, even if they are derived from simple models, invites thinking about some possible mechanisms. It is hard to imagine explanations that point to the quality of parenting per se. Parents, regardless of sexual orientation, are equally motivated to provide the best care possible for their children. It is reasonable, however, to posit that gay and lesbian parents and their children face challenges that may make parenting more difficult.
He concludes that Regnerus’s study has the potential to address the shortcomings of previous studies, but with an important caveat:
The analyses in the Regnerus paper are provocative but far from conclusive. These very preliminary findings should not detract from the real importance of this paper, the description of a new data set that offers significant advantages. Whether the New Family Structures Study has the possibility of unsettling previously settled questions depends in equal parts on richness of the information collected, as well as the willingness of scholars to make use these data.
Regnerus’s larger data set and variable list will become publicly available in the fall so that other researchers to perform their own sets of calculations and comparisons.
Commentary #3: Cynthia Osborne, of the University of Texas LBJ School of Public Affairs. Osborne, like the others, also noted the heterogeneity (i.e. the large in-group differences) of the Lesbian Mother and Gay Fathers groups:
To increase the sample size of children who experienced a same-sex parent, Regnerus included respondents in either the LM or GF comparison groups if they reported that their parent ever had a same-sex relationship. Although this decision has a lot of merit, it makes comparisons across groups somewhat of a challenge. Because the LM group is comprised of young adults who experienced multiple family forms and transitions, it is impossible to isolate the effects of living with a lesbian mother from experiencing divorce, remarriage, or living with a single parent.
She also questioned the decision of what to use as an outcome variable verses a control variable. For the straight populations, the control variables included divorce parents, step-parents, adopted parents, single parents, etc. Among the outcome variables were questions about sexual molestation, forced sex, and whether the family received welfare. She wondered what the data might look like of those were used as control variables rather than outcome variables, since each of them can have a strong effect on childhood outcomes. But while there are all sorts of comparisons that might be made, assigning a cause for the outcome cannot be done based on the data at hand:
Importantly, one cannot clearly link having a lesbian mother (or gay father) with any of these outcomes. As stated earlier, the group is comprised of young adults who experienced multiple family structures, not only a same-sex parent household (indeed, some of the respondents never lived with the mother’s same-sex partner). It is quite possible, for example, that many or most of the negative outcomes result from the divorce of the young adult’s biological parents that preceded the mother’s same-sex relationship. …
…The concern for Regnerus is not Type II errors (saying something is NOT significant when it is), but the possible attribution of differences to living in a same-sex household rather than to experiencing multiple family structures in childhood, one of which happened to be a same-sex parenting relationship.
Osborne emphasized that researchers in this particular field — those who focus on children of same-sex parents — bear a particular responsibility for how their research is presented to the public:
The focus on children of same-sex parents seems, then, to be driven more by the sensitive political and social issues surrounding same-sex relationships than by evidence that this family structure is increasing rapidly or, for that matter, harmful to children. Because the topic is so politicized, scholars must pay even more careful attention to the presentation and interpretation of their findings. Although scholars are trained to use great care to disentangle the causal versus selection effects of family structure and child well-being, we understand that true causation can never be determined because we cannot randomly assign children to various family structures. Consumers of research on children of same-sex relationships, by contrast, may not always have the same training or be so careful in their interpretations. The results of scholarly studies are often scrutinized by pundits and legislators to support their pre-existing ideas of differences or ‘‘no differences’’ across groups.
…Regnerus (2012) finds substantial differences across groups and uses great care to note that his descriptive analysis does not imply causation and that the LM respondents may have lived in many different family structures. Still, the rigor of the study may lead some advocates to claim that growing up with a same-sex parent causes harm and should, therefore, be illegal.
I think we can bank on Osborne’s concern coming true. I’m willing to lay odds that we will see it coming true today.
Mark Regnerus’s Rejoinder. Regnerus’s response was brief. His first comment was to reiterate Amato’s and Osborne’s concerns about the political rammifications of the study:
I recognize, with Paul and Cynthia, that organizations may utilize these findings to press a political program. And I concur with them that that is not what data come prepared to do. Paul offers wise words of caution against it, as did I in the body of the text. Implying causation here—to parental sexual orientation or anything else, for that matter—is a bridge too far.
Regnerus acknowledged that “the sample size of respondents whose parents report a same-sex relationship is substantial but not large enough to explore some of the more fine-grained distinctions that may well be present.” He also said that he is already planning to use the detailed dataset in future studies, and invited other researchers to mine the data when it becomes publicly available in the fall. Finally, he ends with this paragraph defending his emphasis on biologically intact heterosexual families:
As each of the three explicitly or indirectly notes, family instability—whatever the sources—is often a top culprit in predicting dysfunction in the lives of children, and the data analyses in my article likewise point in this direction. In fact, the most significant story in this study is arguably not about the differences among young–adult children whose parents who have had same-sex relationships and those whose parents are married biological mothers and fathers, but between the latter and nearly everyone else. Contexts of instability—whether in gay or straight households—appear suboptimal for children’s healthy long-term development. While much is made in the scholarly literature about ‘‘resilient’’ youth—those who thrive despite the odds against them and in lieu of an optimal family context—resilience is, on average and perhaps by definition, not normal. Moreover, even resilient children would likely prefer to have engaging parents who are not simply in their lives but in their households. Adults of good will, and most family scholars, typically agree on this. Whether some relationship arrangements are more systematically prone to disorganization than others is an important and empirically-testable question.
Deseret News Covers “Kids of Gay Parents Study,” Finds No Flaws
June 10th, 2012
Deseret News in Salt Lake City this morning published a lengthy 2,100-word article on Mark Regnerus’s study of adult children of parents In same-sex relationships, making it their top story this morning. Not only did Deseret News find no flaws in the study (Surprise!), they even made it the subject of an editorial saying that “family structure counts” in the childhood development. “Those findings,” the editorial reads, “if true, are significant because in the legal battles over same-sex marriage it has been largely accepted that there is no discernable (sic) difference between the outcomes for children of heterosexual parents and the children of gay parents.”
But are those findings true? Anyone who has an understanding of the design of experiments would look at the study and see its obvious flaws. Deseret News, unsurprisingly, saw no flaws, nor did any of the experts they contacted to comment on it. The one person in a position to be critical of the study could not comment on it because Deseret News refused to show a copy of it to him, saying they were honoring an embargo of that study. Unfortunately, all the other commenters Deseret News interviewed who applauded the study were privy to what the study said. They weren’t restrained by the embargo.
I think you can see what’s happening. Robert George is a senior fellow at the Whitherspoon Institute, which provided $695,000 for this study. He also sits on the board of the Bradley Foundation, which provided an additional $90,000. George also just happens to be a member of the Deseret News Editorial Advisory Board. The man gets around. That’s not to say that Robert George is the man responsible for all of this, but I do think it’s safe to say that there will be a massive media campaign orchestrated around this study.
Update: The Washington Times also has a laudatory article, although it’s considerably shorter. Again, no voices are available to intelligently discuss the study. “Jennifer Chrisler, executive director of the Family Equality Council, declined to comment on the studies, which she had not seen.” [Emphasis added.] Come on! What’s with the secrecy?
First Look at Mark Regnerus’s Study on Children of Parents In Same-Sex Relationships
June 10th, 2012
A new study slated for the July issue of the journal Social Science Research claims to show significant differences in adverse outcomes among children raised by gay and lesbian parents when compared to children raised by both biological parents in a heterosexual-headed household. The study is not yet online but I have been provided an advance copy. [Update: it is now available for purchase.] It is by Mark Regnerus of the University of Texas’s Department of Sociology and Population Research Center, and is titled, “How Different are the Adult Children of Parents Who Have Same-Sex Relationships? Findings from the New Family Structures Study.”
This study finds “numerous, consistent differences, especially between children of women who have had a lesbian relationship and those with still-married (heterosexual) biological parents.” The results of this study would up-end some thirty years of established scientific research which showed that gay and lesbian parents are, on the whole, just as good as their straight counterparts. It would, at least, if the study’s methodology were designed to prove that point. But as is the case with all studies, the conclusions drawn by this study are only as good as the methodologies used to inform them.
The Study’s Sample
On that score, there is one significant strength to this study which makes it stand out. Unlike prior studies, the New Family Structures Study (NFSS) is based on a national probability sampled population. This is the gold standard for all social science studies, and it’s extremely rare for a study to achieve that mark. As far as I am aware, all of the studies to date of gay and lesbian parenting use non-representative convenience samples. National probability samples, unlike convenience samples, are important because they alone can be generalized to the broader populations, to the extent that key characteristics in the design of the probability sample (demographics, etc.) match those of the general population. Convenience samples can’t do that. (For more information on convenience samples versus national probability samples, click here.)
So why don’t the other studies use national probability samples? Believe me, every researcher would much rather work with national probability samples instead of convenience samples. But virtually nobody can afford the huge cost of putting such a study together. It is a massive undertaking, and the cost of creating such a data set is just too prohibitive. Regnerus however has overcome this limitation (PDF: 74KB/12 pages) with a generous $695,000 grant from the Witherspoon Institute and a supplemental $90,000 grant from the Bradley Foundation. With more than three quarters of a million dollars, he has the kind of funding that other researchers can only dream of.