Box Turtle Bulletin

Fun With Polls

In the wake of the California Supreme Court’s decision on marriage equality, there have been several polls attempting to measure the reaction of the state’s citizens. They have had conflicting results.

An LA Times poll reported May 23 tells us that Californians oppose gay marriage by 52% to 41%. A Field poll released five days later reported just the opposite, that Californians favor gay marriage by 51% to 42%.

If we believe a survey by the anti-gay activist group Capital Resource Institute, Californians support banning gay marriage by 56%. If we were to accept a USA Today / Gallup poll as it is being reported, we would believe that two thirds of Americans favor gay marriage.

Why are there so many contradictory conclusions? Part of the answer can be found in the way that questions are presented.

Take, for example, the USA Today poll. In this, the respondent was asked to determine if the decision to marry was “strictly a private decision between the two people” or whether “the government has the right to pass laws to prohibit or allow such marriages” for a series of hypothetical couples. Respondents were asked about mixed religion and mixed race marriages along with same sex couples.

The dichotomy between “private” and “government prohibition” along with the grouping of same-sex with mixed-marriage and mixed-faith couples is almost certain to yield results that have little or no reflection on how most Americans view gay marriage.

There are undoubtedly those who think that a union between two persons of the same-sex should be private but who also believe that it should not be recognized by the state. And without the leading questions about currently illegal marriage prohibitions, the respondants would not be coached into rejecting same-sex prohibitions.

These types of polls where a desired result is falsely constructed are called “push polls” and are favorites of political campaigns that seek to present their candidate or issue as a winner.

The claims of the anti-gay Capital Resource Institute can also be dismissed completely. CRI didn’t even pretend to use a credible polling agency, relying instead on an advertising agency that “ensure[s] that [their] political, public policy and service organization clients have their messages reach the households they have targeted, usually based on location or anticipated household demographics.”

But neither the LA Times nor the Field poll were constructed to yield a desired result. The Times asked:

Do you approve or disapprove of the California Supreme Court’s decision last week to allow same-sex marriage in California?

and allowed “strongly approve”, “somewhat approve”, “somewhat disapprove”, “strongly disapprove”, and “don’t know” as answers. The Field Poll allowed only “approve”, “disapprove” or “no opinion” and asked:

Do you approve or disapprove of California allowing homosexuals to marry members of their own sex and have regular marriage laws apply to them?

The questions about voting on the constitutional amendment were also similarly worded:

Times: A proposed amendment to the state’s Constitution that may appear on the November ballot would reverse the court’s decision and state that marriage is only between a man and a woman. If the election were held today, would you vote for or against the amendment.

Field: There may be a vote on this issue in the November election. Would you favor or oppose having the state constitution prohibit same-sex marriage, by defining marriage as only between a man and a woman?

The Times found the amendment passing 51% to 36% and the Field Poll found it failing 51% to 43%.

So how do we decide which poll to believe? Are we to be encouraged or worried?

ABC New’s polling director, Gary Langer, provides some guidance:

Sample differences can matter (the Times poll was among all adult Californians, the Field Poll among registered voters only, and both noted big differences among areas of the state and demographic groups). Timing can matter, too (the Field Poll was done May 17-26, an unusually long 10-day field period; the Times poll, May 20-21, a short one). So can the order of questions, and these are worth a look.

Langer states that “Both polls are high-quality, with clear, balanced questions” and does not conclude as to which poll best reflects public sentiment.

So I guess the answer is that it’s just not possible to tell at this time.

For those who need extra encouragement, you can look to how well the Field Poll compared to California’s Proposition 22, an initiative that restricted marriage (on a stututory level) to opposite-sex couples. If we can guestimate from this graph, in 2000 about 40% of Californians supported gay marriage. About 39% of California voters opposed the proposition. This suggests that the Field Poll is not necessarily far off from the opinions of voters.

However, as the conflicting polls show, opinion on this issue is difficult to measure and may be subject to influence. It is of utmost importance that a carefully crafted campaign be designed and funded to appeal to the better nature of California voters.