|
|
Chapter 11: Good Polls, Good Pollsters, Good SponsorsBy Alan F. Kay, PhD
© 2004, (fair use with attribution and copy to author)
Aug. 16, 2004
Good sponsors must take responsibility for hiring good pollsters and conducting good polls. But how does a sponsor become a good sponsor and how can a good sponsor convince the public audiences for its polls that its poll findings can be trusted? This book has given many tips on how to spot the spin in polls and more generally what, besides the absence of spin, is needed to make a good poll. But what does a sponsor that wishes to produce high quality political polls have to do to be judged of top quality, and how can it demonstrate that it is aspiring to do something that is much larger and more difficult than current commercial level polling? Such a sponsor is obligated to demonstrate that it is trustworthy, and that sponsor may have to go to great lengths to succeed. Let's look at some of the things that a public-interest polling sponsor could do to help convince others it is performing at a high-standard level.
A Public-interest Polling Sponsor’s Obligations
Primary Purpose Public-interest polling has been defined as political polling where the sponsor’s primary intention is to uncover, as reliably and stably as possible, what the public most wants for governance. The sponsor is presumed competent and sincere. It matters little if the sponsor has other desires, hopes, or intentions for surveys, as long as all decisions as to the design, conduct, analysis, promotion and distribution of the findings give the highest priority to finding the public’s wants for governance. Uncovering the “public-interest” has to be the primary purpose, with priority over all other purposes.
Credibility and Trustworthiness. It is up to the sponsoring organization(s) to establish a relationship with the audiences for its polls to persuade them of its true intentions and thus to demonstrate that it is a credible public-interest polling organization. This can be done in many ways. ATI has used the following ways in varying degrees. It is hard to see why anyone sincerely wishing to be a public-interest poll sponsor would not be pleased to follow most, if not all of them and, where practical, even strengthen them.
| 1. | The methods used to perform the surveys should conform as a minimum to
the highest generally accepted commercial polling standards. |
| 2. | Limitations of the methods used should be acknowledged frankly and
fully, while efforts should be made to remove or overcome significant
limitations. |
| 3. | The survey and question design should be carried out by a team that
includes experts in political polling and in the issue area(s) under
investigation, representing a wide range of viewpoints across the political
spectrum. The team should include some with knowledge of, or easy access
to, the enormous range of policy proposals available from the government,
political leaders, policy organizations, and others. The process of
designing, conducting, and analyzing the survey data should be considered a
collaboration between experts and the public. The important and amazing
results of public-interest polling will not be achieved without both the
experts fulfilling their assigned role and the public making its
judgments. An important duty of the team is to create questions that are
fair, balanced, and accurate, as defined in Locating Consensus for
Democracy, pp. 350-353. |
| 4. | One of the most important functions of the team is seeking, culling and
refining for inclusion in the surveys a wide range of policy-choice
options. This is an absolutely essential feature of public-interest
polling. If the choices offered are only those that leading politicians and
the major news media poll on, it cannot be public-interest polling. To the
extent that the survey team does not fairly and fully carry out its role,
the full benefits of public-interest polling will not be realized. |
| 5. | Every member of the team should sign off on the survey report, should be
recognized for his/her role in the report, and if team members disagree with
any finding, those members’ views should be welcomed, acknowledged, and
carried in some section of the report. |
| 6. | The sponsor should make clear in its written materials, particularly in
each survey report, what its intentions are. It should make plausible why
it wishes to conduct surveys in issues and subjects it has chosen. It
should explain how it is financed, and how it intends to use the resulting
information. All results, including the full and complete wording of the
interviewers’ script and its own analysis, should be made public shortly
after the results are available. If the sponsor can state that it does not
conduct special-interest polling as well, that will add to the sponsor’s
credibility. |
| 7. | The sponsor should be open to cooperating with any other public-interest
polling organization by a willingness to share data and methodology used so
that others can confirm, build on, or find limitations to the sponsor’s
findings. |
| 8. | The sponsor, as long as it remains an active public-interest polling
organization, should encourage members of the public with unusual interests
in the issues it surveys to become involved. Because they are leaders,
pundits, or experts, or have had considerable practical or theoretical
experience in the issue or because the issue impinges on them more than on
others, such people should be invited to submit new versions of policy
proposals or previously untested proposal arguments, both pro and con, for
testing public support, as soon as such can be scheduled into the sponsor’s
ongoing survey research program. If there is any charge at all for such
inclusion, it should not exceed a fair allocation of the survey’s full
costs. The sponsor should be open to including questions in upcoming
surveys that others challenge them to include. |
| 9. | Individuals who are key in the sponsoring organizations or are
themselves sponsors should make clear their own biases on the policy
questions in each survey by saying honestly how they would respond to these
questions. Sponsors should voluntarily take the test that they ask others
to voluntarily take! |
| 10. | The sponsor(s) must take responsibility for managing the design team, especially the pollsters, to make sure that those of the preceding items that the sponsor claims to be following actually are being followed. |
If a sponsor conforms to these, or to most of these, suggestions, I believe that its credibility as a bona fide public-interest polling organization will be quite readily established and will increase as long as it continues to abide by these suggestions. Then, others who claim to poll in the public’s interest would not be able to successfully fake it for very long.
[Click here to read on into Chapter 12 ...]
Alan F. Kay is a mathematician, social scientist and pioneer of public-interest polling. He has authored Locating Consensus for Democracy and numerous public policy articles and holds several patents. (see www.publicinterestpolling.com)
