FEBRUARY 2002

Post-September 11th Test of Methods to Improve Mail Response Rates

       Following the attacks that occurred on September 11th and the subsequent anthrax scares, Beta Research and McPheters & Company entered into testing of various methods to lift respondent confidence in the validity of the survey instrument and, hence, improve response rates. The first test involved an 8-cell design that looked at the effects of various combinations of variables on response rates to mail surveys. The second involved use of a telephone pre-recruit to obtain consent to fill out the same mail questionnaire also used in Test I.

TEST I

Test I involved use of a cover letter, $1 incentive, and 1-page (front and back) questionnaire, which was mailed to 8 nationally representative sample replicates of 250 each. Advance postcards were mailed on November 9th, and surveys were mailed on November 14th. Combinations of three variables were tested. The variables were:

• Use of an advance postcard alerting the respondent that the survey would be arriving shortly.
• Contact name and phone number, appearing on the envelope, so respondents could call to verify the validity of the survey.
• Use of Peel 'N Seal response envelopes, so respondents would not need to lick the envelope.

The control cell did not make use of any of the methods tested; i.e., there was no advance postcard and no contact name. It included a response envelope and, as was the case with every cell, live postage was affixed to the envelope. All fieldwork was closed on December 19th.

The advance postcard in combination with the regular envelope and the name and phone number of a Beta contact generated the highest response rate Ð 40.5%. This response rate was significantly higher than all others involving the use of a regular envelope Ð and more than a third higher than the control cell. Interestingly, not only did a smaller proportion of the sample respond to the control cell, but respondents in this cell were significantly less affluent than respondents in the remaining cells.

Our conclusion is that advance postcard notification and the name and phone number of a contact with whom survey validity can be verified, in combination with a response envelope with live postage, increases respondent confidence and results in higher response rates. We expect to be using this method to generate higher response rates for our clients in the months ahead.

TEST II

Prior to November 14th, we used an RDD sample to get up-front commitment to fill-out a mail questionnaire. We called 7008 numbers, of which 1831 were ineligible by virtue of being business or government offices, fax/modem lines, disconnected, beepers, etc. The remaining 5177 eligible numbers yielded 993 respondents who agreed to participate Ð or a yield of 19.2%.

On November 14th, the questionnaire was then mailed out to the 993 who agreed to fill it out. The package mailed was the same as that used for the control cell in Test I. As was the case with Test I, the fieldwork was concluded on December 19th. Of the 993 mailed, we received completed questionnaires from 659 Ð or 66.4%. The demographic profile of the respondents was similar to that for the other test cells. While the net response rate was a low 12.7%, we believe that for certain low incidence populations this method may have applicability, in that it provides a way to undertake at reasonable cost, research that would otherwise be cost prohibitive.

SUMMARY OF TESTS

Use of advance post cards and identification of a contact with whom survey validity can be confirmed increase response to mail surveys. While a telephone pre-recruit generates relatively low response rates in the initial phase, it yields high rates of cooperation in the second phase, and provides a relatively cost-effective way to survey low incidence populations or to conduct surveys where the cost/survey administered is unusually high.

We expect to conduct further experimental designs over the course of this year in an effort to enhance both telephone and mail response. If you would like to receive copies of earlier write-ups regarding other steps that can be used to enhance response rates, please do not hesitate to call, write, or email us.

Back to Top