Understanding the 2003 NBES Methodology

An Interview with the President of O'Neil Associates

A concise summary of the National Business Ethics Survey (NBES) methodology reads as follows:

The 2003 NBES is a national telephone survey conducted in the United States between the dates of January 31, 2003 and March 17, 2003. Survey respondents were selected from a randomly generated list of residential telephone numbers. A total of 1,503 interviews were completed. Eligible survey respondents had to be at least eighteen years of age, currently employed, working for an organization that employs at least two people; and working at least 20 hours per week for their primary employer. Respondents were assured that their individual responses would remain confidential.

For those not regularly immersed in survey research, the meaning of such methodological descriptions may not be entirely clear. In the following interview, we asked Dr. Michael J. O'Neil, President of O'Neil Associates, Inc. (www.oneilresearch.com) -- the company that managed and conducted the data collection for the 2000 and 2003 NBES -- to help clarify some frequently asked questions for our readers.

Q: What exactly did O'Neil Associates do on the NBES projects?

A. We were selected by the Ethics Resource Center to survey a random sample of the nation's population over the telephone. The interviews were conducted using CATI software from our centralized telephone facility in Tempe, Arizona. In this respect, this project was highly similar to many of the 1300 projects our firm has conducted over the last 20 years.

Q. What is CATI?

A. CATI stands for Computer Assisted Telephone Interviewing. This technology allows our interviewers to read the questions one-at-a-time from a computer screen rather than using pencil and paper. This does not mean computers are conducting the interviews - that job is still performed by humans, but computer make the task of asking survey questions much easier. Using CATI reduces the number of mistakes both in reading complex questions and recording the answers.

Q. Whom did you call to participate in the NBES?

A. We contacted individuals in residential households in the continental U.S. We excluded from the survey persons who were (1) under 18 years of age, (2) not employed at least 20 hours per week or (3) did not work in an organization with at least two employees. We excluded these people because many of the survey questions would have been inapplicable to them.

Q. How did you get their phone numbers?

A. We purchased a list (sample) of numbers that had been selected through Random Digit Dialing. The main implication of Random Digit Dialing is that it provides us with both listed and unlisted numbers. This is important because, as a group, people who have unlisted numbers may answer the survey questions differently than those who have listed numbers.

Q. How did you get numbers for people who are unlisted?

A. The company from which we purchased the numbers has a list of all the working area codes and exchanges in the country. They take those known prefixes and randomly fill in the last four digits to complete a phone number. After they generate a long list of such numbers, they delete known business numbers, numbers connected to machines, and non-working numbers. As you may expect, going through all this trouble makes conducting a survey more expensive, but O'Neil Associates believes, and the Ethics Resource Center concurs that the improved accuracy in results makes it worth the effort.

Q. When did you call people and how often did you call?

A. We conducted all of the interviews between January 31st and March 17th, 2003. We also made multiple follow-up calls (up to 10) in order to interview people whom we could not reach in our initial calls. Ninety percent of telephone numbers not initially reached were called four or more times and two-thirds were called six or more times. Having seven weeks to conduct the survey gave us the time to make a number of these follow-up calls.

Q. How accurate are the survey results?

A. The sampling error for this survey is plus or minus 2.5% or less at the 95% confidence level. This means that if we had repeated the survey 100 times, an average of 95 times out of 100 would yield results within an interval of plus or minus 2.5% around the survey's result. For example, when we report that 65% of employees witness misconduct, the true value of that number is not exactly 65% but it is very likely somewhere between 62.5% and 67.5%.

Q. Can you give us an idea of your qualifications to perform this research?

A. O'Neil Associates has conducted over 1,300 opinion research projects since 1981, for a wide range of clients from Fortune 100 companies to utilities, banks, hospitals, public agencies, and other major institutions.

Personally, I have more than 25 years experience in designing, conducting, and analyzing survey research projects, which includes both doctoral and postdoctoral work in university departments specializing in this type of research. I did graduate work at the National Opinion Research Center of the University of Chicago and postdoctoral studies at the Survey Research Center of the Institute for Social Research at the University of Michigan. The latter appointment was the result of the award of a National Science Foundation Postdoctoral Fellowship in Survey Research Methodology for research and study in the interview methodology section of the Survey Research Center. I was subsequently appointed to the faculty at the University of Michigan and later to the faculty at Arizona State University, where I was Director of the Public Opinion Research Program. I earned my Ph.D in Sociology, with a specialization in research methods, from Northwestern University.