Surveys as a Tool for Training in Scientific Integrity
Michael W. Kalichman
Scientific integrity is about more than rules, regulation, and compliance. Much of what we do as scientists requires decisions that must be made in the absence of clear guidelines. Questions about topics such as data management, publication, and the use of animal or human subjects often represent difficult ethical challenges. To learn about such concepts, which are frequently complex and nuanced, it is essential to have the chance to think actively rather than merely listen to a lecture or read some text. Grappling with tough cases through discussion is one common approach to stimulate an active learning process. One excellent way to generate discussion is to survey attitudes, perceptions, and experience of course participants.
Surveys, like the scenarios used for case study discussion, require trainees to examine their own perceptions and assumptions. Through the process of this reflection, it is possible to refine existing standards, identify new standards, and develop strategies for responding to difficult questions. The characteristic that distinguishes surveys from case study discussions is that answers are not typically open-ended. Instead, the respondents are asked to answer forced-choice questions that are either categorical (e.g., yes/no) or quantitative (e.g., the degree of agreement or disagreement). These answers then can be reduced and summarized for discussion of patterns and correlations within a particular group (e.g., this year’s students) or between groups (e.g., those who have had versus those who have not had training in scientific integrity). Although the subtleties in complex cases may not always emerge in discussion, it is often possible with surveys to elicit information about common trends and attitudes that would otherwise be lost. The following includes both general observations about the use of the sample surveys and some specific comments about the use of each.
The surveys included below parallel the topic areas of this text. The following points should be kept in mind when employing them as teaching tools. First, because some surveys overlap, their selection is at the discretion of the instructor. Further, not all surveys will be appropriate to meet the needs of a specific course, instructor, or group of students. Second, these surveys should not be viewed as definitive; instructors may want to develop new surveys to meet specific instructional objectives. Third, nearly all of these surveys are suitable for administration during class or a workshop, but some may be more appropriate as homework assignments. For purposes of homework or distribution, these forms can be found as PDF files at this book’s website, www.scientificintegrity.net, and printed for convenient use, or suitable response sheets may be prepared by the instructor. Fourth, simply completing these surveys can have value in stimulating reflection on personal values and the normative conduct of science. However, analysis and discussion of survey results are a key part of this exercise. The instructor could do the analyses, but it may be even more valuable to have trainees summarize the data, select results of interest, present their findings, and lead class discussion about interesting results. Usually, it is not necessary for the survey discussants to focus on the responses to each and every item in the survey. Instead, identifying questions that reveal differing attitudes and perceptions on the part of the respondents is desirable. These should be used to stimulate class discussion, allowing the discussants to state their positions and the rationales underlying their responses. Such discussions allow students to invoke their critical thinking skills in articulating their arguments. Equally important, these discussions frequently uncover multiple points of view, many of which have merit and can be appropriately defended.
Using surveys to collect information may fall into the category of human subjects research. In such cases, institutional review board (IRB) approval must be sought before any work is begun (see chapter 5). The definition of human subjects research centers on the fulfillment of criteria related to the subjects and to the investigational process and goals. Human subjects are defined as living individuals about whom an investigator obtains (i) information, specimens, or other data through intervention or interaction with the individual or (ii) identifiable private information. The word “research” is meant to encompass systematic investigation designed to develop or contribute to generalizable knowledge. If you use the surveys in this appendix as tools for stimulating class discussion, and nothing more, your actions do not constitute contributing to generalizable knowledge. Under these conditions, administering these surveys and presenting the resulting data for purposes of discussion do not constitute research. However, the authors of this appendix and this book encourage users to check with their institutions’ review boards to verify whether such use is exempt, can be expedited for review, or requires full IRB review. If your use of these surveys extends beyond the immediate purpose of classroom instruction, it is likely that IRB review and approval would be needed.
Survey 1: Overview (chapters 1 and 2)
This survey is modified from one originally used to study perceptions about research misconduct at the University of California, San Diego (2). This survey has potential value as an introductory exercise for a course in scientific integrity. Ideally, students would be asked to complete the questionnaire immediately before or at the beginning of a workshop or the first meeting of the course. In addition to the raw data being of interest (e.g., what percentage of respondents believe they have firsthand knowledge of plagiarism), secondary analyses and discussion about the meaning of the answers are at least as important. Examples of specific analyses that might be of interest are (i) the correlation between position (question #1), years of experience (#2), or experience as an author (#3) and the answers to questions about misconduct experience (#4 to 13) and (ii) discussion of the various possible interpretations of, for example, 10% of respondents reporting firsthand knowledge of data fabrication or falsification. It can also be of interest to compare results of this survey to those that are published (1, 2).
Survey 2: Research misconduct (chapters 1 and 2)
The primary focus of this survey is research misconduct as defined by federal regulatory agencies to include fabrication, falsification, and plagiarism. Some areas that may lend themselves to fruitful discussion include different types of plagiarism (#1 to 3), distinctions between different kinds of data falsification (#5 versus 6), personal willingness to commit possible misconduct (#4 to 8), responsibilities for whistle-blowing (#10 to 12), and allocation of blame versus punishment (#13 and 14).
Survey 3: Mentoring (chapter 3)
The results of this survey can provide the basis for discussions on the responsibilities of mentoring. All of these questions address potential roles for mentors, heads of research groups, and/or thesis supervisors. One possible use of this survey is to have it completed by both students and their thesis supervisors, followed by a presentation and discussion of the results in class.
Survey 4: Publication (chapter 4)
The initial two questions are based on a brief scenario to distinguish between knowing what one should do and one’s willingness to do so. The following questions address two key areas: (i) the reasons for publishing a paper (#3 to 7) and (ii) a variety of publication practices that may be viewed as more or less acceptable (#8 to 13). For both sets of questions, interesting discussions can result from identifying relative differences and considering possible rationales for those differences. A concluding set of questions (#14 to 17) address criteria for retracting a published manuscript.
Survey 5: Authorship (chapter 4)
This is the second of three surveys on the topic of publication. The opening questions focus on a junior scientist’s dilemma about adding a senior scientist’s name to the list of authors on a forthcoming publication. The remaining questions highlight both what should be criteria for authorship (#3 to 9) and what should not be criteria for authorship (#10 to 15). Typically, these questions will readily reveal a wide range of views about how authorship should be defined.
Survey 6: Peer review (chapter 4)
This survey is specifically designed as a homework assignment. The goals are 3-fold. First, trainees are asked to think about the practice of a manuscript reviewer asking that one of her or his postdocs review a manuscript (#1 and 2). Second, trainees are asked to discuss related questions with at least one active investigator (#3 to 6) as well as offer their own views (#7 to 10). Third, several questions (#11 to 14) focus on the trainee’s responsibility if asked to participate in a review assigned to someone else. Finally, the trainees are asked to think about when it would not be appropriate to accept an assignment for manuscript review (#15 to 17).
Survey 7: Human subjects (chapter 5)
Research with human subjects is distinguished by the obligation to consider and protect the interests of those who have volunteered to be in a research study. The initial questions in this survey (#1 to 7) focus on the role and domain of the IRB. The remaining questions address the circumstances under which potential research subjects might enroll in a research study (#8 to 12) as well as whether a study should be approved (#13) or stopped early (#14). Discussions of these questions are likely to reveal perceptions and attitudes that are mutually incompatible, even though all are potentially acceptable under current regulations.
Survey 8: Animal subjects (chapter 6)
Although there are many specific regulatory controls for the use of animals in research, this remains an area about which the public and even the biomedical science community are sharply divided. The initial questions (#1 to 4) consider the role of the Institutional Animal Care and Use Committee in reviewing research with animal subjects. The remainder of the survey form (#5 to 16) is designed to encourage trainees to think about their personal criteria for accepting or rejecting the use of animals in research. The survey can be completed in class, but it may be useful as a homework assignment to allow for more thoughtful consideration. For the purposes of analysis, it should be of value to compare the relative importance placed on species (where is the cut-off and why?), the adverse consequences of the experiment (pain, distress, or discomfort), and the balance between the utilitarian value of the studies (increased understanding of the mechanisms of cancer versus cosmetic safety) and deontological responsibilities to protect the rights or welfare of the individual. It is to be expected that opinions will vary widely, even among scientists who use animals in their research.
Survey 9: Conflicts of interest (chapter 7)
Financial conflicts of interest in academia and science have become a matter of serious concern in recent years. The survey begins by exploring the risks of conflicts of interest (#1 to 4). Different perceptions about disclosure are next considered (#5 to 9), and the survey concludes with a focus on possible protections from bias due to conflicts of interest (#10 to 13). Discussion of these issues will typically help to sharpen rationales about what we are worried about and what solutions are most likely to have an impact.
Survey 10: Collaboration (chapter 8)
This survey deals with a defining characteristic of modern science: collaborations. The very first question (#1) asks for student perspectives on the importance of collaboration to good research. This is followed by several questions (#2 to 7) about the risk of problems in collaborations. The remaining questions (#8 to 13) list issues that might be seen as important to be addressed before initiating a collaboration. Discussions of these questions are likely to reveal that scientists, unfortunately, tend to not think about these issues until they become a problem.
Survey 11: Data ownership (chapter 9)
This survey opens with a case discussion scenario to address some possible reasons for ambivalence about sharing of research data (#1 to 5). Questions about ownership and sharing of data (#6 to 19) should serve to illustrate the importance of these issues and some of the dilemmas faced by researchers.
Survey 12: Record keeping (chapter 10)
Although most scientists had the experience of an early lab course in which rigorous standards were proposed for keeping a lab notebook, it is not necessarily the case that these standards for record keeping are common practice. The first questions on this survey (#1 to 9) offer a variety of possible characterizations of how research records should be kept. The focus of the remaining questions (#10 to 13) is the disposition of those records following completion of the project. Discussion of the answers to these questions will hopefully reveal the variation in common practice but also encourage a shift toward improved record keeping.
Survey 13: Science, technology, and society (chapter 11)
The application of advances in molecular genetics has created ethical challenges as well as fears related to misuse of these technologies. The survey opens with some questions (#1 to 14) about what people would and would not want to know about their possible future. It is likely that these questions will lead to sharp divisions of opinion during class discussion. This division of opinion then raises further questions about who should give consent for research studies involving searches for genetic predispositions (#15 to 19). The difficulty of such questions is further exacerbated when a study risks providing information about a particular behavioral trait in a given racial group (#20 to 22).
1. Eastwood S, Derish P, Leash E, Ordway S. 1996. Ethical issues in biomedical research: perceptions and practices of postdoctoral research fellows responding to a survey. Sci Eng Ethics 2:89–114.
2. Kalichman MW, Friedman PJ. 1992. A pilot study of biomedical trainees’ perceptions concerning research ethics. Acad Med 67:769–775.
Some organizations conduct public surveys on matters related to research. These reports are generally available in the public domain. Typically, the results are compiled in useful formats and can be presented to catalyze discussion in a variety of relevant areas. For example, a survey done by the Wellcome Trust Monitor reported survey data on such things as public awareness and understanding of science, participation in medical research, and attitudes toward genetics. The list of organizations that engage in such survey activities is given below, along with their URLs. At these sites you can search for survey results or reports that may be useful in generating class discussion. Specific links to current surveys of interest posted by these organizations will be listed and updated on this text’s companion website: http://www.scientificintegrity.net.
Science and Engineering Indicators, National Science Foundation
Virginia Commonwealth University Life Sciences Surveys