Aerial view of the 
University of Montreal
Guidelines for Research on the Internet
By Attila Szabo & Robert Frenkl
>
This text was published in a journal article.
Therefore the proper reference for this text is:
Attila Szabo and Robert Frenkl (1996). Consideration of research on Internet:
Guidelines and implications for human movement studies. Clinical Kinesiology
Vol. 50(3), pp. 58-65
The guidelines below were developed on the basis of the authors'
experiences and they were formulated to agree with general
research guidelines, the netiquette, and with the guidelines set by
the American Psychological Association (1990) for research with human
subjects. These guidelines are presented only to facilitate the adoption
of the conventional research methods to novel research projects on Internet.
The authors DO NOT assume responsibility for any negative consequences,
whether direct or consequential, with regard to the use, or exclusive use,
of these guidelines. The use of these guidelines is subject to proper referencing.
© Attila Szabo & Robert Frenkl, 1996
1) Researchers must identify themselves, their affiliation,
and provide contact coordinates to the readers whereby potential
subjects can verify the legitimacy of the study. Inquiries from
the subjects', along with their remarks and/or observations, must
be encouraged. The latter can lead to valuable experiences in the
design of follow-up studies.
2) The potential participants should be assured about the
confidential treatment of personal information. They should be
advised clearly about all the conditions involved in the securing
of confidentiality (e.g., the ways or means of answering).
3) Subjects' consent for participation should be obtained.
This task is a more difficult on the Internet than in laboratory
studies because the subjects' signature cannot be obtained. An
alternative method is the cautioning of the possible subjects
that the completion of the forms, questions, or questionnaires
involved in the inquiry signifies their consent to participate.
This is legally similar to the means used by credit card and
computer software companies. These companies warn costumers that
the use of the credit card or the breaking of the software-
package seal, respectively, legally signifies the acceptance of
the terms and conditions associated with the use of their
product. Apart from this measure, subjects must be reminded that
their participation in the study is totally voluntary.
4) Information regarding the purpose of the study, criteria
for participation, and method of response (i.e., instructions)
must be clearly and straightforwardly presented. The use of
ambiguous words or sentences should be avoided. A friendly, non-
pressuring and attention captivating text should be developed.
Pilot testing of this text, either inside or outside the
Internet, may be useful.
5) Some incentives, that may motivate subjects to answer the
questions, should be introduced. For example, the promise to make
the results available to the readers of the group, when they are
available, may be viewed as an incentive for participation.
6) An answer reliability-checking method should be embedded
in the questions. A large number of questionnaires in Exercise
Science contain items that should be reversely rated (for an
example see the Commitment to Running questionnaire (Carmack &
Martens, (1979)). When oppositional items are rated identically,
deception may be suspected. At that point it is up to the
researcher to decide, in accordance with general methodological
principles of research, what measures should be taken. Whether or
not there are reversely rated items in a given questionnaire, the
internal consistency of the employed questionnaire should be
determined for the Internet sample and compared with the
originally reported internal consistency of the instrument. In
performing this manoeuvre researchers must take in consideration
the comparability of the sample in question with the sample on
which the original internal consistency value was based and set
the acceptable discrepancy between the two values accordingly.
7) The use of forced-choice answers (i.e., questionnaires or
rating scales) is preferred, whenever possible, over open-ended
questions to discourage the respondents from recounting
irrelevant information. (Saris, 1991). Further, the reliability-
checking of the answers to open-ended questions is more difficult
than the verification of the forced-choice answers.
8) The amount of information sought from the subjects should
be reasonable. Posting a study on a Usenet discussion group is
like publishing it in a newspaper or a magazine. The questions
should be short, clear, and interesting to motivate the potential
subjects' cooperation. On the other hand, Web sites may be seen
as more "private". Only people interested in the project (that
they have learned about in another source such as papers, Usenet,
magazines, or other media ads) will visit a given Web site. Given
that the intent to participate is there at the beginning, lengthy
questions (i.e., demanding about 10 to 30 min of response time)
may be posted on specific Web sites. In contrast to the Usenet,
where a person logs-in for other reason(s) than participation in
a study, specifically set-up Web sites or home pages are visited
by people who intentionally wish to take part in a given study.
In giving some thought to the difference, an important question
arises: Is there homogeneity between Usenet respondents who come
across the posted studies "accidentally" and Web site respondents
who make an extra effort for taking part in a studies? The answer
to this question remains to be determined in future studies.
9) The study (in form of a Usenet message) should be posted
at a reasonable frequency. The latter should be decided on the
basis of the size of the targeted discussion group (i.e., number
of messages per week) and the total posted message/answer ratio.
For example, a discussion group may be considered large if there
are more than 100 messages posted every day, medium-sized if the
number of postings is between 30-100, and small if the number of
messages is below 30. It should be noted, however, that the size
of a given discussion group is not proportional to the number of
answers that the researchers can expect. Therefore, the rate of
replies, and the obtainable number of subjects, may be estimated
only from the number of answers received after the first posting.
From the authors' experience, it is reasonable to assume a 20-30%
decline in the answer rate with each posting within a discussion
group (this estimate may change with time).
On a Web site the study should be made available for a
period which is in accord with the sample size required to
achieve the statistical power needed for the analyses of a
particular question. The location of the Web site, or the Web
page address, where the study is posted, should be advertised in
different media. The extent and frequency of advertisement should
be estimated on the basis of the response rate gathered via pilot
projects. If possible and deemed cost-efficient, the target Web
site may be linked to frequently visited home pages. For example,
almost all Universities and Research Centers in the United States
and other developed countries have their own home page that may
be joined by scientists in those institution for research
purposes.
10) If cross-cultural mixing is not desired, the posting
should be targeted not only to a specific group, but also to a
specific geographical location (i.e., North America). This may
be necessary in the examination of sociological issues in which
cultural factors may influence the data. While it is reasonable
to ask participants that they must be fluent in English, or the
language in which the study is posted, at this time no selection
on the basis of ethnicity, religion, income, occupation or sexual
preference should be attempted on the Internet (see point 13).
11) If answers are gathered via electronic mail (e-mail), an
automatic reply, containing a thank you note, should acknowledge
the receipt of the completed questionnaires. A thank you sentence
should be included with the original recruitment message as well.
12) Volunteer subjects should be reminded that if they have
any doubt about the meaning of a question or an instruction, they
should feel free to contact the researcher for an explanation.
13) No studies should be attempted in which specific ethnic,
political, religious, or some sort of class issues are addressed.
Further, no request for such information should be incorporated
in studies on the Internet. Any sensing of discrimination (even
if unintentional) may be viewed as offending by some readers and
could arouse deceptive demeanors and hostility.
14) The solicitation of demographic data should be kept at
minimum. While subjects should be identified somehow, people are
skeptical about giving out personal information. It is easy to
ask the name, gender, age, height, and weight of the subjects,
however, anything else may be a sensitive point. Physical address
of the participants should not be asked (unless it is marked
optional) on the Internet. It arouses skepticism. The Internet
researcher may need to be contented with the name and e-mail
address of the subjects. This is usually a reliable method to
contact a subject when that is needed. If the e-mail address is
insufficient for the scope of a given study, that study may be
difficult to be performed on the Internet.
15) The researcher should not engage in argument(s) with the
readers of the Usenet group on which the study is carried out.
Sometimes critical "counter-messages" are posted by some readers.
If such messages are unfounded, they should be politely explained
through private channels, or in some cases they should be simply
ignored. However, if a "counter-message" contains an instructive
criticism (which may often be the case), the issue should be
addressed. Posting the study on a Web site decreases this problem
because readers may not feel "intruded" by the posted study.
These guidelines are not exhaustive and experience will add
to these points. The result will be a firm framework for research
on the long run. But to get there, the work should begin.
If you have any comments or suggestions with regard to these
guidelines please e-mail the first author at:sportpsyc@geocities.com
Contact the Journal of Computer Mediated Communication
© 1997