From LISTSERV@uacsc2.albany.edu Tue Jan 5 16:00:22 1993 Date: Tue, 5 Jan 1993 15:55:18 -0500 From: Revised List Processor (1.7e) Subject: File: "EJRNL V1N1" To: pirmann@cs.rutgers.edu _______ _________ __ / _____/ /___ ___/ / / / /__ / / ______ __ __ __ ___ __ ___ _____ / / / ___/ __ / / / __ / / / / / / //__/ / //__ \ / ___ \ / / / /____ / /__/ / / /_/ / / /_/ / / / / / / / / /__/ / / / /______/ /______/ /_____/ /_____/ /_/ /_/ /_/ \___/_/ /_/ March 1991 _EJournal_ Volume 1 Issue 1 ISSN # 1054-1055 An Electronic Journal concerned with the implications of electronic networks and texts. University at Albany, State University of New York ejournal@albnyvms.bitnet There are 421 lines in this issue. CONTENTS: Electronic Journals of Proposed Research 226 lines. by Robert K. Lindsay Mental Health Research Institute University of Michigan DEPARTMENTS: Letters 11 lines. Reviews 10 lines. Supplements to previous texts 12 lines. Pointers to texts appearing elsewhere 21 lines. Information about _EJournal_ 92 lines. ------------------------------------------------------------------------------- This electronic publication and its contents are (c) copyright 1991 by _EJournal_. Permission is hereby granted to give away the journal and its contents, but no one may "own" it. Any and all financial interest is hereby assigned to the acknowledged authors of individual texts. This notification must accompany all distribution of _EJournal_. ------------------------------------------------------------------------------- Electronic Journals of Proposed Research by Robert K. Lindsay Mental Health Research Institute University of Michigan Asking the right question and asking the question right, it is often said, are the most important steps in science. After that, some say (exaggerating for emphasis), answering the question is just a mopping up operation. Nonetheless, the bulk of scientific publications are devoted to reports of the "mopping-up." The question-posing steps are most commonly relegated to grant proposals, where they are examined by a small number of reviewers, at times ill-matched to the task. Thus, proposals do not benefit from the open peer commentary that scientific publications receive, nor do they elicit recognition and reward, except, at times, in the form of money. It is safe to assume that some good ideas, especially if they are truly novel, never see the light of public scrutiny nor receive the resources necessary to carry them to fruition. The other side of the coin is that some effort, at least, is devoted to executing flawed designs or answering unimportant questions, as reflected in the remarkably low citation rates of research papers (Hamilton, 1990, 1991). In emerging fields, especially, the result is often that published research is followed by retractions, qualifications, and belated criticisms. Further, it is likely that some of the best suggestions are delayed or withheld altogether >from the proposal process because their originators lack the time or resources to carry them to the point where the rewards of recognition will accrue, and do not wish others to reap the benefits of developing them. Why do we not concentrate our most intense scrutiny on the most important steps? The current system of limited, anonymous, and selective peer review without opportunity for rebuttal or clarification is insufficient. Far more productive would be journals of open peer commentary (following the format of the very successful Behavioral and Brain Sciences) that publish research questions and experimental designs, followed by a dialogue of counterproposals and modifications before the expensive and time intensive steps of experimentation, field work, model construction, and data analysis are performed. The result would be detailed and carefully designed studies of acknowledged importance. These proposals would then be in the public domain: they could be carried out by anyone with the means and skill, or they could be referred to in applications to funding agencies. The emerging electronic global communications network will soon make possible the implementation of such publication channels in a manner far more useful than print. The fruits of efforts now underway to exploit the potential for electronic collaboration has opened the door to electronic publication. This technology could do more than simply improve the efficiency of document dissemination and speed the processing of proposals in the old mode. The opportunity to publish more than research "results" will be at hand. Electronic dissemination, combined with the publication of proposals, hypotheses, experimental designs, and criticism, could reward those who excel at idea generation but eschew the later stages, speed the progress of science, save resources now devoted to projects destined to be inconclusive, widen, rationalize, and make more fair the grant review process, while easing the (also unrewarded) burdens of grant review. Further, it could decrease the frequency with which good ideas lie fallow because of the vagaries of referee selection and the current review process. Mention of electronic networks conjures visions of bulletin boards, with scraps of paper tacked up, electronically, into a hodge-podge of junk mail that has so little of merit that most of us soon stop searching for the good stuff. The present suggestion is not intended to be a bulletin board, but rather a refereed journal. The most common contribution for the journal would not be a one paragraph "idea," but a detailed proposal of several pages that includes hypotheses, experimental designs, and the logic of the argument. In other words, the typical proposal would be similar to the scientific portion of an NSF or NIH proposal, without the investigator and institutional information and all of the other administrative paperwork. Editors and referees would select proposals that were serious and had prima facie merit, and request some referees to write reviews. Referees would evaluate and critique only the ideas, not the qualifications of the proposer or institution. The proposal would be "published" immediately, and the reviews would be distributed as they came in. Short, unsolicited reviews would also be distributed as they were received and edited (not refereed). The result could be the source of information for a revised and presumably much better proposal which could serve as the basis of a conventional proposal to a funding agency, either from the originator of the contribution or from someone else. The funding agency would be immensely helped by the extensive, documented review, and awards decisions could be made much more rapidly, with less money wasted pursuing ill-conceived ideas. The central idea of this proposal is to extend critical peer review to the earliest and presumably most critical steps of science, the problem and/or experiment definition. This would be of some value even if carried out in traditional ways. Such review is in fact now done for the very largest projects, where the costs of mistakes are most obvious. The human genome project, for example, is going through an extensive planning and review stage. Most large engineering and business projects are extensively reviewed before investing in them. It is my contention that smaller scientific projects should also receive broader review than they now get. This is not primarily to give overlooked ideas a better chance at funding, but rather to improve the proposals that do get funded before serious design flaws cost time and money. Savings are increasingly important, since the federal science budget has not kept up with the supply of researchers and the cost of equipment (Lederman, 1991). I do not think that the present system of peer review is always appropriate, because it is too narrowly specialized, anonymous, overly selective, unrewarding to the referees, and slow. In electronic networks I see the possibility of correcting these problems. An electronic journal of proposed research is based on the assumption that refereed and edited material could benefit from the speed of distribution, wide accessibility, machine searching and editing, and low user marginal cost that are the most important merits of electronic distribution. Such journals would not be substitutes for current funding procedures. In fact, the journals would not be involved in making monetary awards, so the conventional route would still be needed to obtain funding. The open peer review process is rather an alternative for those who, for whatever reason, choose to use it. Any investigator unwilling to submit a proposal for open inspection and perhaps implementation by another person would simply not do so. Currently there are, of course, many grant awards made primarily on the basis of the institution/investigator qualifications and abilities, with scientific specifics being secondary. That is entirely appropriate and should continue; my proposed journals do not address the issue of reviewing investigator qualifications, but would confine themselves to scientific questions. I believe there are many occasions where the new avenue would be attractive to serious workers. What would induce a person with a good research idea to use this avenue rather than the customary method of seeking funding and the opportunity to carry the idea to fruition, thus obtaining the intellectual credit that goes with this? There are several possible motivations. Science has long been compartmentalized as different disciplines, and the specializations get narrower and narrower. Some sciences also specialize along methodological lines: experimental versus theoretical, apparatus A versus apparatus B. Yet it is still not institutionally acknowledged that some people are better at generating ideas, some are better at criticism, some are better at seeing the implications of hypotheses, and some are better in the laboratory or at the computer. It is time to exploit these differences and reward them equally with recognition and citation, while bringing our best weapon of reason-- open peer commentary--to bear on all aspects of scientific work. Second, some research requires expensive or one-of-a-kind equipment and the appropriate infrastructure of human and other resources. Not everyone with appropriate knowledge and creativity has access to the necessary facilities, which cannot be duplicated for a single project. Third, many of our most highly creative scientists already have a full plate of research, and would be eager to see some of their good "excess" ideas explored. Then, too, there may be cases in which a proposer fully intends to do the research but wishes to have the benefit of constructive advice from a scientific community that extends beyond a limited network of colleagues and advisors. In still other cases, the "mopping up" operation may indeed be straightforward and could be done by support staff elsewhere at low cost. Finally, and more speculatively, there may in the future emerge another source of research ideas born of computer technology and artificial intelligence. The scientific literature is a vast warehouse of information which certainly must contain the seeds of many important discoveries. Yet these seeds are dispersed among different, often disconnected literatures that are not commanded by any specialist. Human efforts augmented by electronic analysis of available literature databases such as MEDLINE have in a few cases already revealed previously unappreciated connections that suggest new experiments (see Swanson, 1987, 1989, 1990). Such discoveries have been and will continue to be made by information specialists rather than those who are qualified to pursue the necessary research. An Electronic Journal of Proposed Research (EJPR) would provide an avenue to make such ideas public. For an EJPR to succeed, it is important that it be edited and that it be indexed into appropriately sized subdisciplines to permit automated, selective browsing and dissemination, so that readers would not have to sift through large files of material on subjects foreign to them. It must provide rapid turnaround; and most important, there must be recognition of the contributors. Validating priority of ideas is also important, although this is not a new problem. If anything, electronic distribution reduces this problem in comparison to print distribution which is slower, less accessible, and more coarsely time stamped. As I noted above, this proposal is not primarily motivated by the fact that good ideas go unfunded, although pointing out that good ideas are not always recognized is not a sour grapes argument, any more than pointing out that awardees are generally worthy is an elitist argument. The beliefs that no good ideas are overlooked by current procedures and that all funded research is the best it can be are too preposterous to bother to counter with specific examples. Problems with current procedures have been noted with concern (for example, Marshall, 1987; Muller, 1980) and even with alarm (Snyder, 1985), and various suggestions have been offered for reform (Koshland, 1985). I have suggested an improvement that addresses these concerns in a new way: wider, open peer review of scientific content, followed by revision and competition for funding, expedited by the wide accessibility and manipulation of documents through currently available technology. Although there would be many such journals in the sense that there would be many specialized editorial boards, they would be one journal in the sense that access over the electronic network is uniform. I have not seen this proposed previously, and I think that it has sufficient potential to merit serious consideration. References Hamilton, D. P. (1990) Publishing by - and for? - the numbers. Science 250(4986): 1331-1332. Hamilton, D. P. (1991) Research papers: Who's uncited now? Science 251(4989): 25. Koshland, D. E. (1985) Modest proposals for the granting system. Science 229(4710): 231 Lederman, L. M. (1991) Science: The end of the frontier? Science. Supplement to Volume 251, Number 4990, January 11, 1991. Marshall, E. (1987) Gossip and peer review at NSF. Science 238(4833): 1502. Muller, R. A. (1980) Innovation and scientific funding. Science 209(4459): 880-883. Snyder, L. There are problems with the review process. Communications of the ACM. 28(4): 349-350. Swanson, D. R. (1987) Two medical literatures that are logically but not bibliographically connected. J. American Society for Information Science, 38(4): 228-233. Swanson, D. R. (1989) A second example of mutually isolated medical literatures related by implicit, unnoticed connections. J. American Society for Information Science, 40(6): 432-435. Swanson, D. R. (1990) Somatomedin C and arginine: Implicit connections between mutually isolated literatures. Perspectives in Biology and Medicine. 33(2): 157-186. [ This article in Volume 1 Issue 1 of _EJournal_ (March, 1991) is (c) copyright _EJournal_. Permission is hereby granted to give it away. _EJournal_ hereby assigns any and all financial interest to Robert K. Lindsay. This note must accompany all copies of this text. ] Robert K. Lindsay robert_lindsay@ub.cc.umich.edu 205 Washtenaw Place Ann Arbor, Michigan 48109 ------------------------------------------------------------------------------- Letters: _EJournal_ is willing publish letters to the editor. But at this point we make no promises about how many, which ones, or what format. Because the "Letters" column of a periodical is a habit of the paper environment, we can't predict exactly what will happen in pixel space. For instance, _EJournal_ readers can send outraged objections to our essays directly to the authors. Also, we can publish substantial counterstatements as articles in their own right, or as "Supplements." Even so, there will probably be some brief, thoughtful statements that appear to be of interest to many subscribers. When there are, they will appear as "Letters." ------------------------------------------------------------------------------- Reviews: _EJournal_ is willing to publish reviews of almost anything that seems to fit under our broad umbrella: the implications of electronic networks and texts. Right now we are hoping to review one hypertext novel, and have no other works-- electronic or printed --under consideration. We do not solicit and cannot provide review copies of fiction, prophecy, critiques, other texts, programs, hardware, lists or bulletin boards. But if you would like to bring any publicly available information to our readers' attention, send your review (any length) to us, or ask if writing one sounds to us like a good idea. ------------------------------------------------------------------------------- Supplements: _EJournal_ plans to experiment with ways of revising, responding to, re- working, or even retracting the texts we publish. Authors who want to address a subject already broached --by others or by themselves-- may send texts, preferably brief, that we will consider publishing under the "Supplements" heading. Proposed "supplements" will not go through full, formal editorial review. Whether this "Department" will operate like a delayed-reaction bulletin board or like an expanded letters-to-the-editor space, or whether it will be withdrawn in favor of a system of appending supplemental material to archived texts, or will take on an electronic identity with no direct print- oriented analogue, will depend on what readers/writers make of the opportunity. ------------------------------------------------------------------------------- Pointers to text appearing elsewhere: This "Department" is Joe Raben's idea. It will appear whenever readers send similar citations. Here is Joe's initial list, now several months older than when he sent it, along with his suggestion: Joe Raben: "One service you might add is a list of relevant articles in places most of your readers would not be likely to look for them. For example, the following just appeared in _Signs: Journal of Women in Culture and Society_ 16:1 (Fall 1990): Ruth Perry and Lisa Gruber, "Women and Computers: An Introduction" Paul N. Edwards, "The Army and the Microworld: Computers and the Politics of Gender Identity" Sherry Turkle and Seymour Papert, "Epistemological Pluralism: Styles and Voices within the Computer Culture" Pamela E. Kramer and Sheila Lehrman, "Mismeasuring Women: A Critique of Research on Computer Ability and Avoidance" ------------------------------------------------------------------------------- Information about _EJournal_: Users on both Bitnet and the Internet may subscribe to _EJournal_ by sending an E-mail message to this address: listserv@albnyvm1.bitnet The following should be the only line in the message: SUB EJRNL Subscriber's Name Please send all other messages and inquiries to the _EJournal_ editors at the following address: ejournal@albnyvms.bitnet _EJournal_ is an all-electronic, Bitnet/Internet distributed, peer-reviewed, academic periodical. We are particularly interested in theory and praxis surrounding the creation, transmission, storage, interpretation, alteration and replication of electronic text. We are also interested in the broader social, psychological, literary, economic and pedagogical implications of computer-mediated networks. The journal's essays will be available free to Bitnet/Internet addresses. Recipients may make paper copies; _EJournal_ will provide authenticated paper copy from our read-only archive for use by academic deans or others. Individual essays, reviews, stories-- texts --sent to us will be disseminated to subscribers as soon as they have been through the editorial process, which will also be "paperless." We expect to offer access through libraries to our electronic Contents, Abstracts, and Keywords, and to be indexed and abstracted in appropriate places. Writers who think their texts might be appreciated by _EJournal_'s audience are invited to forward files to ejournal@albnyvms.bitnet . If you are wondering about starting to write a piece for to us, feel free to ask if it sounds appropriate. There are no "styling" guidelines; we would like to be a little more direct and lively than many paper publications, and less hasty and ephemeral than most postings to unreviewed electronic spaces. Some subscribers may notice that we had to make up an incorrect name for you when we moved our distribution list to the Listserv utility. You can change it to whatever you want by sending the SUB message (above), using the name you prefer. This issue's "feature article," and those from other issues of _EJournal_, will eventually be available from a Fileserv at Albany. We plan to distribute a "table of contents" to a broad population occasionally, along with instructions for downloading. We are aware that leaving an essay's references at the end of the text makes it clumsy to consult the citations and return to a place in the text. We are trying to work out a convention--perhaps even "footnotes"--that will make the process easier. ------------------------------------------------------------------------------- Board of Advisors: Dick Lanham, University of California at Los Angeles Ann Okerson, Association of Research Libraries Joe Raben, City University of New York Bob Scholes, Brown University Harry Whitaker, University of Quebec at Montreal ------------------------------------------------------------------------------- Consulting Editors - March 1991 - Inaugural Issue ahrens@hartford John Ahrens Hartford ap01@liverpool Stephen Clark Liverpool crone@cua Tom Crone Catholic University dabrent@uncamult Doug Brent Calgary djb85@albnyvms Don Byrd Albany donaldson@loyvax Randall Donaldson Loyola College ds001451@ndsuvm1 Ray Wheeler North Dakota eng006@unomal Marvin Peterson Nebraska - Omaha erdt@vuvaxcom Terry Erdt Villanova fac_aska@jmuvax1 Arnie Kahn James Madison folger@yktvmv Davis Foulger IBM - Watson Center george@gacvax1 G. N. Georgacarakos Gustavus Adolphus gms@psuvm Gerry Santoro Pennsylvania State University geurdes@hlerul55 Han Geurdes Leiden jtsgsh@ritvax John Sanders Rochester Institute of Technology nrcgsh@ritvax Norm Coombs Rochester Institute of Technology pmsgsl@ritvax Patrick M. Scanlon Rochester Institute of Technology r0731@csuohio Nelson Pole Cleveland State ryle@urvax Martin Ryle Richmond twbatson@gallua Trent Batson Gallaudet usercoop@ualtamts Wes Cooper Alberta University at Albany Computing Services Center: Isabel Nirenberg, Bob Pfeiffer, Kathy Turek; Ben Chi, Director ------------------------------------------------------------------------------- Editor: Ted Jennings, English, University at Albany Managing Editor: Ron Bangel, University at Albany ------------------------------------------------------------------------------- State University of New York University Center at Albany Albany, NY 12222 USA