Workshop on Electronic Texts · Library of Congress
SESSION II. SHOW AND TELL
AM also has a manuscript collection, the life history portion of one of the federal project series, which will contain 2,900 individual documents, all first-person narratives. AM has in process about 350 African-American pamphlets, or about 12,000 printed pages for the period 1870-1910. Also in the works are some 4,000 panoramic photographs. AM has recycled a fair amount of the work done by LC's Prints and Photographs Division during the Library's optical disk pilot project in the 1980s. For example, a special division of LC has tooled up and thought through all the ramifications of electronic presentation of photographs. Indeed, they are wheeling them out in great barrel loads. The purpose of AM within the Library, it is hoped, is to catalyze several of the other special collection divisions which have no particular experience with, in some cases, mixed feelings about, an activity such as AM. Moreover, in many cases the divisions may be characterized as not only lacking experience in "electronifying" things but also in automated cataloguing. MARC cataloguing as practiced in the United States is heavily weighted toward the description of monograph and serial materials, but is much thinner when one enters the world of manuscripts and things that are held in the Library's music collection and other units.
In response to a comment by LESK, that AM's material is very heavily photographic, and is so primarily because individual records have been made for each photograph, FLEISCHHAUER observed that an item-level catalog record exists, for example, for each photograph in the Detroit Publishing collection of 25,000 pictures. In the case of the Federal Writers Project, for which nearly 3,000 documents exist, representing information from twenty-six different states, AM with the assistance of Karen STUART of the Manuscript Division will attempt to find some way not only to have a collection-level record but perhaps a MARC record for each state, which will then serve as an umbrella for the 100-200 documents that come under it. But that drama remains to be enacted. The AM staff is conservative and clings to cataloguing, though of course visitors tout artificial intelligence and neural networks in a manner that suggests that perhaps one need not have cataloguing or that much of it could be put aside. The matter of SGML coding, FLEISCHHAUER conceded, returned the discussion to the earlier treated question of quality versus quantity in the Library of Congress. Of course, text conversion can be done with 100-percent accuracy, but it means that when one's holdings are as vast as LC's only a tiny amount will be exposed, whereas permitting lower levels of accuracy can lead to exposing or sharing larger amounts, but with the quality correspondingly impaired.
In pragmatic terms, funding from the Packard Foundation would expedite the transcription of thousands of documents waiting to be put on disk in the WPP offices. Further, since the costs of collecting, editing, and converting the Founding Fathers documents into letterpress editions were running into the millions of dollars, and the considerable staffs involved in all of these projects were devoting their careers to producing the work, the Packard Foundation's suggestion had a revolutionary aspect: Transcriptions of the entire corpus of the Founding Fathers papers would be available on CD-ROM to public and college libraries, even high schools, at a fraction of the cost-- $100-$150 for the annual license fee--to produce a limited university press run of 1,000 of each volume of the published papers at $45-$150 per printed volume. Given the current budget crunch in educational systems and the corresponding constraints on librarians in smaller institutions who wish to add these volumes to their collections, producing the documents on CD-ROM would likely open a greatly expanded audience for the papers. TWOHIG stressed, however, that development of the Founding Fathers CD-ROM is still in its infancy. Serious software problems remain to be resolved before the material can be put into readable form.
Funding from the Packard Foundation resulted in a major push to transcribe the 75,000 or so documents of the Washington papers remaining to be transcribed onto computer disks. Slides illustrated several of the problems encountered, for example, the present inability of CD-ROM to indicate the cross-outs (deleted material) in eighteenth century documents. TWOHIG next described documents from various periods in the eighteenth century that have been transcribed in chronological order and delivered to the Packard offices in California, where they are converted to the CD-ROM, a process that is expected to consume five years to complete (that is, reckoning from David Packard's suggestion made several years ago, until about July 1994). TWOHIG found an encouraging indication of the project's benefits in the ongoing use made by scholars of the search functions of the CD-ROM, particularly in reducing the time spent in manually turning the pages of the Washington papers. TWOHIG next furnished details concerning the accuracy of transcriptions. For instance, the insertion of thousands of documents on the CD-ROM currently does not permit each document to be verified against the original manuscript several times as in the case of documents that appear in the published edition. However, the transcriptions receive a cursory check for obvious typos, the misspellings of proper names, and other errors from the WPP CD-ROM editor. Eventually, all documents that appear in the electronic version will be checked by project editors. Although this process has met with opposition from some of the editors on the grounds that imperfect work may leave their offices, the advantages in making this material available as a research tool outweigh fears about the misspelling of proper names and other relatively minor editorial matters.
Completion of all five Founding Fathers projects (i.e., retrievability and searchability of all of the documents by proper names, alternate spellings, or varieties of subjects) will provide one of the richest sources of this size for the history of the United States in the latter part of the eighteenth century. Further, publication on CD-ROM will allow editors to include even minutiae, such as laundry lists, not included in the printed volumes. It seems possible that the extensive annotation provided in the printed volumes eventually will be added to the CD-ROM edition, pending negotiations with the publishers of the papers. At the moment, the Founding Fathers CD-ROM is accessible only on the IBYCUS, a computer developed out of the Thesaurus Linguae Graecae project and designed for the use of classical scholars. There are perhaps 400 IBYCUS computers in the country, most of which are in university classics departments. Ultimately, it is anticipated that the CD-ROM edition of the Founding Fathers documents will run on any IBM-compatible or Macintosh computer with a CD-ROM drive. Numerous changes in the software will also occur before the project is completed. (Editor's note: an IBYCUS was unavailable to demonstrate the CD-ROM.) ****** +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ DISCUSSION * Several additional features of WPP clarified * +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Discussion following TWOHIG's presentation served to clarify several additional features, including (1) that the project's primary intellectual product consists in the electronic transcription of the material; (2) that the text transmitted to the CD-ROM people is not marked up; (3) that cataloging and subject-indexing of the material remain to be worked out (though at this point material can be retrieved by name); and (4) that because all the searching is done in the hardware, the IBYCUS is designed to read a CD-ROM which contains only sequential text files. Technically, it then becomes very easy to read the material off and put it on another device.
LEBRON remarked several technical and behavioral barriers to electronic publishing, for instance, the problems in transmission created by special characters or by complex graphics and halftones. In addition, she noted economic limitations such as the storage costs of maintaining back issues and market or audience education. Manuscripts cannot be uploaded to OJCCT, LEBRON explained, because it is not a bulletin board or E-mail, forms of electronic transmission of information that have created an ambience clouding people's understanding of what the journal is attempting to do. OJCCT, which publishes peer-reviewed medical articles dealing with the subject of clinical trials, includes text, tabular material, and graphics, although at this time it can transmit only line illustrations. Next, LEBRON described how AAAS and OCLC arrived at the subject of clinical trials: It is 1) a highly statistical discipline that 2) does not require halftones but can satisfy the needs of its audience with line illustrations and graphic material, and 3) there is a need for the speedy dissemination of high-quality research results. Clinical trials are research activities that involve the administration of a test treatment to some experimental unit in order to test its usefulness before it is made available to the general population. LEBRON proceeded to give additional information on OJCCT concerning its editor-in-chief, editorial board, editorial content, and the types of articles it publishes (including peer-reviewed research reports and reviews), as well as features shared by other traditional hard-copy journals.
Among the advantages of the electronic format are faster dissemination of information, including raw data, and the absence of space constraints because pages do not exist. (This latter fact creates an interesting situation when it comes to citations.) Nor are there any issues. AAAS's capacity to download materials directly from the journal to a subscriber's printer, hard drive, or floppy disk helps ensure highly accurate transcription. Other features of OJCCT include on-screen alerts that allow linkage of subsequently published documents to the original documents; on-line searching by subject, author, title, etc.; indexing of every single word that appears in an article; viewing access to an article by component (abstract, full text, or graphs); numbered paragraphs to replace page counts; publication in Science every thirty days of indexing of all articles published in the journal; typeset-quality screens; and Hypertext links that enable subscribers to bring up Medline abstracts directly without leaving the journal. After detailing the two primary ways to gain access to the journal, through the OCLC network and Compuserv if one desires graphics or through the Internet if just an ASCII file is desired, LEBRON illustrated the speedy editorial process and the coding of the document using SGML tags after it has been accepted for publication. She also gave an illustrated tour of the journal, its search-and-retrieval capabilities in particular, but also including problems associated with scanning in illustrations, and the importance of on-screen alerts to the medical profession re retractions or corrections, or more frequently, editorials, letters to the editors, or follow-up reports. She closed by inviting the audience to join AAAS on 1 July, when OJCCT was scheduled to go on-line.
Given that context, PERSONIUS described the College Library Access and Storage System (CLASS) Project, a library preservation project, primarily, and what has been accomplished. Directly funded by the Commission on Preservation and Access and by the Xerox Corporation, which has provided a significant amount of hardware, the CLASS Project has been working with a development team at Xerox to develop a software application tailored to library preservation requirements. Within Cornell, participants in the project have been working jointly with both library and information technologies. The focus of the project has been on reformatting and saving books that are in brittle condition. PERSONIUS showed Workshop participants a brittle book, and described how such books were the result of developments in papermaking around the beginning of the Industrial Revolution. The papermaking process was changed so that a significant amount of acid was introduced into the actual paper itself, which deteriorates as it sits on library shelves. One of the advantages for technology and for the CLASS Project is that the information in brittle books is mostly out of copyright and thus offers an opportunity to work with material that requires library preservation, and to create and work on an infrastructure to save the material. Acknowledging the familiarity of those working in preservation with this information, PERSONIUS noted that several things are being done: the primary preservation technology used today is photocopying of brittle material. Saving the intellectual content of the material is the main goal. With microfilm copy, the intellectual content is preserved on the assumption that in the future the image can be reformatted in any other way that then exists.
An underlying assumption of the CLASS Project from the beginning was that it would develop a network application. Project staff scan books at a workstation located in the library, near the brittle material. An image-server filing system is located at a distance from that workstation, and a printer is located in another building. All of the materials digitized and stored on the image-filing system are cataloged in the on-line catalogue. In fact, a record for each of these electronic books is stored in the RLIN database so that a record exists of what is in the digital library throughout standard catalogue procedures. In the future, researchers working from their own workstations in their offices, or their networks, will have access--wherever they might be--through a request server being built into the new digital library. A second assumption is that the preferred means of finding the material will be by looking through a catalogue. PERSONIUS described the scanning process, which uses a prototype scanner being developed by Xerox and which scans a very high resolution image at great speed. Another significant feature, because this is a preservation application, is the placing of the pages that fall apart one for one on the platen.
Ordinarily, a scanner could be used with some sort of a document feeder, but because of this application that is not feasible. Further, because CLASS is a preservation application, after the paper replacement is made there, a very careful quality control check is performed. An original book is compared to the printed copy and verification is made, before proceeding, that all of the image, all of the information, has been captured. Then, a new library book is produced: The printed images are rebound by a commercial binder and a new book is returned to the shelf. Significantly, the books returned to the library shelves are beautiful and useful replacements on acid-free paper that should last a long time, in effect, the equivalent of preservation photocopies. Thus, the project has a library of digital books. In essence, CLASS is scanning and storing books as 600 dot-per-inch bit-mapped images, compressed using Group 4 CCITT (i.e., the French acronym for International Consultative Committee for Telegraph and Telephone) compression. They are stored as TIFF files on an optical filing system that is composed of a database used for searching and locating the books and an optical jukebox that stores 64 twelve-inch platters. A very-high-resolution printed copy of these books at 600 dots per inch is created, using a Xerox DocuTech printer to make the paper replacements on acid-free paper.
Putting the issue of CD-ROM in perspective before getting into generalities on networked information, LYNCH observed that those engaged in multimedia today who wish to ship a product, so to say, probably do not have much choice except to use CD-ROM: networked multimedia on a large scale basically does not yet work because the technology does not exist. For example, anybody who has tried moving images around over the Internet knows that this is an exciting touch-and-go process, a fascinating and fertile area for experimentation, research, and development, but not something that one can become deeply enthusiastic about committing to production systems at this time. This situation will change, LYNCH said. He differentiated CD-ROM from the practices that have been followed up to now in distributing data on CD-ROM. For LYNCH the problem with CD-ROM is not its portability or its slowness but the two-edged sword of having the retrieval application and the user interface inextricably bound up with the data, which is the typical CD-ROM publication model. It is not a case of publishing data but of distributing a typically stand-alone, typically closed system, all--software, user interface, and data--on a little disk. Hence, all the between-disk navigational issues as well as the impossibility in most cases of integrating data on one disk with that on another. Most CD-ROM retrieval software does not network very gracefully at present. However, in the present world of immature standards and lack of understanding of what network information is or what the ground rules are for creating or using it, publishing information on a CD-ROM does add value in a very real sense.
LYNCH drew a contrast between CD-ROM and network pricing and in doing so highlighted something bizarre in information pricing. A large institution such as the University of California has vendors who will offer to sell information on CD-ROM for a price per year in four digits, but for the same data (e.g., an abstracting and indexing database) on magnetic tape, regardless of how many people may use it concurrently, will quote a price in six digits. What is packaged with the CD-ROM in one sense adds value--a complete access system, not just raw, unrefined information--although it is not generally perceived that way. This is because the access software, although it adds value, is viewed by some people, particularly in the university environment where there is a very heavy commitment to networking, as being developed in the wrong direction. Given that context, LYNCH described the examples demonstrated as a set of insular information gems--Perseus, for example, offers nicely linked information, but would be very difficult to integrate with other databases, that is, to link together seamlessly with other source files from other sources. It resembles an island, and in this respect is similar to numerous stand-alone projects that are based on videodiscs, that is, on the single-workstation concept. As scholarship evolves in a network environment, the paramount need will be to link databases. We must link personal databases to public databases, to group databases, in fairly seamless ways--which is extremely difficult in the environments under discussion with copies of databases proliferating all over the place.
The notion of layering also struck LYNCH as lurking in several of the projects demonstrated. Several databases in a sense constitute information archives without a significant amount of navigation built in. Educators, critics, and others will want a layered structure--one that defines or links paths through the layers to allow users to reach specific points. In LYNCH's view, layering will become increasingly necessary, and not just within a single resource but across resources (e.g., tracing mythology and cultural themes across several classics databases as well as a database of Renaissance culture). This ability to organize resources, to build things out of multiple other things on the network or select pieces of it, represented for LYNCH one of the key aspects of network information. Contending that information reuse constituted another significant issue, LYNCH commended to the audience's attention Project NEEDS (i.e., National Engineering Education Delivery System). This project's objective is to produce a database of engineering courseware as well as the components that can be used to develop new courseware. In a number of the existing applications, LYNCH said, the issue of reuse (how much one can take apart and reuse in other applications) was not being well considered. He also raised the issue of active versus passive use, one aspect of which is how much information will be manipulated locally by users. Most people, he argued, may do a little browsing and then will wish to print. LYNCH was uncertain how these resources would be used by the vast majority of users in the network environment.
LYNCH next said a few words about X-Windows as a way of differentiating between network access and networked information. A number of the applications demonstrated at the Workshop could be rewritten to use X across the network, so that one could run them from any X-capable device- -a workstation, an X terminal--and transact with a database across the network. Although this opens up access a little, assuming one has enough network to handle it, it does not provide an interface to develop a program that conveniently integrates information from multiple databases. X is a viewing technology that has limits. In a real sense, it is just a graphical version of remote log-in across the network. X-type applications represent only one step in the progression towards real access. LYNCH next discussed barriers to the distribution of networked multimedia information. The heart of the problem is a lack of standards to provide the ability for computers to talk to each other, retrieve information, and shuffle it around fairly casually. At the moment, little progress is being made on standards for networked information; for example, present standards do not cover images, digital voice, and digital video. A useful tool kit of exchange formats for basic texts is only now being assembled. The synchronization of content streams (i.e., synchronizing a voice track to a video track, establishing temporal relations between different components in a multimedia object) constitutes another issue for networked multimedia that is just beginning to receive attention.
Underlying network protocols also need some work; good, real-time delivery protocols on the Internet do not yet exist. In LYNCH's view, highly important in this context is the notion of networked digital object IDs, the ability of one object on the network to point to another object (or component thereof) on the network. Serious bandwidth issues also exist. LYNCH was uncertain if billion-bit-per-second networks would prove sufficient if numerous people ran video in parallel. LYNCH concluded by offering an issue for database creators to consider, as well as several comments about what might constitute good trial multimedia experiments. In a networked information world the database builder or service builder (publisher) does not exercise the same extensive control over the integrity of the presentation; strange programs "munge" with one's data before the user sees it. Serious thought must be given to what guarantees integrity of presentation. Part of that is related to where one draws the boundaries around a networked information service. This question of presentation integrity in client-server computing has not been stressed enough in the academic world, LYNCH argued, though commercial service providers deal with it regularly.
Concerning multimedia, LYNCH observed that good multimedia at the moment is hideously expensive to produce. He recommended producing multimedia with either very high sale value, or multimedia with a very long life span, or multimedia that will have a very broad usage base and whose costs therefore can be amortized among large numbers of users. In this connection, historical and humanistically oriented material may be a good place to start, because it tends to have a longer life span than much of the scientific material, as well as a wider user base. LYNCH noted, for example, that American Memory fits many of the criteria outlined. He remarked the extensive discussion about bringing the Internet or the National Research and Education Network (NREN) into the K-12 environment as a way of helping the American educational system. LYNCH closed by noting that the kinds of applications demonstrated struck him as excellent justifications of broad-scale networking for K-12, but that at this time no "killer" application exists to mobilize the K-12 community to obtain connectivity. ****** +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ DISCUSSION * Dearth of genuinely interesting applications on the network a slow-changing situation * The issue of the integrity of presentation in a networked environment * Several reasons why CD-ROM software does not network * +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ During the discussion period that followed LYNCH's presentation, several additional points were made.
LYNCH reiterated even more strongly his contention that, historically, once one goes outside high-end science and the group of those who need access to supercomputers, there is a great dearth of genuinely interesting applications on the network. He saw this situation changing slowly, with some of the scientific databases and scholarly discussion groups and electronic journals coming on as well as with the availability of Wide Area Information Servers (WAIS) and some of the databases that are being mounted there. However, many of those things do not seem to have piqued great popular interest. For instance, most high school students of LYNCH's acquaintance would not qualify as devotees of serious molecular biology. Concerning the issue of the integrity of presentation, LYNCH believed that a couple of information providers have laid down the law at least on certain things. For example, his recollection was that the National Library of Medicine feels strongly that one needs to employ the identifier field if he or she is to mount a database commercially. The problem with a real networked environment is that one does not know who is reformatting and reprocessing one's data when one enters a client server mode. It becomes anybody's guess, for example, if the network uses a Z39.50 server, or what clients are doing with one's data. A data provider can say that his contract will only permit clients to have access to his data after he vets them and their presentation and makes certain it suits him. But LYNCH held out little expectation that the network marketplace would evolve in that way, because it required too much prior negotiation.
Ideas discussed in this chapter
Text from Project Gutenberg, public domain in the United States.