Dealing With Commonwealth Science and Technology Records

Part of our mandate, of course, concerns the records of Commonwealth science and technology activities. In Australia, the majority of science and technology research is sponsored by the Commonwealth government, a proportion of which is directly undertaken by Commonwealth agencies. These agencies include fairly high-profile organisations such as the CSIRO and the Defence Science and Technology Organisation, but also a myriad less well known agencies such as the National Resource Information Centre, part of the Department of Primary Industries and Energy and the Alligator Rivers Region Research Institute, which is in the Department of the Environment, Sport and Territories.

All of these agencies produce Commonwealth records and, whether they are aware of it or not, are subject to the provisions of the Archives Act. Under the terms of the Act, all Commonwealth agencies have a responsibility towards their records. That responsibility includes not disposing of any records without our permission; assisting the Archives to decide on the disposal status of their records (i.e. how long they should be kept); and, following that decision, destroying valueless records as soon as possible and transferring those with long term temporary or permanent value to the Archives when they are no longer needed.

It follows then that the Australian Archives has a substantial role to play in 'recovering' science in this country. We are uniquely placed in the Australian context because we are responsible for the records not only of research activities, but also for the records of the policy decisions which create the impetus for much of that research. If, in the future, a researcher wants to examine the development of the Australian Government's policy on monitoring the Greenhouse Effect and the implementation of that policy, or the success or otherwise of the 'clever country', then the Australian Archives will be the place to come for the records.

That is the theory; that is what we want to achieve. In practice, of course, the situation is not nearly so clearly defined. At the Archives we are faced with between 2500 and 3000 current Commonwealth agencies, not to mention the thousands of now defunct or superseded agencies, whose records we already hold. It is inevitable that a number of agencies, for one reason or another, have slipped through our net. Unfortunately, some agencies with science and technology functions fall into this category. Before we can guarantee that the researcher bent on knowing everything about the Commonwealth Government and the Greenhouse Effect or the clever country will be satisfied with our holdings, a number of things must happen and happen quickly.

Recognising this fact was the first step, and I'm pleased to be able to say that the Archives has moved substantially beyond that point. I would like to discuss the progress we have made in dealing with the Commonwealth's science and technology records.

What sets science and technology records apart?

1. The number and diversity of science and technology agencies and the apparent complexity of the records they create appears more daunting to a non-specialist than other types of Commonwealth records.

2. For one reason or another, science and technology agencies' staff are less likely to be aware that they are creating Commonwealth records which are the proper concern of the Australian Archives.

I will examine both of these issues briefly for it seems that the crux of any perceived or real problems we have with science and technology records lies within.

The diversity of science and technology agencies and their records

At the same time, there is no doubt that the number of records produced by the Commonwealth has been increasing exponentially. This is partly a direct result of the increase in the number of functions that fall under the Commonwealth umbrella, but it is also a reflection of changes in technology that have made the duplication of records and methods for collecting information quick and easy. The photocopy machine has a lot to answer for - world-wide an estimated 5000 million copies are being produced today.² Where records are concerned, quantity breeds contempt - the more there are, the more difficult it is to find the records that do have long term value, and conversely, the easier it is to destroy them by mistake in the attempt to 'rationalise' a records storage problem (whether that problem is shelves groaning with paper, or an overloaded computer memory).

We are dealing with growing quantities of records, but the formats in which science and technology records are created and kept have always been diverse and will continue to change and challenge the Australian Archives as well. We can and will cope with a surprising variety of record types. Notebooks, logbooks, photos, video and audio recordings, maps, charts, models, paintings, magnetic tapes and disks have all made their way to our repositories. Apart from having some special requirements for storage and access, including more closely controlled environmental conditions, they cause the Archives few problems - except for the last, the electronic records. These have proved difficult to deal with using traditional archival practices and it is worth discussing the issues involved, if only briefly.

Science and technology researchers have been particularly innovative in developing an array of computer applications to facilitate data collection and manipulation. Therein lies a Pandora's Box of archival concerns. Electronic records have changed the definition of record from something that is obvious and immutable, to something hard to grasp and categorise neatly. The laundry list of archival woes could continue ad infinitum; however, I will make just three points.

Firstly, technology has affected the way records, and the information they contain, are created, used and stored. Widespread use of computers has meant that while it is certainly possible to store much more information in a small physical space, it is also possible to destroy or lose access to that information with awesome ease.

My second point is that technology has opened up avenues for data collection and manipulation that were never a possibility when paper was the only option. But the danger lies in letting the collection of information be technology driven - collecting information because we can, rather than because we have the increased capability to make use of it. Think of readings that were once taken manually, once a day, that can now be recorded every second. Think of the terabytes of remotely sensed data downloaded each year from orbiting satellites.³ Considerable resources have had to be employed writing data compression programs to make the data useable!

Finally, improved and expanded telecommunications mean that communication between people is easier, but the record of that communication is ephemeral. Letters are replaced by a telephone call or electronic mail dialogue, or information is exchanged through electronic bulletin boards - records that rarely make it into durable formats.

From the Archives' perspective, these developments impose extra risks on records with long term value for research. Even in agencies where we have tame records managers willing to undertake archival work, electronic records have tended to be viewed as being outside the agencies' record keeping system. They have been described with imagination as 'islands of unmanaged information'⁴ - perhaps 'island continents' is a more realistic analogy! We therefore have to rely more heavily on agency staff to help identify the records with value. There are also problems of obsolescence, where information becomes inaccessible because of technological change, deletions are possible at the accidental or deliberate touch of a button, and control of information lies much more firmly in the hands of users. When we do determine that particular electronic records should be kept, there are additional concerns about how and where the records should be retained.

Science and technology agencies and the Australian Archives

In general, we can live with this. During the time since the Archives Act was proclaimed, we have concentrated our efforts towards firstly dealing with agencies that have substantial quantities of largely temporary records and thus are likely to have storage problems that can be resolved, because most of the records can be destroyed sooner rather than later. These we euphemistically called 'the big easies'. Secondly, we have directed our efforts towards dealing with agencies that are likely to be creating the most valuable records. In theory, this leaves the agencies with more manageable quantities of records or with less valuable records as the residue.

Science and technology agencies tend to be one of the groups left behind. While they might meet the criterion of not holding overwhelming quantities of records, in terms of the importance of their activities in the Commonwealth scheme of things, they should have been high on our list of agencies creating valuable records. With a few notable exceptions, such as the CSIRO, and the Anglo Australian Telescope, this was not so, and is an issue that needs addressing.

The reasons lie partly in the diversity of agencies and the specialisation of their records that I discussed earlier. Why go searching for trouble that you know exists, when there are easier tasks at hand? A corollary to the post-war diversity of science and technology activities is their wide geographic distribution. Why go so far from home to find trouble? There aren't many inducements unless the agency in question happens to be on an island in the Whitsunday Passage.

The second part of the explanation lies in the attitudes of researchers to their records and the record-keeping practices at the agencies. Independence in research does not sit comfortably with conforming to requirements for the proper management of the records produced, when those requirements are set by a non-scientific body such as the Australian Archives. The records created in these agencies are far more likely to be maintained according to personal preference and to be regarded as personal rather than Commonwealth property, than are records anywhere else in the Commonwealth.

Personal records do have a habit of ending up mixed with official records, and that can add to the confusion. It leads to a rather ad hoc approach to their retention and destruction, and it is as likely as not, that all of the records will go with an individual researcher if he or she moves on.

Solving the problems

Selecting the valuable records

We make no apologies for being ruthless in selecting records for retention. The Commonwealth, and its tax payers, cannot afford the luxuries of the past - we cannot afford to keep more than is absolutely necessary. The advantages of destroying the dross are not only economic ones. Once the valueless records are destroyed, the resources we do have can be much more usefully put to work caring for and providing access to valuable records. There is a substantial benefit for researchers.

However, no one who has appraised records to determine which should be kept and which should be destroyed will tell you that it is an easy task, particularly for modern records. There are always some obviously valuable and equally obviously valueless records, but by far the greatest proportion lies in the grey area between. Each of those records might have its five minutes of research glory, but this must be balanced against the costs of keeping each of them indefinitely.

There is an abundance of archival theory that is a sound basis for appraising records, but on its own it does not make it easy to deal with the sheer quantity and variety of modern government records, particularly science and technology records. As I mentioned earlier, the agencies range from large organisations, to single laboratories in remote locations. The records come in every conceivable format, using every conceivable (and inconceivable) code to make interpretation difficult. The information contained in the records can be user friendly and within the realm of understanding of a simple person, or can be so arcane as to defy understanding even by other scientists in the same discipline, let alone the average appraisal officer from the Australian Archives.

We felt that the appraisal solution lay in finding any common threads that exist within this disparate group of organisations and their records, that could be used to provide non-scientific people with techniques to make the right decisions about the records. As background, we looked at any dealings we had had with science and technology agencies and our knowledge of their activities and records, and we looked at the current literature, particularly that from the United States.⁶ This review resulted in some fairly general observations about the agencies and their records which formed a pattern that could be used as a basis for an appraisal model. This was formalised into a set of guidelines that our operational staff can use on the records of science and technology agencies.

The model that provides the framework for the guidelines divides science and technology functions into four major activities that are likely to be undertaken in one form or another in each Commonwealth agency carrying out science or technology based research. These are:

1. Establishing the research program or project
2. Designing and implementing the research methods
3. Disseminating results
4. Administering the program or project

By extension we can look at the sorts of records generated by these activities and which of them are likely to have long term value for research.

As a government archives we are interested in keeping those records that demonstrate the development of government policy at the highest levels and the implementation of those policies in the significant programs. These are the records that show evidence of how government functions. Within our guidelines, this is the first stage of the model encompassing the establishment of the research program. As a general rule, our appraisal staff do not have substantial problems identifying the records that will be most valuable for these purposes because they do not differ dramatically from the policy records of any other Commonwealth function. A researcher interested in looking at the development of government science and technology policy should be well served by the records we hold or will hold in the future.

Similarly, the records that document the administration of particular programs or projects are likely to present little by way of special appraisal problems. Indeed, many of them already have disposal coverage in the form of our General Disposal Authorities, which we have prepared to cover the records common to most Commonwealth agencies.

As a government archives, we are also interested in keeping those records that show the implementation of government policy through its significant programs. Within our guidelines for science and technology records, these activities are the second two phases of the model. Our hypothetical researcher may be interested in continuing a particular piece of scientific research and would want access to the core research data as a means of building upon or revising the scientific knowledge base. These records are the ones most likely to cause non-specialist appraisal staff headaches. They come in the widest range of formats and lend themselves least to common understanding. The data itself is difficult, but there is also confusion over what to keep because of the overlap between published and unpublished material. The advice contained in the guidelines is mainly directed towards providing a means of dealing with these core research records. The most important piece of advice is for the Archives to involve agency staff wherever possible.

The guidelines include a broad classification of science and technology records that identifies likely retention periods for the most commonly occurring types of information. These can be adapted for the records of specific agencies to form disposal classes that can be authorised by the Archives.

We have also developed a number of complementary guidelines that give general definitions of the appraisal criteria of the Archives, and others that document procedures for dealing with older records and records in a variety of non-paper formats, such as sound and moving image records and photographic records.

Considerable resources have been devoted to developing guidelines for appraising electronic records. They have special significance for the records of science and technology agencies, simply because of the volume of research information kept in electronic format and its likely long-term value. An indication of the continuing value of these records is demonstrated by a recent survey of databases in the USA which suggested that out of 300 scientific databases, 130 (or about 40%) of them should be kept. This compares with keeping an average of only 9% when databases from a wider range of subject areas are considered.⁷

Our guidelines are primarily about ensuring that electronic records with these sorts of continuing values are identified as soon as possible, before data is lost or software and hardware become obsolete. The best approach is for these decisions to be made during system development. That way arrangements can be made for the valuable records at the outset. They can then be backed up in a systematic fashion and migrated with system upgrades.

These guidelines are currently in the Disposal Manual which is for internal distribution to our operational staff. They have already proved valuable in appraising the records of science and technology agencies, including for example, the Australian National Parks and Wildlife Service. Of course, their value is limited to assisting appraisal of the records of agencies that come to us for advice, or agencies that we target for disposal work. These have tended to be the larger and more centralised agencies. Small, specialised agencies are now high on our list of priorities and much of the work we undertake in the next few years will be directed towards developing techniques to provide them with appropriate archival advice and services.

The next step is to work out exactly the dimensions of this task; how many agencies are we talking about, what sorts of functions are they responsible for, how receptive will they be to our concerns about their records. By developing profiles of information on the agencies we will be in a much better position to give them appropriate and timely advice. We already produce a range of guides for agency staff to use in dealing with records. These include appraisal guidelines, advice on planning disposal programs in agencies, information on transferring records and storing records with the Archives. In the future we hope to develop the science and technology appraisal guidelines further and make them available to staff in the agencies, and then target those agencies for archival work. By getting agencies involved we can tap into a rich source of subject expertise that we can never hope to duplicate ourselves.

Making the records we hold available for research

The records in our custody are controlled by the Australian Archives Records Information Service database, more commonly known as RINSE. This database is available on-line to the public in each of our Regional Offices. There are a number of different ways to interrogate the system to find information on agencies, functions, and record series. RINSE also supplies the links between agencies and functions and their records, which allows researchers to make the connections that exist between policy matters and researchers throughout the network of committees, councils, funding channels and the budget process.

This is supplemented by the Australian National Guide to Archival Material I - ANGAM I - which is available on microfiche and describes agencies of the Commonwealth Government that have existed since Federation and the records those agencies created and controlled. ANGAM II is a database listing all records that have had access decisions made about them. This database is also available on-line to the public and can be searched using keywords.

Despite the comprehensive nature of our control over the holdings, we are still concerned that the listings of some records are inadequate to obtain access, unless the searcher knows very specifically what he or she is looking for. There is scope for developing other sorts of finding aids, including subject guides. We have issued a guide to family history sources for genealogists and a guide to lighthouses in Australia. Work is continuing on other subject guides, and in 1993 we will be undertaking a joint pilot project with the Australian Science Archives Project to test the feasibility of producing a guide to the Australian Archives' holdings of science and technology records. I suspect that this will be the most ambitious subject guide yet attempted, and perhaps the most rewarding.

The project to reduce our unevaluated holdings will also contribute to improving our knowledge of what is actually held. There are many treasures waiting to see the light of day but, equally, much rubbish that should be destroyed to improve the chances of survival of those records with lasting value.

Information is also held in RINSE about personal records collections held by the Archives. These include the records of some individual scientists who have been closely involved with the Commonwealth during at least part of their careers. Such collections provide the Archives with the opportunity to retrieve many Commonwealth records that have ended up mixed with the personal records of individuals, while maintaining the integrity of those record collections. There is scope for us to make sure researchers employed by the Commonwealth on significant projects are aware of this service, although it becomes more difficult to identify the most significant players as research projects have become more and more team or cooperative efforts between the Commonwealth and other organisations.

This leads me on to my final point, that the current science and technology research environment within the Commonwealth which encourages joint ventures with private organisations or other governments will have a significant impact on aspects of the Archives' work in the future. Who will own the records? We have been dealing with this problem in another guise over the past few years, as the Government has steadily privatised and corporatised a number of its functions. We have requested a string of legal opinions from the Attorney-General's Department to determine who owns the records after privatisation has occurred. If records are somehow removed from Commonwealth ownership, then the Archives has but a slim chance of ensuring that a complete record of significant Commonwealth activities is retained.

Similar confusion over records' ownership will arise from the current crop of Cooperative Research Centres and joint ventures unless specific steps are taken to determine who has responsibility for the records and where they are to be retained. As with electronic records, this should be done at the earliest opportunity so that valuable records are not lost before they are identified and arrangements made for their proper retention or disposal.

At the Archives we understand that the ways in which science and technology are carried out, and the ways in which research is communicated and recorded are changing. Only a dynamic and flexible approach that includes the cooperation of researchers themselves will ensure that the records of science are 'recovered', that they remain in the public arena, and are accessible in the future.

Notes:

¹ R.W. Home, 'Science on Service, 1939-1945', Australian Science in the Making, ed. R.W. Home, Melbourne, 1988, pp. 220-251.

² Denis Muller in the Age, 29 October, 1992, p. 10.

³ David Bearman, 'Electronic Records of Science', Archives and Museum Informatics, vol. 6, no. 1, 1992, pp. 14-15.

⁴ Dagmar Parer, 'Towards Better Management of Commonwealth Records', Informaa Quarterly, vol. 8, no. 3, 1992, pp. 9-10.

⁵ Australian Archives Annual Report 1991-92, Canberra, 1992, p.36.

⁶ Particularly Joan K. Haas, Helen Willa Samuels and Barbara Trippel Simmons, Appraising the Records of Modern Science and Technology: a Guide, Massachusetts, 1985; and Clark A. Elliott ed.,Understanding Progress as Process: Documentation of the History of Post-War Science and Technology in the United States: Final Report of the Joint Committee on Archives of Science and Technology, Chicago, 1983.

⁷ National Academy of Public Administration, The Archives of the Future: Archival Strategies for the Treatment of Electronic Databases: a Study of Major Agencies in the United States Government, Washington, 1991.

Published by ASAP on ASAPWeb, 2 August 1997.
Designed by Tim Sherratt

Updated by: Elissa Tenkate
Date modified: 25 February 1998