Andrew Waugh

A bellwether is a sheep with a bell around its neck. Since sheep flock together, the shepherd could track the movement of the flock by the sound of the bell on the bellwether. In a similar way, I argue in this note that email is a bellwether records system; the problems that archivists and records managers have in ensuring that email records are captured and managed are indicative of the problems we have (and will have) in managing records from other computer/internet based systems. These problems have many causes, but one root cause is that we are still trying to manage records as if they were mid twentieth century paper files. We need to move to models of record management that address the strengths of computer/internet records systems. In this time of transition we need to accept that the resulting records produced now will not be perfect, but systems will get better if we adapt and change our advice.

The problem with email

From an archival/records perspective email systems are terrible records systems.

There are three fundamental problems that make email systems poor record systems. First, some email systems allow users to edit stored copies of emails that have been sent or received. The records consequently cannot be shown to have integrity. Second, management of disposal is non-existent. Users can delete any email that they send or receive at any time. IT departments can, and do, impose disposal decisions by fiat based on the age of the email (or the quantity a user accumulates), and these time periods are usually absurdly short and certainly not based on any analysis of the functions the records support. The final issue with disposal is that users do not dispose of ephemeral emails, or emails with short retention periods, leading to a clutter of irrelevant records. The final problem with email systems is access to records. The email records themselves are scattered amongst the inboxes of the various users and cannot be easily accessed as a whole by an organisation. Even within an individual user space, records are not organised in any consistent way and often related records cannot be found easily. Looking outside the email system itself, the email records are divided from related records held in other records systems and so it is difficult to retrieve the full story.

Typically these problems are managed by a policy of requiring users to file emails into a formal record system such as an EDRMS. [1] Brogan [2] describes research identifying the limitations of this approach – a significant number of users do not file records. Given the cost and lack of compliance, NARA has recently promulgated a Bulletin[3] that offers agencies the option of permanently retaining the complete email accounts of senior staff, while retaining the email accounts of other staff for a fixed period. No classification or filing would ordinarily be done in this approach.

A contrarian viewpoint

From an archival/records perspective, email systems are fantastic records systems.

Evidence of the value of email systems as record systems can be found in any modern governance or accountability investigation, such as a royal commission, audit report, ombudsman’s investigation, or even in investigative journalism. Without exception, emails form the key planks of a modern investigation and feature prominently in the final report. In 2012, the then Victorian Auditor General and Victoria’s Deputy Ombudsman, independently, told a seminar of Victorian records professionals that, while their investigators looked at the records, the smoking gun was always in the email. A recent example of such an investigation is the Debelle Royal Commission[4] in South Australia which resulted in the setting up of an independent review of the South Australian State Records Act[5].

Governance and accountability, of course, are only one of the societal reasons for keeping records. However, the quality of the record trail that is being exploited by investigators is of value for other uses of records, including the taking and defending legal actions, and historical research. Why are emails of such value, and what does this tell us about current records and modern information management?

Unselfconscious creation

The answer to the value of email systems is an old archival principle: unselfconscious creation. Users of email systems are simply doing their work. The work is carried out by exchanging information with the other work participants; the information being exchanged by the email system. The email system captures copies of the emails sent and received, and consequently a trail of records is captured of the work as it is being carried out.

The power of unselfconscious creation is highlighted by considering an EDRMS as is it used in many organisations today. Often, the work in these organisations is carried out in some other system – for example, email or a business system – and records are manually captured into the EDRMS. This is, of course, self-conscious creation of the record. At best, where users routinely file records this can approximate unselfconscious creation. At worst, users carefully select the records that are filed, and the result is very self-conscious creation of records.

Collaborative Environments

Email systems are an example of a collaborative environment. Collaborative environments are powerful record systems. People do work by collaborating with others. The act of collaborating is identical with Bearman’s concept of transactions. By capturing the transactions that make up the collaboration a strong record of the work is created.

This is not new. Traditional paper based files were a collaborative environment. The file was the medium of work: the topmost record detailed the current task, and the records underneath contained the background and context of the task. The file was moved from actor to actor to carry out the work. But as computers were introduced into workplaces, the doing of the work was often separated from the keeping of the records. Today, the work is normally carried out in a collaborative environment on a computer, and the ‘records’, if they are kept at all, are kept in a separate system, such as an EDRMS.

Current computer systems abound in collaborative environments. These include business applications such as case management systems, registration systems, intelligence systems used by police, and even EDRMS. In the broader society, collaborative systems include social media such as Facebook. Email is a bellwether collaborative environment for two reasons: it has been in use for a very long time and is extremely widespread – in particular its ubiquity facilitates ad hoc communications between organisations.

Computer systems within organisations are currently being transformed by the ubiquity of the internet: Collaborative systems are no longer being hosted within the organisation. Either the systems are being hosted by third parties over the internet (e.g. Gmail instead of an organisation supplied email), or internet based collaborative systems are being co-opted for use within organisations. In either case, the result is another step change in the challenge of recordkeeping. Future collaborative environments will be developed and hosted outside organisations that we as recordkeepers and archivists can influence.

Back to email

In this short note I am not arguing that email is the perfect record system, or that EDRMS cannot be a powerful records system. What I am arguing is that the importance of the collaborative environment in which the work is actually done has been lost sight of, and it seems to be acceptable for the record system to be an independent, loosely coupled, system. In my view this is not a viable approach going forward. Collaborative environments are simply too ubiquitous, and the future will see an even greater range of such systems and they will be largely independent of the organisations we belong to, and too massive to influence. Instead, I suggest that archivists and records managers need to change our focus on ‘recordkeeping’ record systems. We need to accept that collaborative environments, for all their faults, are the record systems that we will have. This will mean reimaging how archival and recordkeeping goals will be achieved, and perhaps some of the current approaches – such as classification and disposal – will need to change. Until new models of recordkeeping are developed, this may mean that records from current collaborative systems will be less than perfect. This is unfortunate, but it is nothing new. We have plenty of recordkeeping systems in custody from the early twentieth century that were not perfect either. We are living in a time of transition and will have to accept imperfection. By ignoring this change we are prolonging the transition for us as archivists and recordkeepers – but not prolonging the change for our organisations as this is happening anyway. As we learn to play to the strengths of these collaborative systems the records will improve. For example, classification may be replaced by a combination of data mining and data visualisation. But moving closer to the collaborative environments will have positive aspects as well – as email shows it could be a return to unselfconscious creation.

About the author

Andrew Waugh is the Senior Manager, Standards and Policy at the Public Records Office Victoria. He started his professional life, however, as a computer scientist at CSIRO and got sidetracked with VERS.

Endnotes

[1] See, for example, National Archives of Australia, Managing Email (undated), http://www.naa.gov.au/records-management/agency/digital/managingemail/index.aspx, last accessed 4 April 2014. Many other State archives in Australia and internationally have similar policies.

[2] Mark Brogan, Clipping Mercury’s Wings: The Challenge of Email Archiving, Archives and Manuscripts, Vol 37 No 1, May 2009, pp13-26

[3] Guidance on a New Approach to Managing Email Records, NARA Bulletin 2013-02, 29 August 2013, http://www.archives.gov/records-mgmt/bulletins/2013/2013-02.html, last accessed 4 April 2014.

[4] Royal Commission 2012-2013 Report of Independent Education Inquiry, http://www.decd.sa.gov.au/educationinquiry/files/links/DebelleInquiry.pdf, last accessed 4 April 2014.

[5] ‘Reviewing the State Records Act: Have Your Say on our State Records Act’, (undated) http://saplan.org.au/yoursay/reviewing-the-state-records-act, last accessed 4 April 2014.