Report prepared by Kate Cumming

Lat month, four fantastic thinkers came together at a Recordkeeping Roundtable event to discuss recordkeeping in the contemporary workplace.

The event was inspired by James Lappin’s paper, Rival records management models in an era of partial automation, and James was the event’s first speaker. 

James Lappin

started by saying that we are at the tail end of the era of partial automation. James identified this era as beginning in 1996. Not coincidentally, this was also the year that the world’s first national standard on records management was published. 

James pointed out that this standard required organisations to consistently and routinely capture records into managed, shared environments, and to organise them by business process. However, in 2021, we now realise this isn’t possible, particularly with key communicative records stored in email and other collaboration systems. Corporate practice is almost universally showing that primarily, these records are automatically filed into organisation’s main email messaging system or within another collaborative platform like Teams and this is where they remain. In a far cry from the aim of records management standards, in these environments they are not organised by business process, but are instead siloed in personal accounts in a simple, chronological filing structure. There they are not managed by corporate rules, not accessible as a knowledge resource and not accessible into the future, once an individual leaves an organisation. 

In the years since 1996, records management requirements have not significantly changed. The challenges of email management are acknowledged, but the advice remains for users to capture key emails into records systems. However, James pointed out that we consistently see in practice that recordkeeping processes which ask people to manually re-file records that have been already ‘filed’ in email accounts or Teams environments by machines are just not practical. 

James pointed out that we are living at the only point in history where there is a disconnect between how we want to organise records and how we can actually organise records. Our challenges are that we are dealing with exponentially increasing data volumes. However the technology environments that generate these vast volumes of information have fixed, inflexible and simple rules managing the information they contain. Thus information managers who are seeking to manage business information as a well governed, connected asset, consistent struggle with their endeavours. 

James said that, rather than seeking to manage records in centralised records environments, information managers should instead develop strategies for managing records in place, in their native environments. 

He finished by foreshadowing that we are at the advent of the era of artificial intelligence. Soon our traditional approaches and requirements may be moot, or may be able to be automated and perfected to an extent that has not previously been possible in the digital era. 

Chris Hurley

started by emphasising that humans are recordkeeping mammals. Traditionally people have always made records, not because they were told to, but because they needed them. At the epoch of the bureaucratic era in the 1960s, organisations had developed sophisticated registry and recordkeeping structures, centralised corporate files and business processes that often generated records in triplicate, so all relevant corporate files could be updated information made available to all business areas that needed it. Recordkeepers in this era were, as Chris put it, tradesmen, not theoreticians. What changed, at the onset of the digital era in the mid 1990s, was the disappearance of middle management who enforced centralised processes and recordkeeping rules, and the growth of technology operated by the individual. 

Recordkeeping theory emerged at this point because, Chris argued, recordkeepers had had to go back to first principles and define and develop approaches that could scale to the exponential volumes of information that technology in the workplace enabled. Old paper-based processes were inappropriate and a new era of recordkeeping theory was born. 

Much of significance was achieved through this process but in practice, more is required to develop approaches that enable recordkeeping approaches to scale to the complexity and extent required by contemporary organisations. 

However, Chris that where it is needed, recordkeeping will always find a way. Even through the complex digital era where there are gaps between theory, best practice and what’s possible, where business requires good records, business has developed ways for good records to be created and kept. In Chris’s experience, this is seldom within corporate records systems. And it is seldom driven by records managers. Instead Chris noted that specific business areas develop custom solutions, or key systems, like customer relationship systems, emerge to document and manage specific actions, commitments and risks. 

Chris also stated that in his experience, it wasn’t usually records managers who were adapting recordkeeping approaches in dynamic business environments. Generally it was risk managers, data managers, data quality staff who were approaching the core issue of needing to access accurate, authentic and reliable records in different ways. 

Chris finished by stating that conversations with these colleagues, who are equally passionate about good records and information management, can be a significant opportunity to change recordkeeping perspectives, methods and outcomes. 

Andrew Waugh

from the Public Record Office of Victoria was the next speaker. Andrew spoke about a research initiative that PROV has been coordinating, to undertake practical experimentation around options for managing email records as archives. 

This project was triggered by conversations PROV had with investigation agencies. These organisations pointed out that while the records in records systems were useful, the real evidence needed by investigators was always inevitably in the corporate email system. Formal recordkeeping systems in the digital environment present a selective history, and this is why they are not terribly helpful to investigators.

Andrew pointed out that a business system is always recordkeeping, it is not selective. Andrew said that today, we don’t have any option other than to work with work with business systems as recordkeeping systems. His advice is to let go of functionality that isn’t there and exploit the functionality that is there. 

PROV has been experimenting with its own internal email environment. With staff consent, it has taken a sample of 250,000 emails and is experimenting with options for managing these as records. They de-duped the email set which instantly reduced it to 143,000 emails. Their next step was to create email threads, using custom PROV developed software, to create chains of connected emails. This was the most impactful approach and reduced the content to 80,000 meaningful chains of correspondence, shrinking the dataset to 32% of its original size. Threading also created hugely meaningful content, that was much more useful as a record of decision making and activity. 

Threading may be criticised as eliminating original order, but Andrew emphasised that in a digital environment, it is challenging to define what original order is. However, a core benefit of working with digital records in their native environments is the native metadata that can be used to allow multiple representations of order to be presented.

Andrew commented that he does not agree with the Capstone approach to email management, because it only prioritises the email of individuals. He believes that threading creates much better and more comprehensive records, and prioritises multiple business contexts rather than a specific personal context. 

Andrew emphasised that the PROV research is showing that the best records are in the systems in which the work is done. Andrew recommended to focus on the outcomes achieved by paper systems, and not on the methods used in paper systems. He argued that moving records out of records systems degrades the fonds. 

A core conclusion was that the strength of using digital business systems as recordkeeping systems is the metadata – exploit this. He recommended using business systems as recordkeeping systems and aiming for satisfactory recordkeeping practice according to the relevant business risk. Don’t aim for perfection.

Nicola Forbes then emphasised that we are about to enter a post digital era. As recordkeepers, she says we need to consider what we want to take from the digital era into the post-digital era. Change is happening rapidly and so we need to decide fast. 

Nicola also said that increasingly, recordkeeping is being defined by software’s strategies and controls, and posed the question, is this OK? She also contended that the post digital is also about a lack of transparency – increasingly, and particularly with AI, there is no transparency about how a system is built or the rules it is applying. She identified this as a significant recordkeeping, accountability and transparency challenge. 

Highlights from the chat

Attendees kept up a lively commentary as the event progressed.

• Unfortunately the reality of the modern workplace (sic) trumps theories and standards.
  
• Yes, agree: the work and the record being the same thing is when recordkeeping is most effective, for lots of purposes.
  
• Real-world recordkeeping is multi-approach, multi-environment and multi-tech.
  
• Offering different orders is now possible and should be the next step for archives.
  
• With AI we don’t need structural ordering, we can virtually aggregate records by function, topic, user, business unit, or any other aspect of their content or metadata (and sentence them as an aggregation).
  
• There is a difference between RM in public bodies (government and agencies) and business corporations. There are different needs in terms and legal requirements of RM and archiving.
  
• By getting the AI to do the work of reading and classifying every item, records managers actually get more agency and control of enterprise records, because they can (for the first time) have full oversight of every record in the enterprise, and aren’t limited by what the users or source systems can do.
  
• From a public sector point of view. Machinery of Govt changes which happen regularly: how do we transfer content and custodianship of that content in these arrangements?
  
• AI ain’t a silver bullet. And when AI fails, who gets fined and jailed? Who ensures that AI is unbiased, well-documented and continuously in sync with current legal and regulatory requirements and business needs?
  
• We don’t use ML, AI. We use rules-as-code, which is 100% transparent and traceable. The AI can’t ever be allowed to make discretionary decisions, it can only support decision making by qualified records managers.
  
• AI changes records manager jobs from ‘miners’, spending all their time digging underground looking for valuable records, to jewellers, crafting those valuable gems into more valuable insights for the business.
  
• Machines can read every word in every item, make them all discoverable, and match them to rules automatically. That frees up records managers to do what people do best – make evidence-based decisions.

• Yes, and create information noise. Which led to the impossibility to take proper decisions.

• Basically the closer you get to sizeable and short-term financial impact, the more rigorous the record keeping requirements will be. ‘Normal’ public administration – regardless of whether it’s university students, care homes, etc, the Records/Info Managers will be told to ‘lump it’.

• We need to use ontologies and rules as code to find the content we are interested in. It’s true that some organisations would rather not find the full story, because it’s harder then to decide what to do. But most want to be able to see the full timeline of a record, which is what AI can do for us.

• AI can become a great failure because they cannot be trained according to organisation specifics.  For two decades RM was promised the idea of the tool that solves all problems with the next application. It never happened.

Some questions to consider

Some of the questions we were left with included..

• We are moving from the era of the records system to the era of metadata. The power of metadata, not the power of records systems, will become the key focus of recordkeeping. What does this mean for practice? What are our next steps now?
  
• With everything now as a service, how do we consider this in recordkeeping strategies?

• If we are moving the focus of our recordkeeping to recordkeeping in place strategies, what practical tools and guidance do practitioners need? How do we adapt our tools and approaches? Where do we focus? Who are our new partners in recordkeeping? How significantly do we need to change our traditional focus?
  
• Everyone assumes all data is kept and available for multiple purposes. Are we cool with that?
  
• With the rapid rise of AI and ML, suddenly recordkeeping won’t be reliant on people creating and filing records, machines can now do the work. Will we suddenly have perfect records? What are the risks and issues that need to be considered?
  
• Massive datasets, complexity, change over time – how is all of this managed?

Audio and transcript