Researchers at the University of NSW are preparing to launch a public beta of a new open source system that could drive a digital revolution in the field of archaeology.
the Federated Archaeological Information Management System (FAIMS) Project, led by UNSW's Dr Shawn Ross, a senior lecturer at UNSW's UNSW's School of Humanities, received funding from the federal government's the National eResearch Collaboration Tools and Resources (NeCTAR) program.
The aim was to develop a new generation of archaeological tools that could work with modern Android-based mobile devices and promote the production of compatible datasets from different archaeological projects.
Developers at UNSW, the Intersect research consortium in NSW and VeRSI in Victoria have worked on a prototype Android app that can be used to gather data in the field that is then collated in a server-based database system.
Dead database walking: MySQL's creator on why the future belongs to MariaDB
From zero to hero: Suncorp's Drupal 'leap of faith'
Port Macquarie school deploys Chromebooks
A workshop held at UNSW in August last year had made clear that a key barrier for the project was the different workflows and terminology employed by archaeologists.
"If you go out and you run an archaeological project and use an Access database, when the time comes to put that in to a repository, somebody — either you or the repository manager — has to spend a lot of time doing manual ontology mapping," Ross explained.
Differences in terminology can extend to even the most basic parts of an archaeological project, for example the volumes of earth excavated at a site. "Some projects call them a context, some call them a locus, some call them a spit, some call them a stratographic unit — there's about five different terms for even this most fundamental thing," Ross said.
"The way things are now, for every single column in your database that you've got, you'd have to map it to the internal ontology of whatever repository you're putting [the data] into and that's really time consuming; it's the biggest expense and time cost for repositories."
To overcome this, developers spent time making sure the database used by the app can be customised to suit the terminology and workflow used by different archaeologists.
"We produced a fairly a well-structured but generic underlying database," Ross said.
The app uses a domain/key normal form database. "You can customise it [for a] project just by changing the data in database rather than the structure of the database. We looked at a number of NoSQL solutions for this, and they're just not mature enough for the Android environment, especially since we had to have them working offline and working with GIS [geographic information system]."
The team went with a DKNF database using SQLite with SpatiaLite extensions. To customise the app for individual projects or teams, XML documents can be fed in that govern the database schema, the user interface and the logic for the interface.
"We're really happy in the end with that solution — having this generalised database that you can customise with a packet of XML documents and we've already got two students who are working as QA people over at Intersect who have learned to write the XML documents," Ross said.
The design "fits with modern design principles that not everyone needs to roll their own," Ross said.
"You can have an underlying data store that you can use [but] each project can define their schema and interface [and] their workflow however they want, but at the same time to try to produce compatible datasets because we're trying to encourage research at not just one site but across sites, across projects."
The developers also employed techniques based on international standards for localisation, using string replacement so that the terms used in the app's interface can reflect the terms used by a particular archaeological team.
Ross said that the app reflects a balance between offering the flexibility desired by archaeologists while still promoting the production of compatible datasets.
The app allows the recording of text, location, imagery and audio data on Android devices, and a server can be deployed at archaeological sites that, when devices come within range, will synchronise the data on them as well as back it up on the server in an append-only data store (to offer versioning of data, if there needs to be a rollback, for example).
"If you're working in the city and let's say you're processing pottery where everyone is more or less in the same place at the same time, everybody [can work] on their mobile devices and it will automatically synchronise them in more or less real time," Ross said.
"But let's say you're doing prospection. You're going out to the middle of nowhere in Western Australia and you're sending teams out and they're going through the countryside looking for any kind of archaeological remains.
"You can send these teams out with their separate mobile devices, they can work totally offline, totally apart from each other, and then at the end of the day when everybody comes back to the base or the camp [the devices] will find the server, hook up to the server, synchronise with one another and a copy of everything goes on the server."
Page Break
As well as being able to attach data captured by mobile devices themselves to records, the system will allow data captured by other devices — images captured by high-resolution SLR cameras or drawings done by hand, for example — to be linked to records.
The team intends to start conducting field testing with third parties, such as archaeological consultancies, by late April or early May.
"At first we'll run [the system] side-by-side with their existing systems so it will give us a chance to customise it for them make sure it's working without having any high risks of data loss," Ross said.
"We'll also be doing a study of what the time comparisons are like; what kind of efficiency gains you can expect to get from using this system compared to existing workflows. We're pretty much planning that by mid-year we're going to be ready to really start working intensively with a larger group of users to get the system deployed in the field."
"We'll work with five to 10 projects to customise and deploy it," Ross said. "We'll work with the consultants or archaeologists to produce the XML documents, go out with them into the field and work very closely with them on customisation and implementation. If they hit a problem or a bug or there's a feature that's not there or doesn't work right, then we'll be able to go back to our developers and turn that round really quickly."
Ross said the team was keen to receive another year of funding support to continue working with clients, as well as add more features, such as the ability to run the whole system online without a local server if the users are in a networked environment.
"Mostly we'd like to continue this intensive work with clients over the next year, and I think if we can do that, with the kind of uptake it looks like we're likely to get, we should be able to have a sustainable system that will have enough revenue [to sustain itself]."
The software is open source but an organisation, most likely the Intersect consortium in the short to medium term, would offer support, such as aiding with customisation.
Rohan Pearce is the editor of Techworld Australia and Computerworld Australia. Contact him at rohan_pearce at idg.com.au.
Follow Rohan on Twitter: @rohan_p