In our previous post we introduced the concept of hybrid artefacts that connect physical objects to their digital records. Let’s unpack this a bit further. Way back in post 9 when contemplating the initial design of Carolan we introduced the idea of ‘mappings’ between the different parts of the guitar and various web pages. For example, the headstock might map to a maker’s certificate, the soundhole to a user-guide and so forth. It’s now time to expand on this idea some more.

A mapping connects a physical thing to various digital media and services (webpages in Carolan’s case, but they could potentially be other digital things). It consists of a bundle of links that are somewhat like the hyperlinks that connect webpages, but in this case with a physical link at one end and a webpage at the other.

Each link has a physical anchor point that is embedded into the material thing. In the case of Carolan this involved inlaying aestheticodes as six distinct patterns on its wooden body. However, we could have employed other technologies such as embedded electronic tags (e.g. RFID tags) that can also be scanned to deliver digital codes. A mapping can include any number of such links.

A mapping

The next step in the argument is to realize that each physical thing may be associated with many different mappings so that it can trigger different digital content when scanned by different people in different contexts. New mappings might be created for particular ‘custodians’ (owners or players), for particular events (e.g., gigs) or localities (e.g., a music shop).

Selecting a specific mapping and then scanning the artefact conjures up a particular story of ‘account’ associated with it. Accounts might include provenance information that helps verify the history of the artefact, or utility information that helps people use and maintain it, or stories with particular personal meanings.

How do these multiple mappings come into being in the first place? While each mapping might be freshly created from scratch, we anticipate that they will very often be appropriated, meaning that the current custodian of the thing may wish to alter its current mappings to point to their own digital materials. In short if you are the custodian of an artefact you should be able to tailor its mappings to your own purposes.

We refer to this kind of appropriation as digital appropriation because the new custodian is remapping the existing physical artefact to new digital content. There is also a second kind that we call physical appropriation. This involves creating a new physical artefact that connects to existing digital content and that acts as a proxy or stand in for the original. Examples include the decorated plectrums, badges and stickers that we created for Carolan back in Post 44.

Digital and physical appropriation of mappings

The new mapping can then be copied and re-appropriated again – digitally or physically – to create further mappings. We call this recursive appropriation (Computer Scientists love recursive structures in which a simple operation can be reapplied multiple times to generate complex structures). In this way, mappings gradually spread outwards from an original artefact to produce a network of physical artefacts and digital media that are connected in complex ways – not unlike the World Wide Web itself, but involving physical things as well as webpages.

We round off our grand conceptual framework with more idea. Each mapping has an owner – or custodian – who created it. This custodian determines the links that may be altered by other people (we say that these are ‘mutable’ links because they can change) and which must remain fixed (‘immutable’). For example, I can give you a Carolan mapping in which the headstock must always point to the official maker’s certificate to show its provenance, but where you can change other links to suit your own needs.

Finally, any derived mappings that you make from my mapping should credit me as the originator. I might also harvest some usage data from when people scan the artefact using your derived mapping so that I can see how far my original mapping has spread and how many times it has been used (a reward for creating and sharing my mapping in the first place). Of course, as Computer Scientists, we know this this needs to work recursively so that credit and harvested usage data propagate back to original custodians as their mappings continue to spread.

Collectively, these concepts of hybrid artefacts, mappings, links, physical and digital appropriation, mutability, credit and harvesting define our conceptual framework for accountable artefacts. Our intention has been to provide a framework for thinking about how the custodians of future Internet-connected things can flexibly attach them to digital content, especially as they pass from person to person.

While these concepts have clearly emerged from our experience with Carolan, we hope that they will be applicable to the design of all sorts of objects that gather rich digital records over their lifetimes as they are passed among custodians – books, gifts, cars, even houses …

Phew! A long academic story. You can read the full account in our paper and we’ll give you an update on how things go when we present it at the Computer-Human Interaction conference (CHI 2016) next week.

The more practically minded among you may be wondering if this is all just academic theorizing. Are these ideas of any practical use? A fine (if cheeky) question. In a forthcoming post we’ll explain how we have implemented this framework in the latest version of the aestheticodes software so that people can create and share their own experiences that mix physical artefacts with digital materials.

Oh, and we’ll get back to the subject of guitars again at some point too.