Quite a lengthy post today about capturing 3D models of physical stuff. It goes round the houses a bit, with some interesting diversions along the way, but does (of course) end up with guitars … so hang in there guitar addicts.
Photogrammetry is the technique of obtaining useful information from photographs. Its applications range from geographers generating maps from satellite images, to engineers precisely measuring objects as they progress along factory production lines, to animators capturing 3D models for games and virtual museums. In spite of this potential, the technology has so far remained out of the reach of everyday folk due to a combination of the cost of equipment, the complexity of the process and the specialist knowledge involved. Up until now anyway.
Our colleague at the Mixed Reality Lab (and collaborator on the Carolan project) Dr Dimitri Darzentas has created a low-cost platform called ScannerBox that enables DIY photogrammetrists to scan their own stuff and generate 3D models. This began life as part of Dimitri’s PhD research where he studied the practice of tabletop wargamers, people who meticulously craft miniature figures that they then deploy in tabletop battles, for example in games such as Warhammer (from Nottingham-based company Games Workshop). Dimitri was interested in how these lovingly crafted objects could tell stories about how they were made and the various campaigns they had fought in, an idea that directly mirrors our approach of associating stories with Carolan.
Dimitri initially developed ScannerBox to capture 3D virtual models of these miniatures at various stages of completion and then publish the results on the popular platform Sketchfab as part of his Mixed Reality Storytelling project. He also embedded selected models into a virtual reality museum so players could don a VR headset and encounter their creations at life size and created an augmented reality experience that overlaid virtual graphics on physical dioramas. Sounds a bit complicated? It’s perhaps easier to watch Dimitri explain and demonstrate his ideas in these two videos produced for the Computerphile Youtube channel.
ScannerBox also captured the attention of several museums who wanted to use it to digitise their own or their visitors’ objects. The Armchair Gallery app developed by Nottingham’s City Arts offers people with dementia and their carers the opportunity to virtually visit various museums without travelling. They used Dimitri’s system to scan the Vestal Virgin sculpture at Chatsworth House to generate a 3D model for inclusion in the app.
Artist Claire Twomey staged an exhibition at Tate Exchange in which thousands of visitors learned to make ceramic objects such as flowers and jugs which they then left behind for other visitors to collect and take away.
Given they each visitor had to leave their own creation behind, it was also ‘digitally fired’ using ScannerBox so they could accessing a 3D model on Sketchfab as a way of remembering it.
You can select and explore the image below to explore one of the models on Sketchfab. The controls take a bit of learning, but with practice you should be able to spin it around in all directions and zoom in and out.
So what has this to do with guitars?
Well of course we wanted to scan Carolan. Given the size of the guitar, this required us to build a reasonably large scanning booth and then figure out how to best position and secure the guitar so that as much of its surface could be photographed as possible while it remained stable. We also used a ‘green screen’ background to make it much easier for the software to extract the guitar from any background clutter in the photos.
You can explore the 3D model that resulted from our first attempt below. You can clearly see the texture of the spruce soundboard, flamed maple back and sides and even the graining of the mahogany neck. Other aspects have come out less clearly, most notably the strings and some of the reflective surfaces such as the tuning keys.
We’ve added annotations to highlight key features of our guitar including its six Artcodes and also the various dents and dinks that we audited back in Post 75. Just select an annotation to jump to that feature.
And for those who are struggling to control the model, here is a screenshot captured from a funky angle …
And here’s a second that shows the detail of the model close up, with one of Carolan’s dinks visible.
Why would one go to the trouble of scanning guitars in this way? What uses might there be for 3D models of guitars?
Shopping for guitars: first, they might be useful in online guitar shops. The buyer could inspect an instrument from many angles, appreciating its features – the grains of woods and details of inlay – but also any patina, including damage, close up. Of course none of this conveys the feels of the instrument and some would argue (me among them) that there is no substitute for playing before buying. But this might be different for collectors where guitars are appreciated as much for their histories, stories and physical condition as their playability and sound.
Identifying lost and stolen guitars: the guitar world is replete with stories of guitars that were lost but then ultimately found again, often involving painstaking detective work to verify their provenance. Why not maintain an online record of your valuable instruments, especially of their distinctive patina, that could help identify them in the future? Might this be an additional after sales service provided by manufacturers or retailers?
Guitar museums: we virtually visited some of the world’s leading guitar museums back in Post 71. Following Dimitri’s approach with Wargaming miniatures, could these curate online exhibitions of their instruments for virtual visitors, an especially attractive idea at this time of COVID-19 lockdown. Players could virtually donate their guitars to museums, along with personal stories, where they are not willing to physically hand them over.
User guides: richly annotated 3D models might extended user manuals showing how to use, maintain, repair and record guitars (see the data about recording techniques we added to Carolan’s user guide way back in Post 38 and Post 41). ind you we’l need to be able to scan the insides as well as the outsides (note to selves – must test that out soon!) to show those on board electronics that are often so tricky to locate and understand. We could also position virtual microphones around the model to show optimum positions for recording the instrument.
Keeping in touch: Finally Carolan is currently locked down away from home in nearby Sheffield where it has been on loan for a couple of months now (more on this soon) and I’m missing it. Just as visitors to Claire Twomey’s exhibition at Tate were able to keep in touch with the artefacts they had made and donated, so viewing the model has helped me remain feeling emotionally connected to Carolan until we are reunited.
If you’re interested in doing your own DIT photogrammetry, you can learn how via Dimitri’s online guide. All that is necessary to are several good images of the object or environment that you wish to scan. The source of this images can be any camera, whether an inexpensive web camera, top-of-the-line SLR camera, or your mobile phone’s camera. Commonly available photogrammetry software then handles the task of analysing the images, comparing them for similarities, and using them to measure the dimensions and geometry of the subject and finally generate a 3D model.
Carolan Guitar 
Looks great! how did you clean the guitar stand?
LikeLike
Hi Adam – thanks for the question. Here’s Dimitri’s response …
“The thing is that the process relies on the subject being immutable – its features must remain the same in all the pictures in order to be picked up for reconstruction. This means that the parts of the subject must remain fixed in the same position for all images.
The stand was not fixed to the guitar and I had to reposition the guitar and stand for every shot. It was impossible to have the guitar sit in the stand in the exact same way every single time – either the stand legs would be at a different angle or the guitar would have been placed on it somewhat differently. It is very difficult to achieve that consistency across hundreds of photographs if you have to move the subject (that’s where you either use something like a turntable, or move the camera around instead).
So what happens is that the reconstruction process decides that the stand is an unreliable feature and decides not to create any 3D geometry for it. It essentially decides that it is not part of the model – which in this case is correct!
It’s almost the same as shooting a picture with a long exposure time. If you set a camera to take a long exposure photograph, say a 15+ second exposure of a night scene, then you could most likely walk straight past the camera and you would most likely never show up on the film.
So if you want to create a 3D scan of a street, using for example a drone, then you would not necessarily need to worry about people walking past – they would not show up in the final model. Static objects would however, such as a parked car for example.
Thus this apparent limitation can be used to your advantage. If you want a part of an object to not become part of the 3D model you can move it inconsistently between the pictures thus improving the chances it will be dropped from the processing.
This way you can save time on post-processing the 3D model – although results may vary.”
LikeLike