Hi everyone, welcome to my A3 contribution. When doing research for this course I encountered a lot of Augmented Reality content, and even worked on a Moveable Feast for it. One thing that I noticed is that to experience AR/VR/MR there is always a device between the user and the experience. I created a short journey, a fictitious piece of hardware that does not get between the user and the experience. Below is a video blog, and below that, a transcript of the video with images. Enjoy!
– Mark Pepe
Transcript
Hi, my name is Mark Pepe, I’m currently a K to 7 Music teacher with Catholic Independent Schools of the Vancouver Archdiocese. I teach general music as well as concerts bands, jazz bands, and choir. I also teach French for grades 5 6 and 7. I am a classically trained operatic tenor, I sing with the Vancouver Opera Chorus, and have performed across Canada, and in Italy. I have Bachelor of Music in Opera Performance and Music Education, Bachelor of Education in Secondary Music, and I am currently a graduate student in the Master of Education Technology program. All at the University of British Columbia.
I’d like to discuss a fictional solution to a problem with augmented reality: hardware, I like to call it the middle man. The hardware between the user and the piece, or overlay, that augments our reality. When we interact with augmented, virtual, and mixed realities we are always looking through a device. This could be pointing a phone at an object, at a piece of art, or scanning a QR code and then we experience this digital phenomenon through the small screen. Or we wear goggles, or glasses, that project an overlay on the lens to our physical world.
Having a little fun here, and thinking about the future, I present the HoloBar. Imagine a device, that looks like the sound bars that give us high quality sound for our entertainment systems, except it projects a three dimensional hologram. The HoloBar gives an unencumbered augmented reality experience by taking away the middle man. The HoloBar could be mounted on the wall, ceiling, or on the ground, because it can be portable.
The HoloBar would use spatial mapping to mesh the real world environment with holographic objects, or even beings (Mahroo, Greci, & Sacco, 2019). That means HoloBar can scan the environment that it’s in to project the hologram to scale.
HoloBar can project a hologram of your favourite artist that will perform your favourite music in your own home. In this sketch, I have a hologram of Maria Callas singing some Puccini. The HoloBar will bring together a high quality three dimensional visual with high quality sound of one’s home audio system. It could be your living room, dining room, kitchen, even outdoors on the patio.
It doesn’t have to be a musical performance. It could be a piece of classical theatre, like Shakespeare, a comedian doing stand-up comedy, a lecture by your favourite thinker. It could even be an object to display, like a Fabergé Egg, a Rodin sculpture, maybe even Leonardo Da Vinci’s Flying Machine.
This is also an opportunity for artists to create a holographic NFT, non-fungible token. Digital arts are currently contributing most of the current growth in the NFT market (OVR.ai, 2021). Maybe, for a very hefty price, John Legend can give you a private concert, and after the performance it becomes your very own personalized NFT. Since NFT trading cards are trending, it can even be a life size hologram of the soccer player Cristiano Ronaldo celebrating a goal.
There is also a social aspect to this. You can converse with your friends, even watch that John Legend performance with your friends in the comfort of your living rooms. During last year’s CES convention will.i.am was interviewed and was asked what the future of music was, and he said VR. But the one problem was the social aspect, and the sound quality. People enjoy going to concerts for the social aspect; seeing their favourite artist with their friends. They don’t go for the sound quality because it’s loud and is all bass (CES, 2021).
But how would this apply to education?
Drawing from my own experience here. I still study singing privately and my teachers are based in Niagara Falls, Ontario, while I’m in Vancouver. We’ve been doing singing lessons over FaceTime for the past six years. It works really well, but one thing that is missing is not being able to see what my body does while singing. For example, through FaceTime it’s very hard to see what my diaphragm is doing. It wasn’t until we were in studying together in Italy when she said, “I can finally see that you’re not breathing low enough.” With HoloBar, my teacher would be right there with me seeing what my body is doing while I sing a challenging musical passage.
In this sketch, I have a guitar teacher giving a private lesson to a young student. The hologram would be of high enough quality that the teacher and student can clearly see what each other is doing. The student here would be able to get a good look at fingerings for various chord voicings, technique for right hand finger picking, and any other technical and musical nuances.
In this sketch, I have what could be a classroom where a teacher, and two students are collaborating on a problem. With the HoloBar, they would be able to have a holographic three dimensional model that would help them solve their problem. A student of architecture can view files from Three dimensional modelling programs such as Revit to help with iterations of their design. Graphs from an Excel spreadsheet can also be displayed to view regressions, and other types of data. This would allow for different types of learning in many subject areas.
We know that AR can, and has been used, to train nurses and doctors, trades people, and scientists in the laboratory. Objects can be seen rather than having to wear cumbersome goggles and glasses. Imagine a mechanic in training, to practice working on the engine of a rare sports car they would be able to use HoloBar to get familiar with the parts and the design (Mahroo, Greci, & Sacco, 2019).
In elementary school, HoloBar can bring the museum to the school, by displaying objects otherwise unavailable for a closer look. In this diagram, a child can be seen looking at a three dimensional cube, why, because of my poor drawing skills. I wish it was a dinosaur instead. Children are curious and want to learn. They could look at dinosaur bones, their favourite animal, or even Disney characters.
Where are we currently for a technology like this?
Well, they’re already on their way. Scientists from the Interact Lab at the University of Sussex have designed what they call a Multimodal Acoustic Trap Display, MATD. Essentially, it’s a small box with small speaker array on the top and bottom, an LED light, and 2mm in diameter bead. Sound waves from the array keep the bead afloat and moving while the LED lights the bead. Since there’s a speaker array, sound can be incorporated, and those same sound waves can also provide some tactile feed back. This is just but one of many examples.
I hope that you enjoyed my journey into what I hope the Augmented Reality experience will eventually become.
Thank you.
References
CES 2021. (2021, January 13). Moving into the Future with will.i.am [Video file]. CES. https://www.ces.tech/Videos/2021/CES-Program/Anchor-Desk/Moving-into-the-Future-with-will-i-am.aspx
CNET (2019, November 29). You can feel and hear these holograms [Video file]. YouTube. https://youtu.be/tzWP-NL3Lck
Hirayama, R., Martinez Plasencia, D., Masuda, N. et al. A volumetric display for visual, tactile and audio presentation using acoustic trapping. Nature 575, 320–323 (2019). https://doi.org/10.1038/s41586-019-1739-5
Mahroo A., Greci L., Sacco M. (2019) HoloHome: An Augmented Reality Framework to Manage the Smart Home. In: De Paolis L., Bourdot P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2019. Lecture Notes in Computer Science, vol 11614. Springer, Cham.
https://doi.org/10.1007/978-3-030-25999-0_12
OVR.ai. (2021, February 2). NFTs on Steroids with Digital Arts Leading the Way. https://www.ovr.ai/blog/nfts-on-steroids-with-digital-arts-leading-the-way/
Hi Mark
I love the idea that you brought into the A3. I think holographic display could be the future. As technology grows to be smaller, I would be fascinated by displayed from mobile devices transforming from a screen to a hologram. I also enjoyed the amount of artwork that made it easy to visualize the idea in action.
I can definitely imagine this technology being combined with virtual reality to create cool situations where you are able to virtually go somewhere else and have the other person be able to interact with you via the holobar.
I am wondering about the extent that you think this technology will go. I can see that holobars could become a way to visually display the amount of information in 3d, however, would it be possible to interact with the said display? I’m thinking of Tony Stark in the Marvel movies where he was able to create holographic technology that you can interface with. This could potentially create new ways of studying as one would be able to visually see and play around with the idea of a geometric shape or a cell structure. Very cool.
Hi Mark,
This is a wonderful idea. I really like that you have the application of this technology mapped out. It really did convince me that this would be a useful tool not only for entertainment but also for teaching and learning purposes as well. Great job!
Emily
Good stuff Mark.
Now you are thinking. A HoloBar could have profound implications in the retail industry. As we, the Holobar also takes into consideration some of the serious problems we are currently experiencing with emergency online teaching. Imagine how our work environment would change with this type of technology. Great A3 Mark!
Hi Brittany,
Thanks! I had a lot of fun creating this project. It’s easy to let our imagination run wild with the potential of such a technology.
Mark
Hi Mark! Great idea, and like others, I enjoyed your presentation of it, as well as the unfettered, “blue sky” imagination behind it. I’m old enough to have seen “Star Wars” when it released, in a theatre, and it’s impossible for me to not recall the similar, projected holograms as imagined nearly 45 years ago. Like hoverboards, food replicators, teleporters and jetpacks, I think the idea has a permanent spot in the canon of our cultural imagination — many of us have been waiting our whole lives for these things.
But if the future is not now, it’s surely soon. As you note, technology already exists (and is implemented on mobile devices) that facilitates the capture of 3-D information, and even a cheap gadget can process more 3-D information now than desktop workstations of a decade ago — this output technology could bring it all together to make 2-D screens look as anachronistic as a newspaper. That said, I’m also reminded of the way that television, in its infancy, was widely assumed to be a great equalizer in education — classes could be taught remotely to isolated students by a “lifelike” teacher, students could watch educational programming rather than read a dusty old book, and for the one-time cost of a television set, anyone could learn anything they wanted via the airwaves! Television seemed, once upon a time, to have had similar potential to this, but failed to realize it.
Do you think we would make better use of such technology now? Have we learned from all of the other technologies that seemed like game changers, but then devolved into high-tech fidget spinners instead?
Hi Sean,
Thanks for the reply. I’m hoping we can make better use of these technologies. When we were preparing our A2 project, a lot of what I found, for music and arts at least, was quite gimmicky. I also found a new article, from Forbes I think, about this billions of dollars that would be invested into AR marketing. If that much money is put into marketing, how much is being put into research and development for practical application. We’ll see!
Thanks,
Mark
Hi Mark,
Thanks for sharing your idea of HoloBar! I love what HoloBar could do and its convenience of experiencing augmented realities without wearing goggles and head sets. While imagining having a ballet instructor teaching my kid in our living room using HoloBar, I am wondering if the images need to be captured using special 3 dimensional technology (like recording the images from all angles) from the other end to be projected as a 3D hologram? If any images or videos that we capture on our mobile devices can create holograms using HoloBar, that would be life-changing in AR industries.
Hi Esther,
Thanks for the reply. We’re stepping closer because of LiDAR technology. Recent phones have them included. I think the most recent iPhones and iPad Pros. The camera uses a pulsing laser beam to detect light and ranges, creating a three dimensional image. Neat stuff!
Thanks!
Mark
Mark – bravo!! This is SO well done! Not only do I love your topic, but your drawings are beautiful. As someone who is not artistic or overly musically inclined, I really appreciate what you’ve come up with. I can see the benefit for individual lessons, but also for larger group lessons and exposing younger children to music and the arts with the hopes of igniting a passion at a young age.
As a Foods teacher I can’t help but think about how the hologram can be implemented in my classroom – how amazing would it be to bring a professional, celebrity chef into the classroom for a demonstration? Or getting a 3D look at a vegetable or fruit that’s now available in the Okanagan? Some cooking techniques, like musical techniques, need an up close observation to replicate the skill – the HoloBar could give this personal, in your face demonstration!
Could the HoloBar be mobile? Do we need life-size holograms or would smaller versions still be effective?
Like Lindsay, I hadn’t heard of NFT’s … adding it to the list of things I need to look into after taking this course (apparently I’m more outdated that I realized!!).
Thank you again for an elegant, well planned project. Amazing work!
Meg.
Hi Meg,
Thanks! First thing I thought of when you mentioned foods is knife technique! How to properly cut veggies, or how to properly fold meringue if one’s making macarons.
The way I drew the HoloBar it doesn’t look too mobile. But maybe future iterations are! May I present, the HoloPuck!
Yes, these NFTs are a recent phenomena. They went a little wild back in February where some were selling for hundreds of thousands of dollars, maybe more. Someone called them a digital deed, so in the future it could be a way of owning a digital artifact. Interesting stuff.
Thanks again!
Mark
Knife skills would be amazing with a hologram!! And we just made macarons in my Foods 11/12 class on Thursday so I really could have used it then to show the macronage technique… I think it could still be just as useful in the HoloPuck version!
Hello Sean! I am excited about the potential of your A3 project. Is there a possibility of becoming a tester for your technology? I am interested in the AR/hologram bar technology since it can aid a teacher’s lesson to a real-life application or create a personal musical experience. The quality of your work and critical thinking process allowed me to understand AR/hologram technology’s complex realities fully. I believe explanation, clarity and comprehension are essential to a successful application of technology. I love how you clearly explained the AR/holo bar’s potential and your confidence in the technology that it will give you a personal experience.
I am interested in the technology’s musical aspect since we are trying to create a new musical experience for our students. From my understanding, please help me if I am wrong, holographic technology needs a transparent glass type material, which allows for the hologram’s projection. Does the holo bar need a glass to project the image?
I love how you created a personal connection with the project, which tells me that it will work and your project’s potential is very high. At this point, I strongly feel that your technology will assist teachers and transform how the information will be transferred in the classroom. It is fascinating to hear your idea. In my opinion, your A3 project is a success. Thank you for sharing your thoughts. I am very humbled and honoured to have worked along side you. Best wishes to you!
Hi Tyrone,
Thanks for the reply and kind words! You are correct, currently holographic technology requires glass, lasers, projections, etc. There is a type of technology called “volumetric display.” One of the best examples I can find is referenced where there are two arrays of very small speakers that essentially make a very small bead float and has life projected on it. It really looks like Leia’s projection in Star Wars. Though I just found that Samsung has developed a holographic display akin to those holographic hockey cards.
Thanks for reading and watching!
Mark
Mark,
I really enjoyed your OER and thought the Holobar would be a great addition to my A3, the AR-Podcast (https://blogs.ubc.ca/etec523/2021/04/03/a3-whos-ready-for-the-ar-podcast/). Removing screens and physical items from a user’s body places the tech more seemlessly into the user’s context. I think that significantly improves the UX of AR and builds on the context-aware affordances of mobile.
I hadn’t heard of NFT’s before your video, but after seeing a few A3’s mention it I can see I’m out of date! I’m still not sure I understand the concept and look forward to learning more. From what you present, I think it’s clear the term will be increasingly common as 5G enables better and better access to AR/VR/MR tech.
I do see the potential for a heavy or hard-wired holobar to no longer be mobile, more akin to an overhead projector. But if it could be worked into a mobile device or shrunk into a light, compact form it could be a useful form of mobile tech. It is interesting to consider that having AR or VR visuals present in our context without having to look through a screen embeds the text more deeply in a user’s context. But that separation from a user’s body may make it less mobile overall (I’m reminded of early readings in the course which suggested our smartphones have become an extension of ourselves). Prehaps the best iteration would be to have the holobar built into our smartphones or tablets? I’d love to hear your thoughts!
Again, great project!
Hi Lyndsay,
Thanks for the reply. I’ve really enjoyed your podcasts, I even follow on Spotify! I’m going to listen to those AR podcasts that you mentioned.
The NFT craze of February may have subsided but it might be the future to own digital artifacts. I read somewhere recently where it was called a digital deed.
Definitely, the device would have to be portable to be exemplary of mobility!
Thanks!
Mark
Hi Mark,
I think that the HoloBar has the potential to change the lives of individuals who cannot go to live performances, due to accessibility issues. I really like how you had accessibility in mind by transcribing your video underneath! Because the HoloBar can project onto people, I also wonder whether or not this technology could help individuals with social interaction. Students with autism could practice social stories with their teachers at home, with their teachers face and body projected onto one of their parents. Because you mention using MR to read bodily cues, this technology could also allow PE teachers to teach remotely or to teach exercise programs to seniors. I thought your video was informative and put together well. I was actually very impressed with your drawing skills and laughed when you made the dinosaur joke. Good job!
Jen
Hi Jennifer,
Thank you for the reply. You bring up great points. One of my references titled “HoloHome,” (which I was originally going to call it, but it was taken by this study!) is actually a smart home concept for individuals with mobility and visual issues. AR and MR have a lot of potential to help all types of populations in the future. Something that I think is currently overlooked, at least to best of my knowledge.
Mark