My A3 is a mobile application Proof of Concept. I named it “Speakeasy.” It combines mobile technology with AI, speech recognition and gamification, and aims to improve language learning education, particularly pronunciation and conversational skills. Enjoy!
https://sites.google.com/view/523-a3-ac/home
Hi Aaron, I appreciated how you used your experience with language acquisition apps and their weaknesses and thought up an alternative app that filled in the gaps. I use Duolingo frequently and, as you mentioned in your reflection, the use of gamification makes it incredibly fun to use. I really enjoyed being able to see what Speakeasy would look like and how it would work. Being able to practice pronunciation, work the muscles in our face, and fine tune our hearing to new languages is challenging to do on apps like Duolingo, but having an app that can identify our weaknesses would be invaluable. In your reflection you also mentioned possible ‘next steps’ such as incorporating AI to facilitate conversations. I was wondering, since the app could be collecting data on your level of English, etc., perhaps they could connect users of similar learning levels who want to practice conversations. I know that when learning a new language, it is important to have conversations with fluent speakers, but practicing basic sentence structures could be a fun addition.
Aaron, I enjoyed your presentation. The graphics are very appealing. Coming from an Asian background, I appreciate how difficult the pronunciation of certain sounds can be. My mother and grandparents had a very difficult time. I think the images of people’s mouths making the sound are very helpful. Have you thought about diagrams showing the tongue position for making some of the sounds that are not self-evident facially? I can see a potential for having lessons for various businesses and industries, where different vocabulary may be used. Well done!
Thanks for your kind feedback Tamaka – yes absolutely, diagrams of inside the mouth would be very helpful. These images would rather be “pop-up” videos that explain the pronunciation in the user’s mother tongue.
Well done, Aaron! The design of your site is easy to navigate and is well organized. I think pronunciation can be very nerve wracking for certain students. I had a few students in my class who spoke English as their second language. They would answer questions frequently, however, I could tell they were nervous to approach certain words due to being unable or uncertain how to pronounce those words. I appreciate your incorporation of gamification as this could make it more relaxing for students, and less nerve wracking. You have a great idea here and with the help of AI, I believe this is a piece of technology we need. Thanks, Aaron!
Hi Aaron. Nice work! This is a well-thought-out application that would be valuable to have in the learning marketplace today. I believe you achieved your objective of coming up with an idea of high usability by identifying a need and gaps that exist in the current technologies that offer solutions. Indeed, pronunciation problems that are specific to different groups of people are something that needs to consider with greater precision to achieve better outcomes. Adding the gamification feature is a smart idea that would allow students to relax, be less self-conscious, and forget about the timidity that could happen in learning a new language. I appreciate how you included various visual examples of how the app would work and think about things you would have done differently and the next steps. Definitely, creating an app must be an iterative process of refinement,
Hi Aaron.
I appreciated the demo photos to give the audience context of your descriptions.
Interestingly, last semester, I had created a game for one of my classes. The topic was Spelling for English Language Learners. After much debate, we opted out not to have a speech corrector or rater. Our worry was that accents become something to be ashamed of. We also worried that this would become exclusion to some groups based on their accents.
On the other hand, from my experience and conversation with multilingual families, a struggle was pronunciation that in turn affected their ability to spell correctly. For example, if they pronounce a word as “Crip” instead of “crisp”, when spelling, especially for younger spellers, they would likely spell it as pronounced. When teaching online, I had recorded myself saying the spelling words and linked it to each Sight Word of the week to support students in listening to the letter sounds and spelling it out. Parents have also voiced how this was a great strategy for students to work on their spelling independently and with more success. So I see value in your education technology as it can be accessed anywhere and anytime, and it supports students in working and learning independently. It would also be awesome to include sight words for younger students or simple sentences for them to learn independently.
My question for you would be: How could you adapt your technology to also support students who are hearing impaired?
Hi Rika – I did notice that relationship between pronunciation and spelling as well! In my case it was “twelfth” vs “twelse”, but we can see how simply learning the “th” sound can improve the pronunciation and spelling of many words. To be honest, I don’t have much knowledge or expertise working with people with hearing impairments, but I would imagine it would require a separate application altogether, rather than an adaptation (e.g. adding captions to the “Quick Hints”)? Another possibility could be Braden’s idea of using mouth-tracking (with the front-facing camera) to teach the user how to position their lips, tongue, and teeth.
Thanks for the presentation on ESL through using artificial general intelligence. To add to your discussion, the Taiwanese Educational Tech Research Development has been actively researching the use of 3D avatars in the EFL elementary classes. It shows a lot of promise.
I checked out LaMDA, which is open source like BERT and GPT-3. Have you located the LaMDA tool?
Hi Aaron, I enjoyed reading your OER and proof of concept mockups for the app (great App name as well haha, makes me want to grab a drink). Feature wise, I think it’s unique enough to differentiate itself from other similar concepts on the market as I appreciated how you put your own design choices into the mix. Smart pronounce in particular is a handy feature which I think could/should be included in more language learning tools. I’m curious to see if the app will be able to include options for learners to practice with each other or have some kind of team learning component to it as from experience, it’s more effective to learn a new language when there are opportunities to practice with other people. Accessibility wise, would it be possible to include an update to assist learners who might have hearing difficulties? On that note, I think it would be great if you could introduce the concepts introduced in the OER to your students in Korea as there’s good potential to apply the ideas into a practical environment (languages are living concepts!), great job!
Thanks for the feedback, John. I do think there is potential for team learning – I considered a sort of ranking system that could be enabled within the Lesson Builder feature. Within a classroom setting, the application is meant to supplement the traditional learning experience – so perhaps the teacher could make use of the students’ data. Rika also had a similar question about hearing impaired users, to which my response was: To be honest, I don’t have much knowledge or expertise working with people with hearing impairments, but I would imagine it would require a separate application altogether, rather than an adaptation (e.g. adding captions to the “Quick Hints”)? Another possibility could be Braden’s idea of using mouth-tracking (with the front-facing camera) to teach the user how to position their lips, tongue, and teeth.
Aaron,
I really like your idea for a variety of different reasons but most importantly for the reason you stated when you were discussing the problems. Language acquisition is continually so challenging for people from all over the world who move to English-speaking countries and the stigma that comes around that is very disheartening. I remember when my mom (who is from Singapore) got her Canadian citizenship, the first remark from the government office worker was that she was so amazed that she did not have an accent. As someone who has experience in this field, you bring up the important issues. Oral language skills are something that I work on with a lot of my students who are new to the country and I think this technology can really help young people and adults alike who require that discreetness to feel more confident.
Hi Aaron, I appreciated how you took existing applications, such as DuoLingo, and enhanced them. I think that your inclusion of self-recording would be powerful for users and help make their learning more personalized and authentic. Having this application easily available would also encourage students to learn new language skills on their own time outside of the classroom and make learning new languages more accessible since the mobile device can use the individual’s accessibility settings they already utilize. I would be curious to see if you could further your design by having the AI make use of the camera on many mobile devices to monitor mouth movement to ensure proper phonetic pronunciation.
That’s actually a great idea (inspired by eye-tracking? :D) – that way students are aware of exactly what they’re doing physically, and also receive feedback from the app. It’s a great way to learn phonetic pronunciation.