Designing AR apps to close the skills gap
Every semester as Dr. John Liu walked through the machine shop in MIT Building 35, he heard his fellow instructors give the same introductory lecture to students over and over again. Liu, a lecturer in mechanical engineering, Digital Learning Lab scientist, and principal investigator of the LEAPGroup, felt there must be a better way to capitalize on his colleagues’ time. As an educator, Liu had simultaneously been thinking about ways to scale manufacturing training to address the industry skills gap across the nation. In 2018, Deloitte estimated 2.4 million manufacturing positions would go unfilled between then and 2028. Liu saw a connection between the millions of unfilled jobs and the limited capacity of lab instruction.
Liu and his fellow LEAPGroup members sought grants from J-WEL, MITx, and others to develop augmented reality (AR) apps for 2.670 Mechanical Engineering Tools, 2.008 Design and Manufacturing II, and 2.674 Micro/Nano Engineering Laboratory with the goal of creating technologies that are proven to have the same efficacy as equivalent in-person instruction. During the summer 2021 semester, Liu’s students in the Undergraduate Research Opportunities Program (UROP), plus one edtech staff member from LEAPGroup, divided into five groups to work on different ways to apply AR in different manufacturing contexts — such as introductory manufacturing disassembly, learning nanotechnology through multi-modal experiences, and reimagining 2.670 in a blended classroom model.
Liu says that the literature review on AR apps in education reveal that many of the apps tend to operate in conjunction with the “lower” modes of cognitive thinking, (e.g. recalling and remembering facts), rather than “higher” modes of instructional design that give students the opportunity to synthesize, evaluate, create, and apply what they’ve learned. Now Liu wants to push the envelope on AR apps so learners evaluate and analyze in non-trivial ways, with the goal of turning manufacturing novices into experts. He hopes these apps will go beyond simple equivalency to the classroom experience, and create richer dynamics between instructors and students by supporting learners more holistically. He says, “I would count that as a win because that comes with benefits such as scaling and flexibility.”
At the conclusion of the summer 2021 program, the five groups who assisted with LEAPgroup’s research presented the progress of their projects to an audience of collaborators, colleagues, faculty, and industry members. The presenters showed a variety of ways in which AR apps can provide scalable, interactive, scaffolded instruction to guide learners through the manufacturing process. The 2.670 course introduces learners to the fundamentals of machine tools use and fabrication techniques, 2.008 integrates design, engineering, and management, and 2.674 encourages creative thinking about underlying science and engineering principles through hands-on experience via building, observing and manipulating micro-and nano-scale structures. While learners taking these courses normally have access to a variety of tools in the machine shop, not every manufacturing classroom has access to these resources, and during the pandemic, Liu’s students also had limited access. The in-person learning experience also presents other constraints: a limited instructor-to-student ratio, a finite amount of classroom time in the machine shop, and a small pool of learners at MIT who have access to this instruction in the first place. AR-based instruction can scale learner engagement on all of these levels.
Learning manufacturing through product disassembly
The authors of the first project, Scaling Hands-On Learning Principles in Manufacturing Through Augmented Reality Disassembly and Inspection of a Consumer Product, were fresh off a paper presentation at the American Society for Engineering Education 2021 Annual Conference (considered the premier conference in this field), where the paper was awarded Best of Division in Manufacturing and Best of Professional Interest Council II (out of 10 divisions). The project explored how an AR app can simulate and expand on the hands-on classroom experience. Normally, instruction in the 2.008 course involves passing around toys and other products to disassemble in class, fostering understanding of how objects are manufactured by taking them apart. Through the app, learners engage in manufacturing principles through 3D visualization of disassembly and physical manipulation instead. The development of the AR app was partly funded by a J-WEL grant in Higher Education Innovation and an MITx Express Exploration grant.
Anyone can download this product disassembly app in the Apple App Store and Google Play, regardless of program affiliation, class size, or available resources to the learner. The app guides learners in “thinking through” processes and problems relevant in manufacturing. Instead of a 2D video on a screen — the primary vehicle of remote learning thus far — the app provides a visually rich experience with 3D AR projections into your physical context (e.g. onto your table or desk). A learner can have interactive engagement, contextualized within their own environment. The potential of this app goes beyond the introductory levels of training. For example, it could be used in the specialized training of taking apart a wind turbine or a nuclear reactor. Liu also hopes that the app can also serve as a platform to study how students learn. All interactions with the app incidentally become data on the user experience, informing how best to teach the material. “We need to optimize the user experience of the app to make sure it does not distract the learner, but are more interested in their learning experience,” he says.
A blended classroom and machine shop
Another project focused on Machine Shop Instruction, which turned fabrication instructions from the 2.670 course into an Augmented Reality app. One of the major bottlenecks of hands-on skills training in manufacturing is the trainer’s time. A J-WEL Workforce Learning Innovation Research grant partly funded the development of a different AR instruction-based app to produce a “blended machine shop” for 2.670. The idea of a blended machine shop was based on the flipped classroom model, which focuses classroom time for “active” forms of learning — such as problem solving, critical thinking, and in-person engagement — and delivers the vital (but repetitive) lectures through supplemental materials that learners access on their own time, replaying and revisiting as needed. Online videos, multimedia materials, or even traditional worksheets could be cost-effective tools but they are not as personally interactive as in-person instruction, and they’re disconnected from the student’s physical context. A flipped classroom combined with AR raises the opportunity to transform the way labs and hands-on skills in manufacturing are taught, because instructors can share their expertise on a larger scale.
Next Steps
Liu next plans to test the apps with larger cohorts of students this fall to further explore the learning effectiveness of this technology. Although one of these apps is already available publicly, most of the apps are still at the pilot stage. Beyond product disassembly, Liu wants to create other authentic learning experiences through AR. He also aspires to make the app a platform to learn more about learning. He asks learners, “Can you do more in less time? Are you more motivated to pursue the development of these skills? Are you applying these skills more often in your field?”
Original Article from MIT Open Learning.
