nui.joshland.orgDeconstructing the NUI

skip to main | skip to sidebar The next evolution of computer interfaces is the Natural User Interface. I analyze and discuss the NUI, multi-touch devices, and gestural interfaces. Monday, November 18, 2013 Kinect for Windows v2 dev kit unboxing (video) I just received my Kinect for Windows (K4W) v2 dev kit and opened it for the first time on camera: This is the alpha version of the hardware that is being sent to K4W developer program members sometime very soon. The final version may differ, and the power supply situation will definitely be smaller. The K4W team gave the K4W MVPs their v2 dev kits today, November 17th, during the MVP Summit. (Microsoft MVP = Most Valuable Professional, which is an award for people to contribute to various technical communities.) The team is also generously allowing developer program members to share and demo the pre-release sensor hardware and applications publicly without any NDA restrictions. I'll follow up with another video in a few days after I am back home and have set up the new sensor with my computer. Note: In the video, I said that the model number for non-USA dev kits would end in 2. There may also be a 3 version. The difference is only which power supply is included. Don't plug in the dev kit to power in the wrong country! at 11:01 PM Links to this post 6 comments Feedback: Labels: k4wdevkit , kinect , Video Tuesday, August 27, 2013 Cross-post: Joshua Blake on Kinect and the Natural User Interface Revolution (part 3) The post below is cross-posted from the Kinect for Windows Developer blog . The introduction is written by Ben Lower of the Kinect for Windows team and the body is written by me (Josh). The following blog post was guest authored by Kinect for Windows (K4W) MVP, Joshua Blake. Josh is the Technical Director of the InfoStrat Advanced Technology Group in Washington, D.C. where he and his team work on cutting-edge Kinect and NUI projects for their clients. You can find him on twitter @joshblake or at his blog, http://nui.joshland.org . Josh recently recorded several videos for our Kinect for Windows Developer Center. This is the third of three posts he will be contributing this month to the blog. In part 1 , I shared videos covering the core natural user interface concepts and a sample application that I use to control presentations called Kinect PowerPoint Control. In part 2 , I shared two more advanced sample applications: Kinect Weather Map and Face Fusion. In this post, I’m going to share videos that show some of the real-life applications that my team and I created for one of our clients. I’ll also provide some additional detail about how and why we created a custom object tracking interaction. These applications put my NUI concepts into action and show what is possible with Kinect for Windows. Making it fun to learn Our client, Kaplan Early Learning Company, sells teaching resources focused on early childhood education. Kaplan approached us with an interest in creating a series of educational applications for preschool and kindergarten-aged children designed to teach one of several core skills such as basic patterns, spelling simple words, shapes, and spatial relationships. While talking to Kaplan, we learned they had a goal of improving student engagement and excitement while making core skills fun to learn. We suggested using Kinect for Windows because it would allow the students to not just interact with the activity but also be immersed in virtual worlds and use their bodies and physical objects for interacting. Kaplan loved the idea and we began creating the applications. After a few iterations of design and development, testing with real students, and feedback, we shipped the final builds of four applications to Kaplan earlier this summer. Kaplan is now selling these applications bundled with a Kinect for Windows sensor in their catalog as Kaplan Move-NG . The Kinect for Windows team and I created the videos embedded below to discuss our approach to addressing challenges involved in designing these applications and to demonstrate the core parts of three of the Move-NG applications. Designing early childhood education apps for Kaplan In the video below, I discuss InfoStrat’s guiding principles to creating great applications for Kinect as well as some of the specific challenges we faced creating applications that are fun and exciting for young children while being educational and fitting in a classroom environment. In the next section below the video, read on for additional discussion and three more videos showing the actual applications. Real-world K4W apps: Designing early childhood education apps for Kaplan (7:32) One of the key points covered in this video is that when designing a NUI application, we have to consider the context in which the application will be used. In the education space, especially in early childhood education, this context often includes both teachers and students, so we have to design the applications with both types of users in mind. Here are a few of the questions we thought about while designing these apps for Kaplan: When will the teacher use the app and when will the students use the app? Will the teacher be more comfortable using the mouse or the Kinect for specific tasks? Which input device is most appropriate for each task? Will non-technical teachers understand how to set up the space and use the application? Does there need to be a special setup screen to help the teacher configure the classroom space? How will the teachers and students interact while the application is running? How long would it take to give every student a turn in a typical size classroom? What is the social context in the classroom, and what unwritten social behavior rules can we take into account to simplify the application design? Will the user interaction work with both adults and the youngest children? Will the user interaction work across the various ways children respond to visual cues and voice prompts? Is the application fun? Do students across the entire target age group understand what to do with minimal or no additional prompts from the teacher? And most importantly: Does the design satisfy the educational goals set for the application? As you can imagine, finding a solution to all of these questions was quite a challenge. We took an iterative approach and tested with real children in the target age range as often as possible. Fortunately, my three daughters are in the target age range so I could do quick tests at home almost daily and get feedback. We also sent early builds to Kaplan to get a broader range of feedback from their educators and additional children. In several cases, we created a prototype of a design or interaction that worked well for ourselves as adults, but failed completely when tested with children. Sometimes the problem was the data from the children’s smaller bodies had more noise. Other times the problem was that the children just didn’t understand what they were supposed to do, even with prompting, guidance, or demonstration. It was particularly challenging when a concept worked with older kindergarten kids but was too complex for the youngest of the preschooler age range. In those cases there was a cognitive development milestone in the age range that the design relied upon and we simply had to find another solution. I will share an example of this near the end of this post. Kaplan Move-NG application and behind-the scenes videos The next three videos each cover one of the Kaplan Move-NG applications. The videos introduce the educational goal of the app and show a demonstration of the core interaction. In addition, I discuss the design challenges mentioned above as well as implementation details such as what parts of the Kinect for Windows SDK we used, how we created a particular interaction, or how feedback from student testing affected the application design. These videos should give you a quick overview of the apps as well as a behind-the-sc...