This was a very interesting BUZZ that I found today and wanted to share with you.
Originally posted on: vrscout.com/news
In case you haven’t seen it, a new tweet from a medical doctor, software entrepreneur, and 3D artist, Fayez Salka shows him using AR technology to bring a virtual piano player to life on a real-world piano.
The video, which has already garnered almost 200K views on Twitter and continues to climb, shows Salka walking through a piano store aiming his phone’s camera at various pianos. Instantly, an AR pianist appears and begins playing different songs from a handful of award-winning pianists while sheet music of the corresponding notes scrolls across the bottom in real-time.
The app uses machine learning to detect the piano, and then figures out the exact 3D position and orientation of your piano in a 3D space down to 1 cm accuracy. It does this just by looking at a single frame and without the use of 3D depth cameras.
During an interview with VRScout, Salka explained the importance of this approach saying, “This is crucial so we can overlay and align the digital pianist exactly on the correct place on your piano,” adding, “What’s cool about our Core ML model is that it works with any piano model or brand under any lighting condition, without the need to use special image targets or markers or the need to scan the piano model ahead of time.”
Though the app isn’t available in the iOS store just yet, Salka and his Massive Technologies Inc partner, Anas Wattar, see the potential of their AR experience as an excellent tool for providing a new way to experience piano music on your smartphone or AR headset, as well as a helpful resource for music education.
“You’ll be able to slow down the song up to 100 times without losing information regarding the hand motion,” said Salka. “This is not possible with the traditional recorded video. Obviously all of that carries on to the AR headset too.”
The full version of the app will also use your smartphone’s microphone to listen to you while you practice, and through machine learning, give you instant feedback on your playing; it even uses the AR pianist to offer you tips on how to exactly move your hands during your performance.
For Salka and Watter, this started out as a passion project and ended up turning into an actual business and a full-time job. As a matter of fact, Massive Technologies had some success with an older AR app called Piano 3D, which was downloaded more than 7 million times and was even featured by Apple multiple times on the front page of the App Store and on demo devices in Apple retail stores.
For now, they are both just really excited that their latest AR experience is building up a lot of buzz from other AR enthusiasts and creators and are looking forward to the next phases.
“We have other exciting things cooking in the oven that we would really love to share later on. But for now, the plan is to soft release the app in hopefully the next two weeks followed by a full release later on.”
Check out Massive Technologies website for updates on the launch of the app and to check out their first app, Piano 3D.
What is 360 VR Video?
360 video is a video that is recorded in all directions at the same time with multiple cameras. The videos are stitched together either internally in the camera or externally using special software.
It then forms one spherical video that is controlled by the viewer, enabling them to look up, down, right or left at their discretion.
Is 360 Video Virtual Reality?
I want you to be the judge.
Recommended MicroSD Card for use with the Fusion 360 camera
How are you watching your 360 VR Video?
Are you watching on your PC?
Is it Facebook? YouTube? Veer.TV ?
Or better still, are you using your Samsung Galaxy Smartphone with your Samsung Gear VR Headset?
Or are you using another smartphone with a Google Cardboard? Google Daydream?
Leave a comment below. Let us know what you like or don’t like. If you have seen a 360 VR video that you think is awesome, tell us, and we will feature it.
What type of VR Headset do you own? Or do you watch with your computer? Smartphone?
Do you have a VR camera? Ricoh Theta S? Ion360? Samsung 360 gear? Insta360?
And if you are looking for 360 Video Production Services, contact Al Caudullo [email protected]
With 37 years of Media experience, we can Deliver More for You in 360 Video. Click here to see what our customers say about our work
Utilizing state of the art technology, Al Caudullo, and his team has been delivering award-winning compelling productions spanning the globe. Exotic destinations blended with unique cultural experiences give viewers an experience like no other.
From vibrant American cities like San Francisco and Las Vegas to the isolated mountains of Mongolia. From ancient civilizations of Cambodia and Myanmar to the lush resorts of Southeast Asia and beyond. All from the comfort of your living room.
