(oculus rift + leap motion) * unity3d = fun^2

okay, so i had some more time and actually tried to find out how to bring all the little hardware pieces together to form a working system. for the moment, it seems to me that the gaming engine unity3d is the best way to integrate most of the stuff i have although there are some nice things with point clouds i’d like to do with the asus xtion sensor (a derivation of primesense’s kinect that they developed together with microsoft).

but let’s start small, otherwise there’s just flat chaos. first of all, i’d like to show you what the current problem is and then describe how i got here. somehow i don’t find any information in the internet, so i think it’s worthwhile to write it up and get some people’s minds around it.

here’s what i built in about ten iterations with unity3d, leapmotion and the oculus rift:


here’s me manipulating the cubes with the leap motion on the table and the hmd on my head. i had to remove the leap motion from the hmd (see my last post taping it) during experimentation as i didn’t get any tracking first and later somehow the hands were mirrored, so i decided to simplify the setup and get it running on the table first.

here’s two things i learned in unity:

oculus leap unity3d

in the box (a prefab from the leap motion package), the rift camera has to be IN FRONT of the leap motion controller icon. the screenshot above shows the correct positioning from top view (leap motion icon is in the middle, right to the lamp icon). it may sound trivial but it took me two hours with many different settings (like the leap motion config menu also allows for changing orientations, etc).

second learning:

z axis not mirrored

the z-axis must not be mirrored. secondly, i scaled the sandbox by 40, this allows you to move freely in the box and your hands seem to be roughly in proportion to the cubes (default size from unity).

one thing i still don’t know: how can i actually get a screen recording while i’m playing? this is unity’s output (ok, ok i did 11 iterations):

unity file outputs

“direct to rift” works for me with my hmd, but on the pc screen there’s just a black window (as in my second video above), without “direct to rift” i see a live window, but the hmd isn’t showing anything. of course, the hmd is then used as a second screen on the pc, and i could move the window to that screen but then it won’t be full screen. and the second problem is that the software actually captures the mouse for controlling the navigation in the virtual space, so i can’t move the window as soon as i the animation actually starts… any ideas? haven’t found anything in the unity “build settings”.

here’s the software that i used:

i was inspired to do the cubes in the box by this video that is a brief tutorial into unity3d and leap motion integration (but has no oculus rift integration).


there’s a second very inspiring video that shows how to get unity running and install the oculus rift package for it (but no leap motion integration):

actually, a couple of things are easier now as you simply can register after downloading unity3d right at first start.

7 thoughts on “(oculus rift + leap motion) * unity3d = fun^2

  1. Si

    The folks at Leap Motion and Unity have done a surprisingly good job of making things easy for us.

    Here’s the process I went through to get OVR configured in Unity with hand tracking through a head mounted Leap Motion Controller:

    Step 1: Download all the files (see video link)
    Step 2: Import OVR and Leap Motion plugins by double clicking and then selecting Import. (Looks like OP already has, but for the sake of completeness…)
    Step 3: Drag the OVR Character Controller preset into the scene.
    Step 4 Drag the Hand Controller Leap Motion preset into the Right camera object inside the OVR Character Controller.
    Step 5: Scale the Hand Controller xyz to 5 and rotate the x and y axes by 270 and 180 degrees, respectively.

    That should at least get you to the fun stuff.

    Quickstart Guide: Oculus + Leap Integration: http://youtu.be/gMHtBae897I

    1. alecz

      That s a good trick but by doing so, when you want to import the new ‘quick switch’ plugin it doesnt work out.
      The plugins from the asset core VR do not work here, i get a black screen…

  2. Si

    While we’re on the subject, if anyone can walk me through running the leap’s camera data through the Rift so I can try making one of those slick AR demos, I’d be much obliged!

  3. LexAN

    I still can’t get it to display on screen and through the Rift EXCEPT inside Unity. There, I have two Game views open. One on the rift’s extended display, and the other on my flat screen. To get it to work, I had to uncheck the Use DirectX11 box in the player build settings.

    Good luck and Happy Rifting!

  4. Pingback: oculus rift, leap motion and unity integration: cubes and hands | Unofficial LEAP Motion tech demos & experiments collection

Leave a Reply

Your email address will not be published. Required fields are marked *

three + 3 =