Feature - Hand Controllers + Simulation - CORE (#216)

* Update profile platform utility * Add and implement TryGetBounds(...) for hand controllers * Update submodule * Move hand visualizers to SDK * Remove mesh and joint visualizer setting * Update SDK submodule * Resolve merge issue * Update inspector namespace * Fix simulation created asset path and name * Fix UpdateController() not in basecontroleller interface * Base hand data provider cleanup * Remove all occurences of default hand controller * Moving state update to hand controller * Pipe hand data update though input system * Fix namespaces and assign asset icons * Remove obsolete hand specific visualization profile * Update submodules * Remove IMixedRealityHandControllerDataProvider since obsolete * Separate simulation and hands * Remove SimulationInput (using MixedRealityPose instead) * Add simulated controller type setting * Update SDK checkout * Cleanup simulated dp * Undo newline file changes * Update submoduules * Remove profile property from data provider and expose settings as properties * Merge SimulatedHandControllerState into SimulatedHandController * Move constructors to top * Rename HandUtils to HandUtilities * Remove HandRay class (returning in separate PR) * Cleanup utils and make extensions * Some more extnsion conversions * Remove not needed setter * Remove dead event definition * Remove setters in hand data snapshot * Add setter to hand data mesh property * Remove cached camera rerefence * Update SDK * Add hand mesh data constructors * Update Oculus submodule * Revert some meta files not related to hands changes * Revert not related change * Remove simulated controller controls from profile * Update interactions in base * Use interaction mapping for movement * Reset editor window options * remove hand joitn pose dictionary * Update SDK checkout * Pull update controller * Update submodules * Remove redundant joint count definition * Fix broken JointCount references * Minor BaseHandController cleanup * Fix copy paste error in windows api checker * Update submodules * Fix missing UpdateController definition issue * Fix controller list management * Update submodules * Update SDK checkout * Update SDK checkout * Update SDK * Define agnostic hand controller update * Fix namespaces and create mr hand controller mapping profile * Fix texture path for hand mapping profile * Update SDK checkout * Update WMR checkout * Update submodules * Update submodules * Update submodules * Upate simulated hand controller implementation to new interface * Make joint poses private * Add comment for simulation specifc update * Update submodules * Update sdk checkout * Update submodules * Move interaction updates to base hand * Remove default interactions defintions * Cleanup using * Update SDK checkout * Remove hand triangles shader * Update submodules * Update SDK checkout * Implement Gizmo utlity to draw wire capsule * Update SDK * Implement GetOrAddComponent extension * Implement bounds calculation for each TrackedHandBounds type * Update SDK checkout * Update SDK * Introduce shared hand profile configuraiton * Introduce base hand controller data provide to consume shared profile * Update simulation to make use of new shared hand data provider settings * Add label to Simulation specific in inspector * Update submodules * Update Oculus checkout * Introduce HandMeshingEnabled profile setting * Raise input source events in base hand controller * Update SDK * Delete now redundant GizmoUtility * Introduce "Hand Tracking System Profile" * Implement hand ray configuration * Update SDK * Update SDK * Update SDK * Update SDK and Oculus checkouts * Update Lumin / Oculus checkout * Update SDK * Remove SimulatedHandController button mappings from BaseHandController * Update simulation input actions in shc * Hand Tracking change requests (#476) * latest changes from dev plus change requests * pulled out new interface into it's own file * updated base controller data provider profile namespace added icon for the data provider as well * updated sdk checkout * Update submodule checkouts * Refactor simulation data provider relationship * Update SDK checkout * updated simulated hand data profile with required fields for IMixedRealityHandControllerDataProvider (#480) * removed ignored meta file * Fixed merge issues * Update copyright headers * Update checkouts * Return of the global hand tracking profile * Update SDK * Fix hands not showing up issue * Simplify ISimulatedHandControllerDataProvider retrieval * Remove BaseMixedRealityHandDataProviderProfile * Remove BaseHandControllerDataProvider * Add hand settings overrides to simulation profile * bumped minor version * cleaned up base hand controller (#496) * cleaned up base hand controller * missed a few * Add docs to explaoin what OnInspectorAdditionalGUI does in simulation * Update submodules * updated oculus * Update submodule * Remove hand ray type setting * Change requests/cleanup definitions (#497) * updated simulated hand controller pose type from class to struct removed allocators each frame when lerping between two poses misc formatting * updated sdk checkout * asset organization * updated wmr checkout * updated lumin commit * Moved stopwatch utility * updated submodules * Added BaseHandDatProvider and BaseHandDataProviderProfile * Add controller pose field to base hand * Fix simulation * Fix naming * Fix UWP running in editor when not configured to * Update submodules * moved constructor and changed access modifier for the constructor * updated sdk checkout * moved numerics extension and updated wmr checkout * updated wmr checkout * updated wmr checkout * updated Lumin checkout * updated Lumin checkout * fixed hand data provider inspectors * Update Lumin,SDK * Update Lumin * updated sdk checkout * Fix platform checks for iOS, Android and WebGL * apply global override if different from platform setting * updated lumin SDK * updated lumin checkout * updated oculus checkout * Update submodules * updated lumin checkout * change-request/hand-tracking (#501) * Added IMixedRealityControllerDataProvider to IMixedRealityController updated all the controller constructors to take a data provider parameter * a bit more cleanup and ensured controllers we being added to the data provider active controller registry * updated wmr checkout * Fix simulated hand controller instantiation * Fix UseSourcePose fallback * Update checkouts * Update SDK checkout * Fix missing data provider parameter for Unity Joystick controller * Rename OpenVR Oculus Touch profile and inspector to avoid confusion * Update checkouts * Update WMR checkout * reverting Oculus Touch rename * Revert visualizer.UseSourcePoseData change * updated sdk checkout * Update SDK * misc formatting * fixed constructor arg exception * updated sdk checkout * fixed merge conflict * fixed documentation * updated simulated hand controller to use data provider reference * Fix pose change raised twice per update * Fix some docs in visualization profile * Update SDK submodule * updated submodules Co-authored-by: Stephen Hodgson <StephenHodgson@users.noreply.github.com> Co-authored-by: Simon (Darkside) Jackson <darkside@zenithmoon.com> Co-authored-by: Stephen Hodgson <hodgson.designs@gmail.com>
XRTK · Apr 7, 2020 · 9ac2ec1 · 9ac2ec1
1 parent bfd7cf9
commit 9ac2ec1
Show file tree

Hide file tree

Showing 111 changed files with 4,540 additions and 27 deletions.
diff --git a/Definitions/Controllers/Hands.meta b/Definitions/Controllers/Hands.meta
diff --git a/Definitions/Controllers/Hands/BaseHandControllerDataProviderProfile.cs b/Definitions/Controllers/Hands/BaseHandControllerDataProviderProfile.cs
@@ -0,0 +1,53 @@
+// Copyright (c) XRTK. All rights reserved.
+// Licensed under the MIT License. See LICENSE in the project root for license information.
+
+using UnityEngine;
+
+namespace XRTK.Definitions.Controllers.Hands
+{
+    /// <summary>
+    /// Provides additional configuration options for hand data providers.
+    /// </summary>
+    public abstract class BaseHandControllerDataProviderProfile : BaseMixedRealityControllerDataProviderProfile
+    {
+        [Header("General Settings")]
+
+        [SerializeField]
+        [Tooltip("If set, hand mesh data will be read and available for visualization. Disable for optimized performance.")]
+        private bool handMeshingEnabled = false;
+
+        /// <summary>
+        /// If set, hand mesh data will be read and available for visualization. Disable for optimized performance.
+        /// </summary>
+        public bool HandMeshingEnabled => handMeshingEnabled;
+
+        [Header("Hand Physics")]
+
+        [SerializeField]
+        [Tooltip("If set, hands will be setup with colliders and a rigidbody to work with Unity's physics system.")]
+        private bool handPhysicsEnabled = false;
+
+        /// <summary>
+        /// If set, hands will be setup with colliders and a rigidbody to work with Unity's physics system.
+        /// </summary>
+        public bool HandPhysicsEnabled => handPhysicsEnabled;
+
+        [SerializeField]
+        [Tooltip("If set, hand colliders will be setup as triggers.")]
+        private bool useTriggers = false;
+
+        /// <summary>
+        /// If set, hand colliders will be setup as triggers.
+        /// </summary>
+        public bool UseTriggers => useTriggers;
+
+        [SerializeField]
+        [Tooltip("Set the bounds mode to use for calculating hand bounds.")]
+        private HandBoundsMode boundsMode = HandBoundsMode.Hand;
+
+        /// <summary>
+        /// Set the bounds mode to use for calculating hand bounds.
+        /// </summary>
+        public HandBoundsMode BoundsMode => boundsMode;
+    }
+}
diff --git a/Definitions/Controllers/Hands/BaseHandControllerDataProviderProfile.cs.meta b/Definitions/Controllers/Hands/BaseHandControllerDataProviderProfile.cs.meta
diff --git a/Definitions/Controllers/Hands/HandBoundsMode.cs b/Definitions/Controllers/Hands/HandBoundsMode.cs
@@ -0,0 +1,22 @@
+// Copyright (c) XRTK. All rights reserved.
+// Licensed under the MIT License. See LICENSE in the project root for license information.
+
+namespace XRTK.Definitions.Controllers.Hands
+{
+    /// <summary>
+    /// Available hand bounds modes.
+    /// </summary>
+    public enum HandBoundsMode
+    {
+        /// <summary>
+        /// Hand bounds mode will only calculate a single bounding
+        /// box encapsulating the whole hand.
+        /// </summary>
+        Hand,
+        /// <summary>
+        /// Fingers bounds mode will create precise bounds for each finger
+        /// and the palm to allow for more precise interactions.
+        /// </summary>
+        Fingers
+    }
+}
diff --git a/Definitions/Controllers/Hands/HandBoundsMode.cs.meta b/Definitions/Controllers/Hands/HandBoundsMode.cs.meta
diff --git a/Definitions/Controllers/Hands/HandData.cs b/Definitions/Controllers/Hands/HandData.cs
@@ -0,0 +1,40 @@
+// Copyright (c) XRTK. All rights reserved.
+// Licensed under the MIT License. See LICENSE in the project root for license information.
+
+using System;
+using XRTK.Definitions.Utilities;
+
+namespace XRTK.Definitions.Controllers.Hands
+{
+    /// <summary>
+    /// Snapshot of hand data.
+    /// </summary>
+    [Serializable]
+    public class HandData
+    {
+        /// <summary>
+        /// Gets the total count of joints the hand data supports.
+        /// </summary>
+        public static readonly int JointCount = Enum.GetNames(typeof(TrackedHandJoint)).Length;
+
+        /// <summary>
+        /// Timestamp of hand data, as FileTime, e.g. <see cref="DateTime.UtcNow"/>
+        /// </summary>
+        public long TimeStamp { get; set; } = 0;
+
+        /// <summary>
+        /// Is the hand currently being tracked by the system?
+        /// </summary>
+        public bool IsTracked { get; set; } = false;
+
+        /// <summary>
+        /// Pose information for each hand joint.
+        /// </summary>
+        public MixedRealityPose[] Joints { get; } = new MixedRealityPose[JointCount];
+
+        /// <summary>
+        /// Mesh information of the hand.
+        /// </summary>
+        public HandMeshData Mesh { get; set; } = new HandMeshData();
+    }
+}
diff --git a/Definitions/Controllers/Hands/HandData.cs.meta b/Definitions/Controllers/Hands/HandData.cs.meta
diff --git a/Definitions/Controllers/Hands/HandMeshData.cs b/Definitions/Controllers/Hands/HandMeshData.cs
@@ -0,0 +1,67 @@
+// Copyright (c) XRTK. All rights reserved.
+// Licensed under the MIT License. See LICENSE in the project root for license information.
+
+using UnityEngine;
+
+namespace XRTK.Definitions.Controllers.Hands
+{
+    /// <summary>
+    /// Snapshot of hand mesh information.
+    /// </summary>
+    public struct HandMeshData
+    {
+        /// <summary>
+        /// Constructs a new hand mesh data snapshot.
+        /// </summary>
+        /// <param name="vertices"></param>
+        /// <param name="triangles"></param>
+        /// <param name="normals"></param>
+        /// <param name="uvs"></param>
+        /// <param name="position"></param>
+        /// <param name="rotation"></param>
+        public HandMeshData(Vector3[] vertices, int[] triangles, Vector3[] normals, Vector2[] uvs, Vector3 position, Quaternion rotation)
+        {
+            Vertices = vertices;
+            Triangles = triangles;
+            Normals = normals;
+            Uvs = uvs;
+            Position = position;
+            Rotation = rotation;
+        }
+
+        /// <summary>
+        /// The vertices of the hand mesh in the initial coordinate system.
+        /// </summary>
+        public Vector3[] Vertices { get; }
+
+        /// <summary>
+        /// Mesh triangle indices.
+        /// </summary>
+        public int[] Triangles { get; }
+
+        /// <summary>
+        /// Hand mesh normals, in initial coordinate system.
+        /// </summary>
+        public Vector3[] Normals { get; }
+
+        /// <summary>
+        /// UV mapping of the hand. 
+        /// </summary>
+        public Vector2[] Uvs { get; }
+
+        /// <summary>
+        /// Translation to apply to mesh to go from initial coordinates to world coordinates.
+        /// </summary>
+        public Vector3 Position { get; }
+
+        /// <summary>
+        /// Rotation to apply to mesh to go from initial coordinates to world coordinates.
+        /// </summary>
+        public Quaternion Rotation { get; }
+
+        /// <summary>
+        /// Returns true, if the mesh data is empty.
+        /// </summary>
+        public bool Empty => Vertices == null || Vertices.Length == 0;
+    }
+}
diff --git a/Definitions/Controllers/Hands/HandMeshData.cs.meta b/Definitions/Controllers/Hands/HandMeshData.cs.meta
diff --git a/Definitions/Controllers/Hands/MixedRealityHandControllerMappingProfile.cs b/Definitions/Controllers/Hands/MixedRealityHandControllerMappingProfile.cs
@@ -0,0 +1,36 @@
+// Copyright (c) XRTK. All rights reserved.
+// Licensed under the MIT License. See LICENSE in the project root for license information.
+
+using UnityEngine;
+using XRTK.Definitions.Devices;
+using XRTK.Definitions.Utilities;
+using XRTK.Providers.Controllers;
+using XRTK.Providers.Controllers.Hands;
+
+namespace XRTK.Definitions.Controllers.Hands
+{
+    [CreateAssetMenu(menuName = "Mixed Reality Toolkit/Input System/Controller Mappings/Hand Controller Mapping Profile", fileName = "HandControllerMappingProfile")]
+    public class MixedRealityHandControllerMappingProfile : BaseMixedRealityControllerMappingProfile
+    {
+        /// <inheritdoc />
+        public override SupportedControllerType ControllerType => SupportedControllerType.Hand;
+
+        /// <inheritdoc />
+        public override string TexturePath => $"{base.TexturePath}Hand";
+
+        /// <inheritdoc />
+        protected override void Awake()
+        {
+            if (!HasSetupDefaults)
+            {
+                ControllerMappings = new[]
+                {
+                    new MixedRealityControllerMapping("Hand Controller Left", typeof(MixedRealityHandController), Handedness.Left),
+                    new MixedRealityControllerMapping("Hand Controller Right", typeof(MixedRealityHandController), Handedness.Right)
+                };
+            }
+
+            base.Awake();
+        }
+    }
+}
diff --git a/Definitions/Controllers/Hands/MixedRealityHandControllerMappingProfile.cs.meta b/Definitions/Controllers/Hands/MixedRealityHandControllerMappingProfile.cs.meta
diff --git a/Definitions/Controllers/Hands/RecordedHandJoint.cs b/Definitions/Controllers/Hands/RecordedHandJoint.cs
@@ -0,0 +1,79 @@
+// Copyright (c) XRTK. All rights reserved.
+// Licensed under the MIT License. See LICENSE in the project root for license information.
+
+using System;
+using UnityEngine;
+using XRTK.Definitions.Utilities;
+
+namespace XRTK.Definitions.Controllers.Hands
+{
+    /// <summary>
+    /// Unity's <see cref="JsonUtility"/> "currently" (stated in 2011) does not support top-level arrays.
+    /// But hey, it's on their "future" road-map. That's why we need a wrapper definition around or items.
+    /// </summary>
+    /// <remarks>The <see cref="JsonUtility"/> also does not support properties, so we gotta use fields. Oh and it also
+    /// does not support auto mapping of JSON naming "items" to C# naming "Items".</remarks>
+    [Serializable]
+    public class RecordedHandJoints
+    {
+        /// <summary>
+        /// Gets the joints recorded in this data set.
+        /// </summary>
+        public RecordedHandJoint[] items;
+    }
+
+    /// <summary>
+    /// A single recorded hand joint's information that may be used to restore the joint pose for simulation.
+    /// </summary>
+    [Serializable]
+    public struct RecordedHandJoint
+    {
+        /// <summary>
+        /// Constructs a new joint record.
+        /// </summary>
+        /// <param name="joint"></param>
+        /// <param name="pose"></param>
+        public RecordedHandJoint(TrackedHandJoint joint, MixedRealityPose pose)
+        {
+            this.joint = JointNames[(int)joint];
+            this.pose = pose;
+        }
+
+        private static readonly string[] JointNames = Enum.GetNames(typeof(TrackedHandJoint));
+
+        /// <summary>
+        /// Name of the joint recorded. Rather use <see cref="JointIndex"/> below for convenience.
+        /// </summary>
+        public string joint;
+
+        /// <summary>
+        /// The recorded pose.
+        /// </summary>
+        public MixedRealityPose pose;
+
+        /// <summary>
+        /// Gets the <see cref="TrackedHandJoint"/> this record represents.
+        /// </summary>
+        public TrackedHandJoint JointIndex
+        {
+            get
+            {
+                int nameIndex = Array.FindIndex(JointNames, IsJointName);
+
+                if (nameIndex < 0)
+                {
+                    Debug.LogError($"Joint name {joint} not in {nameof(TrackedHandJoint)} enum");
+                    return TrackedHandJoint.None;
+                }
+
+                return (TrackedHandJoint)nameIndex;
+            }
+            set => joint = JointNames[(int)value];
+        }
+
+        private bool IsJointName(string s)
+        {
+            return string.Equals(s, joint);
+        }
+    }
+}
diff --git a/Definitions/Controllers/Hands/RecordedHandJoint.cs.meta b/Definitions/Controllers/Hands/RecordedHandJoint.cs.meta