We are modeling an HRI scenario in virtual reality using Unity and the Oculus Rift. Our problem is, though, that Unity only allows objects to be either kinematic or physics. This means, either we have a nicely running inverse kinematic model of a 6-DOF industrial robot, but struggle with physical-virtual interaction with objects, or we have a physically correct behavior of the robotic parts, but lose the inverse kinematic model.
Does anyone of you also work with Unity as a platform for HRI VR and came across these issues, too?
Thank you very much in advance,
Jonas
Dear Jonas,
I suggest to you alinks and attached files in topics.
-Evaluation of Human Robot Collaboration in Masonry Work Using ...
https://www.researchgate.net/.../287800928_Evaluation_of_Huma...
-Projects | GVU Center
gvu.gatech.edu/research/projects
Best regards
Thank you very much for your answers.
@ Jose: Yes, you are right, that probably we have to lose one of the above-mentioned features. But still, I don't get it why it is possible with "real robots". There has to be a neat way to implement the same behavioral features in Unity.
You can't really cut corners with this.
And Unity doesn't have a goal in developing the research and physics part of their engine because their main objective is to have a platform oriented for mobile Games.
So, you either compute your inverse kinematics or you use the animation.
But what I have seen in the past year was a new library that has been developed by other people, called APEngine (Advanced Physics Engine) which will allow you to have better physics in you HRI. This was a problem in my application but I don't know if it will solve yours.
Hi Jonas,
As Elias says, you can use a virtual industrial manipulator in Unity as a "kinematic" object, i.e. an object still able to collide with other surrounding objects but not affected in its movements by these collisions. If you think about it, this is how real industrial manipulators work. They are designed in a very robust way (high gear ratios, powerful servos, etc.) to react to and reject external disturbances. If a real manipulator is forced out of its trajectory by a (catastrophic) event you can be sure that either something really bad will happen to the system or (if it's cleverly designed) internal safety systems will intervene to halt or shut down the robot. What I am trying to say is that it is probably pointless to simulate an industrial manipulator in Unity as a physical object. This does not apply obviously for compliant and "soft" manipulators, but in this case you probably need to go down to a much more refined simulation level, where a proper modelling of (compliant) robot actuators is involved. In this case the IK would be used to provide only a reference for actuators trajectory and how the two things (physics and IK would interact) would interact is part of the "magics" that Elias was talking about :-) .
We have here two examples using VR interaction as well as a human in the loop example in combination with planning simulation with robot.
Article Seven Levels of Detail to structure use cases and interactio...
Hi Jonas,
I'm not quite familiar with the Unity-specific limitations, but have you considered controlling the robot from "outside" using a software-in-the-loop approach with a virtual robot controller or robot controller simulation module? Basically forget about modeling the kinematics (and dynamics etc.) in Unity, but mirror the internal robot model state from the VRC. We used that for simulation bin picking processes in Siemens NX MCD, also with physics enabled, and it worked just fine Conference Paper Virtual Commissioning of 6 DoF Pose Estimation and Robotic B...
. Also, you get a much more realistic robot behavior, since it is controlled by the actual controller and program. However, you will not get a physical feedback to the real controller (joint torque etc.). If you want to imitate a collision detection through the robot for HRI topics, I would either add simulated force sensors to the robot and tool (if Unity supports that), or just use a collision-based setting of a variable/signal that triggers as a safety stop on the robot. See also here:Conference Paper A system for human-in-the-loop simulation of industrial coll...
Greetings
Max
Have you added colliders to your simulation? It seems that it is the only thing you are missing.
https://docs.unity3d.com/Manual/CollidersOverview.html
https://www.youtube.com/watch?v=bh9ArKrPY8w
- Badges
- Science topic
- Similar topics
- Correction