Repository navigation
Can Keypoints annotation work with Non-Humanoid models? #368
Description
Hi, I wonder if the keypoint annotation can be also apply to non-humanoid models? For example, a robot arm?
When I force to rig the robot arm with humanoid, it fails with message "Invalid Avatar Rig Configuration. Missing or invalid transform: Required human bone 'LeftLowerLeg' not found".
If I rig with "generic", I don't know how to continue.
Can you shed some light on how to achieve that?
Activity
ChumbaWambalala commentedon Sep 8, 2021
Hi, for custom joints, try the Joint Labels.
StevenBorkman commentedon Sep 9, 2021
Hi, and thanks for reaching out.
The KeypointLabeler can be used to do exactly this right now. The keypoint labeler works with a keypoint template to assign the points of interest that you are trying to label. By default, and the typical use case, the keypoints in the template are associated with an animation rig of a humanoid model. But this does not have to be the case.
The perception package, when cloned from github, has sample scenes included with it. The 'PoseSample' does exactly what you are looking for. After opening the 'PoseSample' scene, click on the Main Camera in the Hierarchy view. In the Inspector window for the main camera, you will see that the PerceptionCamera game object contains multiple KeypointLabelers (only one of them can be active at a time). The first KeypointLabeler, the one with the CocoKeypointTemplate is for the human model. The second KeypointLabeler, the one with the BoxKeypointTemplate, is the one used to track the corners of the box. By looking through this scene I think you will probably find all of the info you need, but here are the general basics to accomplishing this:
Create a new KeypointTemplate by right clicking in Assets and choosing 'Create->Perception->Keypoint Template'
Create entries in your new keypoint template for all of the keypoints that you want to track
Create game objects (no geometry is needed to be associated to them) for the points of interest that you want to track and put them in the scene where they should be. In our example these would be the all of the gameobjects under the KeypointCube for the corners of the box.
Add a JointLabel to each of these new gameobjects
Under the TemplateInformation for the JointLabel associate each label to a label found in your new KeypointTemplate
On your PerceptionCamera add your new KeypointTemplate to KeypointLabeler 'Active Template' field
I hope this helps. I am going to close out this issue now, but if this does not answer your questions, please feel free to re-open it.
Steve