The GitHub repository for this tool is available here.
Grasp Pose Estimation produces 6DoF gripper poses for robotic pick operations from sensor data. The module can consume RGB and depth streams (e.g., from an RGB-D camera) or precomputed depth/RGB frames. Internally the system can use GraspGen-style models to infer grasp candidates directly from visual+depth input without requiring a separate 6DoF object pose estimation step.
The final output is always a single or ranked set of 6DoF grasp poses:
- position: x, y, z (meters) — target gripper position in the sensor or robot frame.
- orientation: quaternion (x, y, z, w) — gripper orientation for approach and closure.
- Generate feasible pick grasps for objects visible to an RGB-D sensor.
- Rapid prototyping of vision-to-grasp pipelines where direct visual grasp proposals are preferred over full 6DoF object pose estimation.
- Integration with downstream motion planning and grasp execution stacks.
- The lighting conditions of the scene might be adjusted for more accurate bottle detections.
- The distance between the camera and target objects has to be adjusted based on the optimal working distance of the depth camera used.
- If poses seem offset, verify camera intrinsics and the camera-to-robot transform.
- If experiencing low-confidence outputs: increase image resolution or finetune the model for your object set.
- Use grasp-specific pre-grasp offsets and approach trajectories to ensure safe execution.
- The accuracy of pose estimation depends on the resolution and quality of depth and RGB sensor data.
- The grasp pose estimation is trained for a specific set of gripper types. Check the GraspGen repository for available gripper types and their dimensions.
- Thin, reflective, or transparent objects are challenging for depth-based methods.
- No explicit collision-aware trajectory planning is performed — generated grasps must be validated by the motion planner.
- Models trained on one set of objects/environments may not generalize without fine-tuning.