Image & video data · United States

Hand manipulation video data with hand keypoints

Skilled hand work filmed up close on people’s own phones: assembling, sorting, folding cloth, preparing food, using tools and handling small parts, at real tables and workbenches. Reviewers add hand keypoints to every frame and label every reach, grasp, move and release as a step.

Specification

Footage
Close-up phone video from one or two angles (head- or chest-mounted, and a phone on a stand beside the work), at real tables, counters and workbenches
Tasks
Assembly, sorting and packing, folding cloth, food prep, using hand tools, plugging in cables and handling small parts
Hand keypoints
Finger and wrist keypoints for both hands on every frame, added and checked by reviewers
Annotation
Steps with start and end times, action, grasp type, object and hand (left, right or both), success or failure
People
Consenting adults, right- and left-handed, novices and people skilled at the task; faces kept out of frame or blurred
Quality check
Every step boundary, grasp label and keypoint track is checked by several trained reviewers

Example of the data

What you see, and the file you get.

Preview
pinch graspM3 screwright hand · 00:04.2
placescrew into bracket holeright hand · 00:05.8
power graspscrewdriverright hand · 00:07.1
actions.csvCSV · JSON · Excel
video_fileviewstartendhandactiongrasp
hands_0031.mp4head3.64.2rightreach_m3_screw
hands_0031.mp4head4.25.8rightpick_m3_screwpinch
hands_0031.mp4head5.86.9rightplace_in_bracket_holepinch
hands_0031.mp4head5.811.6lefthold_bracketpalm
hands_0031.mp4head7.111.6rightturn_screwdriverpower
hands_0031_side.mp4side7.111.6rightturn_screwdriverpower

Sample rows. Also as JSON or Excel.

Contact us to get the dataset

Tell us what you need. We walk you through the data, its formats and how it is checked on a short call.

Contact us

What it is used for

  • Robot dexterity

    Teach robot hands how people grasp, turn and place real objects.

  • Learning from demonstration

    Train manipulation policies on human video with every step and grasp labelled.

  • Hand pose estimation

    Train and test hand tracking on close-up, fast and partly hidden hands.

  • AR, VR and wearables

    Build camera-based hand tracking and gesture input that holds up on real tasks.

Questions

What comes with the video?

Hand keypoints on every frame and step labels with start and end times, grasp type, object and hand, all added by reviewers on the video’s own timeline.

Which tasks are recorded?

Assembly, packing, cloth folding, food prep, tool use, cables and small parts, by novices and by people skilled at the task. Your own task list and grasp taxonomy can be used.

Can I see samples first?

Yes. Request samples with the form on this page and we will send them with the full specification.

Tell us what your model needs.

Request samples and the full specification. We reply within a couple of business days.

Request samples