No Eyes, No Problem

Benchmark

Each row is one policy, shown in simulation and on the real hand. Every policy was deployed zero-shot, with no training or tuning on the hardware. Each hardware clip is one of the 10 trials recorded for that policy.

^* Benchmark results come from 10 separate hardware trials per policy. The 112.5-rotation demonstration run at the top of the page is not one of them and is not included in these statistics.

Evaluation

Hardware numbers are the mean ± standard deviation over 10 trials, with the min–max range in small type. Each trial started with both balls placed by hand and ran until a ball left the hand, with no time limit. Simulation numbers come from 768 ten-second episodes across three training seeds. A dash means the metric wasn't measured or reported.

Full rotations: complete 360° rotations before either ball leaves the hand; one full rotation is two successful ball exchanges. Time to drop (t_drop): time from both balls being seated until either ball leaves the hand. Speed: R₁₀ / min(10, t_drop), where R₁₀ is the number of full rotations in the first 10 s (in simulation, t_drop is the episode survival time). η (exchange efficiency): 2R₁₀ / (f_gaitT), the fraction of executed gait cycles that complete a ball exchange, with T = min(10, t_drop); η ≤ 1. Drop rate: the fraction of episodes (simulation) or trials (hardware) in which a ball leaves the hand within 10 s.

^* Benchmark results come from 10 separate hardware trials per policy. The 112.5-rotation demonstration run at the top of the page is not one of them and is not included in these statistics.

^† Nagabandi et al. report ~54% success at a single 180° half-rotation within 10 s (read as ~0.5 full rotations), inferred from a video demonstration. ^‡ Yuan et al. report ~3 full rotations within 10 s (~0.30 rot/s). ^§ Yang et al. show a single qualitative 180° half-rotation on hardware without object vision. None of them report trial-to-trial variance, time to drop, or how long rotation can be sustained.

Key findings

• Touch improves exchange completion. With tactile input zeroed at deployment (C3-TacOff), the policy keeps almost the same gait frequency as C3 (~8.1 vs 7.9 cycles), but exchange efficiency drops from η = 0.90 to 0.58. The gait is still executed; many cycles simply fail to produce an exchange. The proprioception-only policy runs one of the fastest gaits yet has the lowest η (0.31), and its joint trajectories are nearly identical with and without balls in the hand. It is replaying a learned cycle rather than adapting to the object.
• Touch sustains the task. C3 completes ~7× as many rotations as C3-TacOff (34.75 vs 5.0) and stays up ~5× longer (94.8 s vs 19.0 s). Its joint trajectories vary in amplitude from cycle to cycle, a sign of online correction. In one trial the thumb failed to push a ball far enough toward the ring finger; it held the incoming ball while the index and middle fingers repositioned the other, and the gait resumed. In simulation, removing touch raises the drop rate from 2% to 32%.

• Simulated performance does not predict hardware performance. C1 has the highest simulated speed (0.96 rot/s) but manages only 0.35 full rotations on hardware, with a 70% drop rate. Its large, rapid commands exceed what the tendon-driven hand can actually track.

• SlewDR makes motion trackable; STAT removes reliance on single taxels. Rate-limiting actions during training (C2) produces smoother commands and brings simulated and hardware speeds closer, but every hardware trial still ends in a drop. When touch is removed from C2 in simulation, its drop rate rises to nearly 60%, showing heavy dependence on continuous tactile input. Adding STAT (C3) raises hardware performance from 0.5 to 34.75 full rotations, and removing touch from C3 raises its simulated drop rate only to 32%. The learned gait becomes the primary driver of motion, with touch providing local correction of the ball configuration.

• Sparse, approximate touch is enough. On hardware, tactile activation shifts toward the proximal middle- and ring-finger sensors compared with simulation, yet transfer still succeeds. We found that an exact sensor-by-sensor correspondence between simulation and hardware is not required.

Hardware

Tactile sensing is expensive, and covering an entire robot hand with commercial tactile sensors costs far more than most labs can justify. Instead, we built our own low-cost tactile layer from 12 force-sensitive resistors (FSRs), which cost only a few dollars each. An FSR is a thin polymer film whose electrical resistance drops as force is applied to it. We read each one as a voltage and threshold it into a binary contact signal, using the hysteresis calibration described in the Method section.

The FSRs sit on the palm, knuckles and phalanges of the Shadow Dexterous Hand Lite. Together with the four BioTac sensors already on the fingertips, they give the policy 16 binary contact channels. We chose the sensor sites from how often each link was contacted in simulation: most often at the knuckle (MCP) regions, then the middle-joint (PIP) regions and the palm. The simulation model mirrors this layout exactly.

Overview

Humans manipulate objects in-hand through touch alone with remarkable ease, but sustained real-world manipulation without vision remains an open challenge for robots. Reinforcement learning can produce sophisticated blind manipulation policies in simulation, yet transferring them to a real dexterous hand is hindered by continuously changing contacts, motion-induced drift in tactile signals, and the gap between simulated and physical dynamics.

We study this problem through Baoding ball rotation on the four-fingered Shadow Dexterous Hand Lite. It is a harder test of in-hand dexterity than single-object reorientation: two free objects circulate simultaneously, interacting with each other and with the fingers, and their phase relationship must be maintained without being observed. The policy receives only proprioception and 16 binary contact signals. Zero-shot transfer is achieved through a sensor-aligned, coupling-aware simulation embodiment, per-channel tactile calibration, and two novel episodic domain randomisations: slew-rate action limits (SlewDR) and Stuck-AT Taxels (STAT).

On the physical hand, the policy completes 225 consecutive 180° rotations (112.5 full rotations) at ~0.36 rot/s in a single continuous trial, without any vision or object-state information. This is the strongest reported hardware result for Baoding ball manipulation in both rotation count and speed, regardless of sensing modality. We further quantify the contribution of tactile sensing relative to proprioception alone, and identify the hand regions where tactile information matters most.

Method

• Blind agent. Each observation frame holds 52 proprioceptive values (joint position, joint velocity, position error and previous action for the 13 actuators) and 16 binary tactile values; four frames are stacked into a 272-dimensional observation. The action is a 13-dimensional joint-position target issued at 60 Hz. Policies are trained with PPO in Isaac Lab on 10 s episodes (600 control steps), using the RoTO reward: a dense term for keeping the balls in the hand and a sparse term for each completed exchange.
• Coupling-aware embodiment. The PIP and DIP joints of the index, middle and ring fingers are mechanically coupled, leaving 13 independently actuated joints. On hardware the coupling is sequential: during flexion the PIP moves first and the DIP only begins once the PIP reaches ~100°, with the order reversed on extension. The default Isaac Lab asset instead drives the DIP as a fixed multiple of the PIP. We apply the sequential coupling explicitly at every environment step, and add collision bodies at the physical sensor locations so simulated tactile coverage matches the hardware.
• Physical randomisation. Ball mass is sampled from 45–100 g, and static and dynamic friction are randomised independently for each ball and for the distal, middle, proximal, knuckle and palm regions of the hand, with higher friction on the silicone-covered BioTac fingertips.
• Slew-rate randomisation (SlewDR). Even with soft gains, the policy commanded joint motion faster than the hardware could execute, likely due to tendon friction and reflected motor inertia that the simulation does not model. We limit the change in commanded position per step, q_t + clip(q^des_t − q_t, ±s q̇^max Δt), resampling the scale s ~ U(0.3, 1.0) at each episode reset in simulation and holding it fixed on hardware.
• Stuck-AT Taxels (STAT). Policies trained on clean tactile input over-rely on individual channels, which hurts transfer when those channels misbehave. At each reset, k ~ U{0,…,6} of the 12 FSR channels are selected and each is frozen at a random value v ~ U{0,1} for the whole episode, while the rest report the true contact state. This acts like episode-level dropout, so the policy cannot trust any single taxel.
• Hysteresis tactile calibration. Static FSR thresholds failed because the baseline reading shifts with hand pose and wire movement, even with an empty hand. Each channel gets its own upper and lower thresholds from its empty-hand baseline (τ^hi = P_99.5 + 5; τ^lo just above the median, or 3 below τ^hi). Contact switches on above τ^hi and off below τ^lo, like a Schmitt trigger.

New insights

In this section we discuss where our current system falls short, what we think could push these results further, and what could make blind in-hand dexterity more common, especially for complex tasks like manipulating two objects at once.

What limits the hand today

• FSR latency caps the speed. The FSRs register a contact slightly after it actually happens, and even a firm press is not always picked up cleanly. That delay makes the hand less reactive, so the motion has to be slowed down on hardware to give the sensors time to catch up. Of our policies, the proprioception-only one runs the fastest gait, since it doesn't wait on touch at all. We believe lower-latency sensors would let a touch-driven policy run even faster gaits with better recovery.
• Going faster trades endurance for speed. When we sped up the hand's motion, rotations came noticeably quicker, but time to drop fell sharply and so did the total number of rotations: the fastest run we tested that still showed sustained motion reached only 10 full rotations. A faint contact still registers near the proximal and middle phalanges of the index and middle fingers, where a ball waits to be pushed by the thumb, and at the thumb tip. But when a ball brushes quickly across the sensors near the middle and ring fingers, it usually isn't registered at all. The policy would have to wait out the sensor latency for a contact that rarely arrives, and it loses the ball. This is why the hand has to move slowly.
• Smooth balls slip past the sensors. Our balls are very smooth, so friction works against us: a ball can slide over an FSR without pressing it hard enough to register a contact. With no contact signal, the policy has nothing to react to and no chance to recover before the ball falls. Again, the practical fix on hardware was to slow the hand down.
• Slip between the fingers goes unseen. A human hand has touch receptors everywhere; ours has 12 FSRs across the palm and phalanges. When a ball gets wedged between two phalanges, it often touches no sensor at all, and the policy has to act on its memory of where the ball might be. Proprioception doesn't fill the gap either: the balls are light and the joints, even at reduced stiffness, are too stiff for a ball's contact to show up in the joint readings.

What we would build next

• Put the sensing where contact happens. Not every contact site matters equally. On Baoding balls, most contact lands on the proximal and middle phalanges and around the knuckles (figure below). This task would benefit most from denser sensing in exactly those regions.
• Low-latency sensors. We already know binary contact works: simple on/off touch was enough for the policy to infer roughly where the balls are. The open question is whether that contact can be sensed in real time, fast enough to catch a ball brushing past. There aren't many cheap alternatives to FSRs that do this today.
• Discretised touch for harder tasks. For more complex manipulation, we suggest discretised tactile readings, a few pressure ranges rather than a single on/off bit. We expect this to give the policy more information than binary contact while staying less noisy, over long horizons, than a continuous signal.

BibTeX

@article{deshmukh2026no,
  title={No Eyes, No Problem: What Does Touch Contribute to Blind Baoding Ball Manipulation?},
  author={Deshmukh, Ayush and Agarwal, Nalin and Miller, Elle and Vijayakumar, Sethu},
  journal={arXiv preprint arXiv:2610.23438},
  year={2026}
}