Varshith Sreeramdass

Playing tennis alongside a person requires a robot partner that is both agile, reacting to fast-moving balls within fractions of a second, and robust, staying reliable over long rallies. This project designs hierarchical reinforcement learning policies for tennis that decouple the racket swing from ball placement, so that each level can be learned and reused independently.

Policies trained in Isaac Lab transfer zero-shot to a real robot arm, sustaining a juggling rally of 140 consecutive real-world ball strikes, ended only by a 2-minute trial cap. Transfer is stabilized by a Gaussian-process model of the racket's spatially varying restitution (manuscript in preparation). Ongoing work evaluates preliminary human-robot teaming through collaborative juggling with data-driven proxy teammates.

Diverse behavior policies are valuable in domains requiring quick test-time adaptation or personalized human-robot interaction. Human demonstrations provide rich information regarding task objectives and factors that govern individual behavior variations, which can be used to characterize useful diversity and learn diverse performant policies. However, we show that prior work that builds naive representations of demonstration heterogeneity fails in generating successful novel behaviors that generalize over behavior factors. We propose Guided Strategy Discovery (GSD), which introduces a novel diversity formulation based on a learned task-relevance measure that prioritizes behaviors exploring modeled latent factors. We empirically validate across three continuous control benchmarks for generalizing to in-distribution (interpolation) and out-of-distribution (extrapolation) factors that GSD outperforms baselines in novel behavior discovery by ~21%. Finally, we demonstrate that GSD can generalize striking behaviors for table tennis in a virtual testbed while leveraging human demonstrations collected in the real world.

Learning dexterous manipulation directly on real robots from scratch is considerably difficult due to the extensive real-robot deployment it requires, and sim-to-real is not straightforward due to limited simulation fidelity. This project investigated demonstration-based, offline, and residual reinforcement learning to make learning feasible for reorientation and manipulation of articulated objects on a multi-fingered robot hand.

For reorientation of objects for tool use (transitions among precision, tripod and power grasps), data-driven RL algorithms (DAPG, AWAC) learned from expert and suboptimal demonstrations; offline RL with online finetuning on the real robot achieved robustness to ±4cm and ±30° of noise in the initial object pose.

For opening a beverage can, a task requiring precise contacts and high torques, the project explored structured policies in two forms: (1) parameterized motion primitives as low-level policies, with gating policies operating in goal and duration spaces, and (2) residual policies that provide adjustments to actions from scripted base controllers. While parameterized primitives accelerated learning in simulation, they were limited in scenarios requiring quick, reactive behaviors. Residual policies showed significant improvements in sample efficiency and robustness, and transferred sim-to-real with robustness to 3mm of object initialization noise, demonstrated at IROS '22.