poster · WHERE 2026, Trento

Proprioceptive quadruped locomotion without gait priors, stage one: a learnable representation shaped by an auxiliary prediction objective organises the locomotion manifold

Three stages towards adaptive and robust locomotion under interoceptive rewards: (1) learnable feature extraction, (2) learnable discovery of motion primitives and (3) the emergence of behaviour.

Janosch Bajorath · Malte Schilling

Poster Lab

What the controller does – Forty robots are initialised in uniformly random orientations. Each recovers to a standing posture and tracks its own velocity command, with walk, trot or bound selected by the commanded speed, under the trained domain randomisation and observation noise. Shown for the RECON + BCE encoder.

Abstract

Learned quadruped locomotion is largely installed rather than acquired. Gait rewards [1], reference motions [2] and pattern generators [3, 4] restrict the state and action spaces to a solution known in advance. We assume that this restriction narrows the search for the locomotion manifold, the low-dimensional subspace on which locomotion takes place. The aim is a learner that discovers this manifold under interoceptive rewards and learns to traverse it. The behaviour that emerges should be adaptive and robust to changes in command and domain [5].

In the absence of such priors, a reward derived from interoceptive quantities provides little information about behaviour, and learning converges to a single gait that is viable across all commanded velocities. The structure the policy requires must then be supplied by the representation, through the features it extracts from the observations [6] and the information it discards. The requirements on such a representation can only be established where the behaviour exists and is labelled. Stage one therefore proceeds top-down. Gaits and clock remain prescribed, and the gradient that shapes the representation is the factor varied. We examine whether an auxiliary prediction objective organises a latent representation that generalises, and whether the policy’s control depends on it.

Contributions

Without gait priors, locomotion must be read and realised from the latent alone.

State–Representation Coupling. The gradient of an auxiliary prediction objective organises the latent by behaviour and keeps this order separate from the domain.
Representation–Action Coupling. With the organised latent, the policy adapts to the domain through the components the behaviour does not occupy, and the latent closes the control loop as its feedback signal.
Environment–Representation Coupling. Withstanding a disturbance requires the latent’s deviation from its mean cycle. With the organised latent, only the components the behaviour does not occupy are required.

Setup

Architecture: a proprioception encoder compresses the observation history into the latent, which the control policy maps with the command to joint targets. A dynamics decoder and a contact head add auxiliary gradients to the encoder. — **Shaping Gradients** – Squiggles trace the gradients that shape the encoder. PPO reaches it through the policy, while RECON and BCE reach it only with the auxiliary objective. The encoder compresses the noisy history $s^{p}_{[t-30:t]}$ into the latent representation $z_t \in \mathbb{R}^{16}$ . The control policy maps $z_t$ and the command $\text{cmd}_t$ to joint targets $a_t$ . The command holds the base twist $v^{\text{base}} \in \mathbb{R}^{3}$ , the phase $\varphi^{\text{foot}}_t \in \mathbb{R}^{4}$ of each foot and the PD gains $k^{\text{joint}}_p, k^{\text{joint}}_d \in \mathbb{R}^{12}$ of each joint. The commanded speed selects the gait, stepping frequency $f$ and duty $d$ , which advance and offset each foot’s phase. The auxiliary objective adds a KL regulariser on $z_t$ , a dynamics decoder predicting $s^{p}$ and $v^{\text{base}}$ and a contact head predicting $c^{\text{foot}}$ .

Robot and control

Robot: Unitree Go1, 12 joints, simulated in Isaac Lab
Control: 50 Hz policy, 200 Hz physics and PD loop
Action: joint position offsets about the nominal pose
Command: forward −1.4…2.5 m/s, lateral ±0.9 m/s, yaw ±2.0 rad/s

Representation and policy

History: 11 samples spanning the last 30 control steps (0.6 s), noisy
Encoder: MLP 512–256–128, ELU, to a 16-dimensional Gaussian latent
Policy: MLP 256–256–128, ELU
Decoders: dynamics 128–256–512, contact 128–128
KL: weight 0.5, free bits 0.1

Training and evaluation

Environments: 4096 in parallel, flat ground
Updates: 7500 PPO updates, 98 304 samples each
Evaluated: checkpoint 7250, deterministic actions
Randomised: friction, restitution, masses, centre of mass, PD gains, motor torque envelope

Gait	Commanded speed	Stride frequency	Duty
Stand	0	–	1.00
Walk	below 0.7 m/s	1.5–2.3 Hz	0.75
Trot	0.7–1.7 m/s	2.3–3.4 Hz	0.50
Bound	above 1.7 m/s	3.4 Hz	0.35

Randomisation ranges and PPO settings

Static friction 1.0–1.25, dynamic friction 0.5–1.0, restitution 0–0.4. Trunk mass −1…+2 kg, other links ×0.95–1.05. Centre of mass ±0.10 m (x), ±0.05 m (y, z). PD stiffness and damping ×0.9–1.1 per robot. Motor torque envelope ×0.9–1.0 per robot. Observation noise is uniform and additive per channel, for example ±0.15 rad/s on the angular velocity and ±0.05 on the projected gravity.

PPO with discount 0.992, $\lambda$ = 0.9, ratio clip 0.2, entropy weight 0.012, four mini-batches, two actor and four critic update steps per update. Adam, actor learning rate $10^{-4}$ adapted to a KL target of 0.015, critic learning rate $8 \times 10^{-4}$ .

State–Representation Coupling

Reading locomotion from the latent requires organisation at two levels. The latent must separate the commanded gaits and speeds (H1), and this order must persist when the domain changes (H2). The encoders are trained with PPO alone or with RECON + BCE added. They are evaluated on steady-state recordings of stand and of walk, trot and bound at five forward speeds each, under nominal dynamics and under domain randomisation with observation noise.

H1 · Behaviour

The latent separates the commanded gaits and speeds.

The commanded behaviours – Stand and the middle of the walk, trot and bound speed bands under nominal dynamics, first at full speed, then at half speed. These are the steady-state conditions the latent is recorded on.

PCA of the latent for the PPO encoder and the RECON + BCE + PPO encoder, coloured by gait and forward speed. — **H1 Behaviour** – PCA of latent $z_t$ per encoder (% variance), each panel on its own scale. Both gradients separate the commanded gaits and speeds but impose different geometries on the behavioural subspace. PPO shapes each gait into a displaced, flattened loop and encodes speed by position. Added RECON + BCE gradients organise the subspace into thin concentric rings centred on the stand condition, with speed as radius and walk and trot orthogonal to bound.

H2 · Domain

This order persists when the body changes.

One command, eight bodies – One trot command on eight robots drawn from the trained domain randomisation, with observation noise. The domain draws alone separate the robots.

Deviation of the latent from its condition mean in the first two principal components, and variance of behaviour and domain per principal component, for both encoders. — **H2 Domain** – Deviation of the latent from its condition mean and variance of behaviour and domain per component, under domain randomisation. With RECON + BCE the domain leaves the gait coordinates nearly unchanged. Relative to behaviour, the variance it induces is more than twenty times smaller than under PPO. The domain is mainly encoded in components 9–16, while behaviour occupies components 1–8.
$D$ is the dispersion of a command condition (gait and speed) under nominal dynamics, the mean distance of its latent states from their centroid. Deviations are in units of $D$ , variances in $D^2$ .

Representation–Action Coupling

Realising locomotion from the latent requires two properties of the policy. It must draw its adaptation to behaviour and domain from the latent (H1), and it must act on the latent in closed loop (H2). At present the command prescribes the gait, while the latent reports the realised motion. H1 relates each principal component to the joint targets under domain randomisation, with totals per condition in the strips. H2 advances the clock by a quarter cycle during steady locomotion and measures the share of the latent in the joint targets, shown for trot.

H1 · Adaptation

The policy adapts to the domain through the components the behaviour does not occupy.

Heat maps per principal component and commanded speed of the latent's motion within a gait, the shift of the joint targets between domain draws and the action response, for both encoders. — **H1 Adaptation** – Share of each component in the latent’s motion within a gait (*Behaviour*), the shift of the joint targets between domain draws (*Domain*) and the action response per unit change (*Sensitivity*). With RECON + BCE the behaviour occupies the leading components and grows with gait and speed, while the domain shift sits on components 9–16, independent of the gait. The action responds most strongly to these. Under PPO the domain shift falls on the gait’s components.

H2 · Latent Feedback

The policy acts on the latent in closed loop.

A quarter-cycle clock step – During a steady trot the gait clock is advanced by a quarter cycle (red line). The feet follow the clock at once: one stance is cut short and the footfalls stay on the clock’s outline. The joint targets do not: their distance from the steady cycle jumps about ninefold and returns to its level within three to four strides, under both RECON + BCE (left) and PPO (right).

H2 Latent Feedback – After a quarter-cycle step of the clock, Feedback shades the latent’s part of one calf target, Propagation gives the latent’s share per joint and cycle and Phase Dependence the part of the step followed at once. The latent’s share rises from about 2 % to 19–67 % and decays over three to four cycles. The part followed at once depends on the phase of the step and falls from two thirds at walk to one third at bound. The latent thus closes the control loop under both gradients.

Environment–Representation Coupling

A disturbance from the environment reaches the policy only through the latent. Robust locomotion requires that the latent registers it as a deviation from its mean cycle and that the policy acts on this deviation (H1). Pushes of 0.2 s are applied at the middle of each gait band, with 0–200 N in twelve levels, four directions and eight onset phases. From the onset, the latent the policy reads is replaced by its mean cycle plus a selected part of its deviation. Both policies are evaluated under nominal dynamics.

H1 · Robustness

Withstanding a disturbance requires the latent’s deviation from its mean cycle.

A lateral push under latent substitution – At trot, seen from behind, RECON + BCE above and PPO below. From the red tick on, the latent the policy reads is its mean cycle plus the retained part of its deviation; the red band is the 0.2 s push of 100 N.

H1 Robustness – Latent Divergence shows one trot trial on the ring of its mean cycle with the deviation as height, before and after the deviation is withheld (Feedforward). Latent Feedback gives the share of each component in the policy’s feedback term. Push Tolerance gives the survival at trot under lateral pushes per retained part of the deviation. Without the deviation the latent diverges and the robot falls without a push. With RECON + BCE, components 9–16 alone nearly preserve the push tolerance (103 N against 118 N), while components 1–8 alone do not. Under PPO neither part suffices.

Conclusion

Stage one. The auxiliary prediction objective organises the latent by behaviour and keeps the domain separate from it. The policy then adapts to the domain and withstands disturbances through the components the behaviour does not occupy. Ongoing work on co-development examines across training epochs whether the latent organises motion before the policy realises behaviour from it.

Stage two. The clock is removed next. The encoder does not observe the clock, and its latent already completes one loop per stride. It should therefore be able to provide the phase. Then the prescribed gaits are removed, so that the learner discovers motion primitives.

References

G. B. Margolis, P. Agrawal. Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior. CoRL 2022.
A. L. Mitchell et al. Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotion. CoRL 2024.
G. Bellegarda, A. Ijspeert. CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion. IEEE RA-L 2022.
J. Lee et al. Learning Quadrupedal Locomotion over Challenging Terrain. Science Robotics 2020.
A. Kumar et al. RMA: Rapid Motor Adaptation for Legged Robots. RSS 2021.
S. Chen et al. SLR: Learning Quadruped Locomotion without Privileged Information. CoRL 2024.