We present GARField (Garment Attached Radiance Field), the
first differentiable rendering architecture, to our knowledge,
for data generation from simulated states stored as triangle
meshes.
While humans intuitively manipulate garments
and other textile items swiftly and accurately, it is a significant
challenge for robots. A factor crucial to human performance
is the ability to imagine, a priori, the intended result of the
manipulation intents and hence develop predictions on the garment pose.
That ability allows us to plan from highly obstructed
states, adapt our plans as we collect more information and react
swiftly to unforeseen circumstances. Conversely, robots struggle
to establish such intuitions and form tight links between plans
and observations.
We can partly attribute this to the high
cost of obtaining densely labelled data for textile manipulation,
both in quality and quantity. The problem of data collection
is a long-standing issue in data-based approaches to garment
manipulation. As of today, generating high-quality and labelled
garment manipulation data is mainly attempted through advanced
data capture procedures that create a simplified state
estimations from real-world observations.
However, this work
proposes a novel approach to the problem by generating
real-world observations from object states.
