Adding Temperature to Remote Robot Control
“Grasping Temperature: Thermal Feedback in VR Robot Teleoperation” shows how combining thermal sensing with VR and Sensory Reality™ Pods increases situational awareness, safety, and telepresence for operators controlling robots in hazardous environments.
KEY INSIGHT
Traditional teleoperation is temperature-blind: operators see the scene but cannot feel heat or cold. This work integrates thermal cameras, VR, a Sensiks Sensory Reality Pod, and a thermal glove so operators directly sense ambient and object temperature, improving safety and awareness around fires, hot machinery, or people outside the visual field of view.
From Robot Sensors to Multi-Sensory Operator Pod
The demonstrator links a multi-camera robot sensor suite to a VR headset, thermal glove, and Sensiks Pod with directional heaters, creating a complete “thermal telepresence” loop.
Robot-Side Sensor Suite
The robot carries three synchronized cameras:
- Zed2i stereo camera for 3D color vision and depth.
- Ricoh Theta Z1 360° camera for full environmental awareness.
- FLIR A655sc LWIR camera for precise thermal imaging and temperature mapping.
Calibration aligns thermal and stereo images so every heat source has an accurate 3D position.
Processing & Data Bridge
A Remote Data Access (RDA) module streams synchronized sensor data to the operator station in real time. Onboard processing:
- Detects “hot” and “cold” objects based on temperature thresholds.
- Projects thermal detections into 3D using stereo depth.
- Tags people and equipment as dynamic heat sources.
This enables intelligent, low-latency mapping of thermal events into the VR world and the Sensiks Pod.
Operator-Side Multi-Sensory XR
On the operator side, three modalities are combined:
- HTC Vive Pro Eye HMD for 3D and 360° visual immersion.
- ThermoReal® glove rendering hot/cold when touching virtual objects.
- Sensiks Sensory Reality Pod with six heaters providing directional ambient heat.
Together, they create a unified thermal experience that mirrors the remote robot’s environment.
What It Feels Like to “Grasp Temperature”
In proof-of-concept trials, operators used the full multimodal setup to distinguish hot and cold objects, detect people, and navigate using temperature cues layered on top of VR visuals.
SCENARIO EXAMPLES
- Approaching hot equipment: the VR view shows a thermal overlay, the glove gets warm when grasping the object, and the pod heater in the corresponding direction intensifies, signaling danger.
- Detecting people outside field-of-view: when someone walks near the robot, a heater in the Sensiks Pod activates from that direction, letting the operator “feel” their presence even before seeing them.
- Surveying an environment: gradients of warmth and cold help identify machinery, leaks, fires, and safe paths through low-visibility or complex spaces.
Where Thermal Telepresence Makes a Difference
Thermal feedback extends robot teleoperation into domains where temperature is critical for safety, performance, and human-robot collaboration.
Hazardous Industrial Sites
- Nuclear facilities and chemical plants.
- Monitoring overheating equipment and process anomalies.
- Detecting hot pipes, leaks, or fire risks in real time.
Emergency Response & EOD
- Firefighting robots navigating smoke-filled environments.
- Search and rescue with body-heat detection.
- Explosive ordnance disposal with thermal monitoring of devices.
Extreme & Remote Environments
- Space exploration and planetary rovers.
- Deep sea surveys and underwater infrastructure inspection.
- Remote care, monitoring, and assisted living scenarios.
Grasping Temperature: Thermal Feedback in VR Robot Teleoperation
Inline access to the original ACM IMX ’22 conference paper on thermal feedback in VR robot teleoperation.
Inline PDF Viewer
Embedded copy of the uploaded publication.
Explore Temperature-Aware Telepresence with Sensory Reality™
Use this thermal research page as a launchpad for pilots that combine thermal sensing, VR teleoperation, and Sensiks Pods in your own domain—from industrial safety and defense to space exploration and remote care.