Seeing, Feeling, Understanding: AI at the Human–Machine Interface in Manufacturing

• Seeing → smart glasses / AR guidance
• Feeling → direct torque control / haptics
• Understanding → datasheet decryption / semantic extraction

Seeing – Smart Glasses / AR Guidance

Wearable AI guides the operator through assembly and maintenance, hands-free and eyes on the part.

Smart glasses for example assist assembly processes or guide maintenance work. With AI in the background, the glasses support the operator by answering contextual questions based on what the operator is viewing and on the available documentation.

The challenge often lies not within the AI, but between concept and implementation:

  • Hardware-aware deployment: Working within the constraints of wearable devices such as battery life, thermal limits and image quality.
  • Inference trade-offs: Balancing on-device responsiveness with cloud-based reasoning to keep latency low.
  • Contextual grounding: Linking AI responses to the actual task, the operator's view and the available technical documentation.

Smart glasses also assist factory workers. Not only with static work instructions, but also with in-context guidance. Digital intelligence is directly overlaid onto the assembly station, so operators get the right information at the right step, hands-free and eyes on the part. A few aspects that make the system promising:

  • Visual and audio workflow guidance: AR overlays combined with synchronized voice instructions walk operators through each assembly step without tablets or paper manuals.
  • Hands-free multimodal interaction: Voice commands let operators control the system while keeping both hands on the workpiece and tools
  • Real-time digital co-pilot: The system bridges the gap between static assembly documentation and dynamic execution, adapting guidance to what the operator is actually doing

AR systems earn their place in the assembly process, making complex tasks faster, safer and more accurate.

Feeling - Direct Torque Control / Haptics

 A 500 Hz torque loop turns the robot into a haptic interface, letting you feel surfaces that don't exist: With low-level robot control using direct torque control, you can feel a surface, that doesn’t really exist. By mapping a virtual sinusoidal geometry into the robot's workspace and running a 500 Hz torque loop, the end-effector becomes a haptic interface and lets you physically feel the peaks and valleys of a wall that isn't there.

This is just one example of how active impedance control can unlock real improvements in industrial applications:

  • Precision surface finishing: Maintaining constant contact force on curved or irregular geometries during grinding, polishing and deburring.
  • Force-guided assembly: Inserting tight-tolerance parts through tactile feedback, emulating the intuitive feel of an experienced technician.
  • Intuitive human-robot collaboration: Hand-guiding interfaces where the robot carries the load but stays genuinely soft to the operator.

Understanding — Technical Datasheet Extraction

 An automated pipeline turns unstructured PDFs into reliable, machine-ready data for Industry 4.0: Technical datasheets are a fundamental asset in manufacturing, serving as the primary source of product specifications. But here is the real challenge: they are typically unstructured PDFs with highly heterogeneous formats, making manual data extraction time-consuming and prone to errors.

Our latest research, "A novel pipeline and benchmark for automated technical datasheets processing" has been recently published in the Journal of Intelligent Manufacturing. We set out to address the bottleneck of converting raw, complex documents into formats that are digestible by machines.

Our findings provide a data-driven analysis of modern LLM capabilities and limitations (such as layout sensitivity and hallucinations), while offering a scalable methodology to develop reliable semantic extraction solutions for Industry 4.0.