Timo Kaufmann

Teaching AI agents to solve the right problems.


I’m a PhD student in the AI+ML group at LMU Munich, advised by Eyke Hüllermeier. My research is focused on developing novel techniques that combine reinforcement learning and preference learning to ensure that agents solve the right problems efficiently and robustly. Specifically, I’m interested in the intersection of these two fields and how they can be used to help agents learn and quickly learn the right behaviors in novel environments. Through my work, I hope to contribute to the advancement of AI and its ability to solve complex problems in the real world.


Sep 14, 2023 I will present our workshop paper on the challenges and practices of reinforcement learning from real human feedback on the HLDM’23 ECML workshop on Friday next week (September 22nd)!
Apr 4, 2023 I am organizing an invited session on Reinforcement Learning from Human Feedback (RLHF) at ECDA 2023.
Mar 28, 2023 I presented our short-paper at the ML4CPS conference. In the paper we discuss the potential of leveraging self-supervised pretraining for reinforcement learning from human feedback in the context of cyber-physical systems.
Mar 15, 2022 I started my PhD at Eyke Hüllermeier’s AI+ML group at LMU Munich!

Selected Publications

  1. arXiv
    A Survey of Reinforcement Learning from Human Feedback
    Timo Kaufmann, Paul Weng, Viktor Bengs, and 1 more author
  2. HLDM
    On the Challenges and Practices of Reinforcement Learning from Real Human Feedback
    Timo KaufmannSarah BallJacob Beck, and 2 more authors
    Accepted at the ECML-PKDD HLDM’23 Workshop.
  3. Reinforcement Learning from Human Feedback for Cyber-Physical Systems: On the Potential of Self-Supervised Pretraining
    Timo KaufmannViktor Bengs, and Eyke Hüllermeier
    Accepted at the ML4CPS 2023 conference.