Timo Kaufmann

Teaching AI agents to solve the right problems.

timo.jpg

I’m a PhD student in the AI+ML group at LMU Munich, advised by Eyke Hüllermeier. My research is focused on developing novel techniques that combine reinforcement learning and preference learning to ensure that agents solve the right problems efficiently and robustly. Specifically, I’m interested in the intersection of these two fields and how they can be used to help agents learn and quickly learn the right behaviors in novel environments. Through my work, I hope to contribute to the advancement of AI and its ability to solve complex problems in the real world.

News

Sep 14, 2023 I will present our workshop paper on the challenges and practices of reinforcement learning from real human feedback on the HLDM’23 ECML workshop on Friday next week (September 22nd)!
Apr 4, 2023 I am organizing an invited session on Reinforcement Learning from Human Feedback (RLHF) at ECDA 2023.
Mar 28, 2023 I presented our short-paper at the ML4CPS conference. In the paper we discuss the potential of leveraging self-supervised pretraining for reinforcement learning from human feedback in the context of cyber-physical systems.
Mar 15, 2022 I started my PhD at Eyke Hüllermeier’s AI+ML group at LMU Munich!

Selected Publications

  1. arXiv
    Problem Solving Through Human-AI Preference-Based Cooperation
    Subhabrata Dutta, Timo Kaufmann, Goran Glavaš, and 7 more authors
    2024
  2. MHFAIA
    Comparing Comparisons: Informative and Easy Human Feedback with Distinguishability Queries
    Xuening Feng, Zhaohui Jiang, Timo Kaufmann, and 3 more authors
    In ICML 2024 Workshop on Models of Human Feedback for AI Alignment (MHFAIA), 2024
  3. MHFAIA
    Relatively Rational: Learning Utilities and Rationalities Jointly from Pairwise Preferences
    Taku YamagataTobias OberkoflerTimo Kaufmann, and 3 more authors
    In ICML 2024 Workshop on Models of Human Feedback for AI Alignment (MHFAIA), 2024
  4. arXiv
    Inverse Constitutional AI: Compressing Preferences into Principles
    Arduin FindeisTimo KaufmannEyke Hüllermeier, and 2 more authors
    2024
  5. RLBRew
    OCALM: Object-Centric Assessment with Language Models
    Timo KaufmannJannis BlümlAntonia Wüst, and 3 more authors
    In ICML 2024 Workshop on Reinforcement Learning Beyond Rewards (RLBRew), 2024
  6. arXiv
    A Survey of Reinforcement Learning from Human Feedback
    Timo KaufmannPaul WengViktor Bengs, and 1 more author
    2023
  7. HLDM
    On the Challenges and Practices of Reinforcement Learning from Real Human Feedback
    Timo KaufmannSarah BallJacob Beck, and 2 more authors
    2023
    Accepted at the ECML-PKDD HLDM’23 Workshop.
  8. Reinforcement Learning from Human Feedback for Cyber-Physical Systems: On the Potential of Self-Supervised Pretraining
    Timo KaufmannViktor Bengs, and Eyke Hüllermeier
    In Proceedings of the International Conference on Machine Learning for Cyber-Physical Systems (ML4CPS), 2023