Problem Statement

A fairly large number of decision-making tasks, such as routing vehicles in a network, deploying targeted ads or even working on drug discovery, can be reduced to an optimization problem. Under specific constraints, we seek to take actions that maximize or minimize a given objective.

Beyond this static problem, the domains of online learning and sequential decision-making study the more interactive case where an actor has to produce a behavior as a sequence of actions to optimize over time.

For instance, recommender systems solve the online problem of recommending content aligned with a user's interests, learning over time from her observed behavior. A robot tasked with reaching a target in a given number of steps would represent a sequential decision-making problem, where the full trajectory is evaluated towards the goal-reaching objective.

This already constitutes an entire body of research, formalized via the multi-armed bandits and reinforcement learning literature as the prominent subjects of study.

Specifically, when the closed-form objective is not known — which is often the case in practice — methods have been developed to learn from weaker forms of feedback, such as (noisy) objective evaluations, preference data, interruptions or even demonstrations from an expert (nearly) optimal player.

It is still an active question how to do this right, in a provably optimal manner, but it has already seen tremendous success in applications such as language modeling (refining language from user preferences), robotics (learning dancing moves from human videos) and controlling nuclear fusion plasma.

In many scenarios, however, agents are not acting in isolation, but need to interact with others. In these multi-agent systems, strategic decision-making gives rise to cooperative, competitive, and mixed behaviors.

Understanding how to act optimally in these cases, under the lens of game theory and optimization, is a massively open question with a lot of distinct challenges. There still exists a variety of simple tasks that we don't know how to formalize and solve, which motivates the study of the fundamentals to enable future progress.

While robots are being progressively deployed in society, human-robot and robotic interactions are gaining both a tremendous potential and a growing exposure to failure risks. More generally, my research tackles how to make such autonomous decision-making systems both optimal and resilient to strategic and adversarial perturbations.

With my colleagues, I had the chance to contribute to online optimization and game theory from different sides, considering both efficient methods for learning optimal behaviors in multi-agent systems and the understanding of strategic exploitability of learned multi-agent behaviors.

In Preference-Based Distributed Welfare Maximization, we propose an online algorithmic framework for the distributed optimization of unknown objectives from active preference feedback. On the complementary task of learning from demonstrations, we show in Matching Multiple Experts how brittle imitation learning techniques can be in multi-agent systems, and make progress on the structural assumptions that allow for low-exploitability in non-interactive settings.

Plenty of questions have still to be addressed, this is only the beginning of the journey. Feel free to reach out if you are curious about our research or want to collaborate.