This webpage is run by Professor Philip Thomas. The views presented on this webpage are those of Prof. Thomas, and may or may not be shared by his co-authors, collaborators, and funding agencies.

Authors of Preventing Undesirable Behavior of Intelligent Machines


Philip S. Thomas

Philip S. Thomas

pthomas@cs.umass.edu
Assistant Professor, College of Information and Computer Sciences, University of Massachusetts Amherst
Homepage
Bruno Castro da Silva

Bruno Castro da Silva

bsilva@inf.ufrgs.br
Associate Professor, Institute of Informatics, Federal University of Rio Grande do Sul
Homepage

Andrew G. Barto

Andrew G. Barto

barto@cs.umass.edu
Professor Emeritus, College of Information and Computer Sciences, University of Massachusetts Amherst
Homepage
Stephen Giguere

Stephen Giguere

sgiguere@cs.umass.edu
Doctoral Candidate, College of Information and Computer Sciences, University of Massachusetts Amherst
Homepage

Yuriy Brun

Yuriy Brun

brun@cs.umass.edu
Professor, College of Information and Computer Sciences, University of Massachusetts Amherst
Homepage
Emma Brunskill

Emma Brunskill

ebrun@cs.stanford.edu
Assistant Professor, Computer Science, Stanford University
Homepage

Student Seldonian algorithm research

The following students are conducting research related to Seldonian algorithms at UMass Amherst, most under the supervision of Prof. Thomas.
Stephen Giguere

Stephen Giguere

Homepage

Stephen is a doctoral candidate in the Autonomous Learning Laboratory at UMass Amherst, and a co-author of the November 22 paper in Science. He created the interface for the Seldonian classification algorithm in that paper, and the interface for the Seldonian contextual bandit algorithm presented at NeurIPS 2019 [link]. He is currently working with Susmit Jha from the Stanford Reseach Institute (SRI) to create Seldonian algorithms that are robust do disributional shift.

Blossom Metevier

Blossom Metevier

Homepage

Blossom is a doctoral student in the Autonomous Learning Laboratory at UMass Amherst, and a the first author of a NeurIPS 2019 paper that presents a Seldonian contextual bandit algorithm [link]. In this work, Blossom and her co-authors show how a Seldonian contextual bandit algorithm can predict who will repay a loan, who will commit a crime in the future, and how to change an online tutorial to improve overall student performance, all while avoiding various types of unfair behavior.

Yash Chandak

Yash Chandak

Homepage

Yash is a doctoral student in the Autonomous Learning Laboratory at UMass Amherst. Yash's research focuses on understanding the challenges preventing real-world applications of reinforcement learning, and developing princpled algorithms to bridge this gap. He is currently working to create Seldonian reinforcement learning algorithms that provide high-confidence fairness and safety guarantees even in the presence of nonstationarity.

Pinar Ozisik

Pinar Ozisik

Homepage

Pinar is a doctoral candidate at UMass Amherst, advised by Professor Brian Levine. Working under Prof. Levine, Pinar's research focuses on computer security. She has shown that Seldonian algorithms, and particularly Seldonian reinforcement learning algorithms, are not robust to attacks. If someone can maliciously change some of the data provided to the algorithm, or if data is unintentionally corrupted, the algorithm's safety and fairness guarantees can be violated significantly. She is currently working on developing Seldonian reinforcement learning algorithms that are secure—algorithms that are robust to accidental data corruption and even malicious attacks.

James Kostas

James Kostas

Homepage

James is a doctoral student in the Autonomous Learning Laboratory at UMass Amherst. His research focuses on improving generalization in reinforcement learning. He is currently working to create a Seldonian reinforcement learning algorithm that can guarantee with high probability that it will be safe and fair on a new problem based on its experience solving similar, but different, previous problems.

Sarah Brockman

Sarah Brockman

Sarah is a masters student in the Laboratory for Advanced Software Engineering Research at UMass Amherst, advised by Professor Yuriy Brun. In her undergraduate honors thesis, Sarah worked to create a Seldonian contextual bandit algorithm that she deployed in a user study to automatically improve online tutorials (intelligent tutoring systems), while ensuring that the algorithm does not change the tutorials in a way that benefits the majority of the students at the expense of a protexted minority. She is a co-author of the paper presenting this work that will appear at NeurIPS 2019 in December [link]. She continues to work in the area of ensuring fairness.

Funding


Research into Seldonian algorithms is currently funded by:
  • Gifts from Adobe [link]
  • The Army Research Office, via IoBT
Research into Seldonian algorithms was previously funded by: