Safe Reinforcement Learning for Sepsis Treatment

Sepsis, a life-threatening illness, is estimated to be the primary cause of death for 50,000 people a year in the UK and many more worldwide. Managing the treatment of sepsis is very challenging as it is frequently missed, at an early stage, and the optimal treatment is not yet clear. There are promising attempts to use Reinforcement Learning (RL) to learn the optimal strategy to treat sepsis patients, especially for the administration of intravenous fluids and vasopressors. However, RL agents only take the current state of patients into account when recommending the dosage of vasopressors. This is inconsistent with current clinical safety practice in which the dosage of vasopressors is increased or decreased gradually. A sudden major change of the dosage might cause significant harm to patients and as such is considered unsafe in sepsis treatment. In this paper, we have adapted one of the deep RL methods published previously and evaluated it to assess whether it has this kind of sudden major change when recommending the vasopressor dosage. Then, we have modified this method to address the above safety constraint and learnt a safer policy by incorporating current clinical knowledge and practice.