Human In The Loop (HITL) is a process in which, as part of the Machine Learning (ML) workflow, experts are asked their opinion about the model’s predictions in order to improve it. We’ll discuss how we created a mechanism to automatically predict the best security policies for network DDoS to protect our customers, and explain how we integrated security experts into our ML process, in order to both optimize the labeling of security policies, and move to production quickly with minimum risk.