Hinge Loss is a useful loss function for training of neural networks and is a convex relaxation of the 0/1-cost function. There is also a direct relation to soft margin support vector machines, as we demonstrate in this short tutorial. With this in mind, we can now build new powerful loss functions for neural network training such as the user loss. See the full set of slides here and a version as video presentation on youtube.

References:

[1] Vincent Christlein. Hand-written Document Analysis with Focus on Writer Identification and Writer Retrieval. PhD Thesis. Friedrich-Alexander-University Erlangen-Nuremberg, 2018.

[2] Shahab Zarei, Bernhard Stimpel, Christopher Syben, Andreas Maier. User Loss - A Forced-Choice-Inspired Approach to Train Neural Networks directly by User Interaction. Under Review. https://arxiv.org/abs/1807.09303

[3] Andreas Maier, Frank Schebesch, Christopher Syben, Tobias Würfl, Stefan Steidl, Jang-Hwan Choi, Rebecca Fahrig. Precision Learning: Towards Use of Known Operators in Neural Networks. International Conference on Pattern Recognition ICPR 2018 (to appear). https://arxiv.org/abs/1712.00374