While I think nonconvex optimization is a very interesting+important area of research, I often feel like the community overstates its importance in ML. Finding a global optimum almost never happens in successful usage of neural nets (we use early-stopping with a validation set) and is not necessarily the best-idea for many applications. Rather, I feel that selecting a proper objective function ...
Source: Discussion on r/MachineLearning




