Convex Optimization

I work in that area :) The skills needed will vary a lot. Hence my concern about studying H&H. It's a good idea, as real analysis is the foundation. But it will take too much of your time to get to something useful. Probably you should try to learn more applied material in parallel and let both threads merge in the future. For maximum likelihood, you need to learn convex optimization right after real analysis. The canonical reference is [1], but there's also a very simple and pragmatic linear algebra textbook by the same author that also covers some of the optimization basics [2]. This might be a good entry point, certainly easier than Spivak or H&H. There's also [3,4], which you probably know about. These are great and emphasize the modeling part. Maximum likelihood (via EM) is in the appendix, and you don't really need to know a lot of math to get going.…

Convex Optimization - Boyd

Attempting to self study statistics and econometrics in a serious way. I just finished Casella and Berger's "Statistical Inference". I'm about halfway through Shumway and Stoffer's "Time Series Analysis", but then next on the list, in the order I tend to read them: - Theory of Point Estimation Casella/Lehmann - Bayesian Data Analysis 3rd ed. - Convex Optimization - Boyd - Econometrics - Hayashi - New Introduction to Multiple Time Series Analysis - Elements of Statistical Learning - Machine Learning: A probabilistic perspective There's a fair amount of overlap between these books, so it's not quite as much as it seems. But i'm hoping to make it through at least a chapter a week this year, which should get me most of the way through them. We'll see how it goes.

I'm not sure "Convex Optimization" by B&V is a good self-study book. I really struggled with it as student, and I struggled to teach with it as a TA. It's one of those "Step 2. Draw the Rest of the Owl!" books. Great as a reference manual, but not great pedagogically. It's very complete and I would struggle to name anything better, though. Mostly, I learned convex analysis by scouring Google for lecture notes written by random professors, then cross-referencing with B&V.

i second the David MacKay recommendation. its a real pleasure to read. for me personally, his coverage of the fundamentals of information theory was amazing. the proofs are easy to follow and explained well so you can match the intuition to formula. i'd also like to recommend Boyd & Vanderberghe's book on Convex Optimization (available here http://www.stanford.edu/~boyd/cvxbook/). This is maybe a little less fun but very deep and still well written and 'illustrated'. My background is in ML and I came to realize over time that a lot of ML is just statistics and optimization theory rehashed by computer scientist.