I agree with you. They assume the level of math of an MIT freshman. And the level of computer experience of a modern 11 year old. So sometimes they lean on the math in situations where a professor today might explain something in terms of a computer. Take a step back and try to understand the concepts that they are explaining, even if your math is a little rusty. I love SICP, it really make functional programming click for me. The other book I recommend in this space is Advanced R.
Also recommended in this reply