After reviewing some different indispensability arguments, I distinguish several different ways in which mathematics can make an important contribution to a scientific explanation. Once these contributions are highlighted it will be possible to see that indispensability arguments have little chance of convincing us of the existence of abstract objects, even though they may give us good reason to accept the truth of some mathematical claims. However, in the concluding part of this paper, I argue that even though there is a valid indispensability argument for realism about some mathematical claims, this argument is problematic as it begs the question at issue. This challenge to indispensability arguments is then used to suggest that if mathematics is making these sorts of contributions to science, then it may be the case that mathematical claims receive some non-empirical support prior to their application in scientific explanation.