The 23 bits of the mantissa determine the precision of the number. The 8 bits of the exponent determine the interval of the number.
Consider the same problem in decimal terms. To keep it simple, assume that a mantissa is a positive number with a precision of 6 decimals and a number of -20 to 20 the exponent. This means that you would be able to store numbers in the range of
(give or take a few zeros)
The mantissa ranges from 1 to 999999. The exponent is just there to move the decimal point.