continuity – Is it possible to define integration over a discontinous domain?

I’m trying to think about this from the Riemannian integration perspective so let me know if Lebesgue integration or something else is better. An example where I seem to be running into problems is with integration over the domain $mathbb{R}setminusmathbb{I}$. If you want to integrate $fcolonmathbb{R}rightarrowmathbb{R}setminusmathbb{I}$ given by

f(x) = e^{-x}

over $D = (0, 4)$, you should be able to get away with Riemannian integration (even though the interval is discontinuous) since the discontinuities are negligible. I guess my intuition for this is that $f(x)Delta{x}$ will disappear in the limit $Delta{x}rightarrow0$ for these discontinuities since they are finite. In a Riemannian sum, as the number of partitions goes to infinity, the contributions (or lack thereof) of 4 pieces will make no difference. So in that sense this integral over a finite domain seems to make sense, or be something we can in a way “get away with”. But what about the integral from $0$ to $infty$? In that, we have an infinite number of missing partitions. What would the difference amount to? How do you compute or even define integration here?