Yes, it is pretty much shotgunning it a bit to just replace the capacitors without testing them, but if you remove one of them, you'll be convinced they're bad, as most likely you'll find a stinky puddle of electrolyte underneath it.
And also, experience has shown that in the vast majority of cases, when things like this happen, replacing the capacitors *usually* solves the problem. And when it doesn't, it's usually some relatively small component that needs to be replaced in addition.
Anyway, if you don't want to deal with recapping, a possible alternative would be to simply retrofit a modern PSU into the old PSU's casing. There's a SeaSonic PSU which is very popular for this purpose as it's very small, but I don't recall the model number (it's posted on the forum in a recent discussion regarding upgrades for the SE/30 PSU, coincidentally). The hardest part is wiring it for soft power, but since the SE/30 doesn't need that, it's just a matter of wiring it in and setting the new PSU to always power on when it's plugged in.
c