They still can be reasonable quality (and most hand-held recorders use them). Those cannot offer the same S/N ratio as true condenser microphones, particularly not large-diaphragm condensers. Now USB mics are usually electret-condenser capsules. Broadband noise, in contrast, is almost sure to originate from the analog circuitry of your microphone, and it cannot be 'weak power' either since USB runs on fixed voltage.
You can get various bits of noise from unstable power supply and inverters for step-up DC/DC converters and ground loops. If your characterization of 'white noise' is somewhat accurate, you are out of luck.
