There are several vendors, who ship systems with Ubuntu or other distributions of GNU/Linux pre-installed. You might pay more for the privilege, but it does buy a level of peace of mind, since you can be sure that the hardware is well-supported by GNU/Linux.
If you do have to buy a machine with Windows bundled, carefully read the software license that comes with Windows; you may be able to reject the license and obtain a rebate from your vendor. Searching the Internet for “Microsoft Windows Rebate” will provide information on how to go about getting a rebate.
Whether or not you are purchasing a system with Linux bundled, or even a used system, it is still important to check that your hardware is supported by the Linux kernel. Check if your hardware is listed in the references found above. Let your salesperson (if any) know that you're shopping for a Linux system. Support Linux-friendly hardware vendors.
Some hardware manufacturers simply won't tell us how to write drivers for their hardware. Others won't allow us access to the documentation without a non-disclosure agreement that would prevent us from releasing the driver's source code, which is one of the central elements of free software. Since we haven't been granted access to usable documentation on these devices, they simply won't work under Linux.
In many cases there are standards (or at least some de-facto standards) describing how an operating system and its device drivers communicate with a certain class of devices. All devices which comply to such a (de-facto-)standard can be used with a single generic device driver and no device-specific drivers are required. With some kinds of hardware (e.g. USB “Human Interface Devices”, i.e. keyboards, mice, etc., and USB mass storage devices like USB flash disks and memory card readers) this works very well and practically every device sold in the market is standards-compliant.
In other fields, among them e.g. printers, this is unfortunately not the case. While there are many printers which can be addressed via a small set of (de-facto-)standard control languages and therefore can be made to work without problems in any operating system, there are quite a few models which only understand proprietary control commands for which no usable documentation is available and therefore either cannot be used at all on free operating systems or can only be used with a vendor-supplied closed-source driver.
Even if there is a vendor-provided closed-source driver for such hardware when purchasing the device, the practical lifespan of the device is limited by driver availability. Nowadays product cycles have become short and it is not uncommon that a short time after a consumer device has ceased production, no driver updates get made available any more by the manufacturer. If the old closed-source driver does not work anymore after a system update, an otherwise perfectly working device becomes unusable due to lacking driver support and there is nothing that can be done in this case. You should therefore avoid buying closed hardware in the first place, regardless of the operating system you want to use it with.
You can help improve this situation by encouraging manufacturers of closed hardware to release the documentation and other resources necessary for us to provide free drivers for their hardware.