There are quite a few options available - new and used, one-piece or up-to-three-pieces.
The essential elements are a turntable with tonearm and cartridge, an RIAA pre-amplifier, and a sound card capable of good recording at 44.1 kSamples/second (or 48 kSamples/second if you don't care about burning to CD-R). You can buy these integrated together, or separately.
If you want to go the "integrated" approach, check out the article at http://vinyl-converter-turntable-review ... views.com/
- it lists a whole bunch of all-in-one "USB turntable" products. These have a USB "sound card" and an RIAA preamplifier (with the necessary equalization circuitry) built right in. Plug into AC power, plug the USB cable into your computer, and the turntable will show up as a new "sound card". You can use the supplied software to "rip" the music, and convert to MP3 or FLAC or Ogg Vorbis or AAC, and burn to CD-R - or, use other commercial or free software of your own choice. Without exception (I think) these devices all show up as standard "USB sound" devices and so should be compatible with any competent recording software.
Or, you can go with "separates"... buy the turntable/tonearm/cartridge from one source, buy the RIAA preamp separately, and either buy a USB sound interface or use the sound card build into your PC. This allows for much greater flexibility, and you can build a ripping system of higher quality that way, but it's more trouble and usually more money (unless you already have some of the components, or are good at scrounging, or both). A lot of the "sound cards" integrated with PC motherboards these days aren't terribly great (e.g. excessive noise levels on the input) and many of them don't even have "line-level" inputs any more (just microphone inputs). If yours is of this sort, plan on getting an external (USB) sound interface... or, if you have a PCI slot to spare, scrounge an old Ensoniq AudioPCI or SoundBlaster PCI card (the old ES1370 cards in particular were very good, and should be quite cheap on eBay). Locating a stand-alone "RIAA" preamplifier isn't always easy... but if you have an old stereo receiver or preamp with a "phono" input, you can plug the turntable into it, feed the receiver's "TO TAPE RECORDER" outputs to the sound card line-level interface, and you're good to go.
You'll want a turntable which is good enough that its rumble, wow, and flutter aren't a problem (less audible than the noise on the best-quality record you're going to play and rip). New stand-alone turntables are still being made; their prices range from reasonable to HOLY FUDD! ridiculous. You'll want a decent cartridge.
You might want to check with http://www.audioadvisor.com/
(I've purchased products from them a few times over the years but have no other association with them). They have both USB and standalone turntables, and the brands that they carry are serious ones, not throwaway junk or kids' toys.
An important part of doing a good "rip" of older LPs is cleaning them before you rip. It's *much* easier to remove dirt, dust, and fingerprints from a record before playing it, than it is to remove the resulting noise and distortion via digital processing.
Bare-minimum: a good LP dust brush.
Strongly recommended: some form of wet-cleaning system. This can be as simple as a bottle of proper cleaning fluid, and a couple of suitable brushes (one to wet-wipe with, and one to dry with) such as the old Discwasher system. Or, it can be much more involved (there are "wet wash and vacuum" systems which work very well indeed).
Consider if you're really serious: peel-off-film cleaning. In these systems, you coat the LP with a thin layer of a special emulsion, let it dry, and then peel it off... the film encapsulates and lifts off an amazing amount of dirt and grit which will escape both dry-brush and wet-brush systems. Old thrift-store records that are so dirty they're painful to play, can come out sounding almost new (actual scratches and damage don't go away, of course).
The best-known film cleaning system these days may surprise you: the "special emulsion" is Titebond II wood glue from your local hardware or home-improvement store. Really. I'm quite serious. You put about one fluid ounce of Titebond II on the LP surface, spread it around with the edge of a credit card, and let it dry. Once it dries, you can peel it off the surface with a bit of tape, leaving a clean record surface behind. Cost is about a dollar per LP if you buy Titebond II in one-quart bottles, less if you buy it by the gallon. Google-search "LP cleaning glue" or some such phrase and you'll find all sorts of references, how-tos, YouTube videos, etc.
A more sophisticated film cleaner is the Reg Williamson recipe, original published in Audio Amateur, which is itself based on the original BBC archivists' recipe. You have to brew it up yourself, using polyvinyl alcohol powder (Elvanol), distilled water, alcohol, some surfactant, and an anti-static additive (the latter is the hardest thing to find). I find it superior to Titebond II - dries faster, peels off more easily and consistently, and it leaves the record completely static-free (pretty much permanently) but it's definitely less convenient than just going down to Lowes or Home Depot for a gallon of Titebond II.