One of the major problems with getting into retro gaming is finding the consoles. Some places they are easily attainable via local pawnshops and the like. Other places they are scarce. A lot of times the games are more common than the systems are. Often the systems are overpriced so it isn't always easy finding them at affordable prices.
Apart from the different experience of playing with an emulator and roms, which is controversial to some, one of the easiest and most affordable ways to get started is with reproduction consoles. There are a lot of them on the market and they're typically quite affordable these days for what they do. You might ask yourself, “How do I know which one to get and is it reliable?” That is what I am here to advise on.
While I have not used/tested every single one on the market, I know enough to share about several of these in a helpful way. While I prefer the original systems and have a few dozen consoles (not counting handhelds), I own and have used a number of these repro systems myself.Possibly the most famous of the systems is the Retron by Hyperkin. The Retron is a multi-cart supported system, meaning one can play games from multiple systems on it. The Retron is currently on its 5th generation. From humble beginnings it now has support for Gameboy, GB Color, GBA, Famicom, Super Famicom, NES, Super Nintendo, Sega Genesis, and Megadrive cartridges. This is the best bet for covering the most amount of systems. The trade off is that it's the most expensive, typically about $150. They're more reliable now than they ever have been so if you can afford the cost, it's worth it.
Another popular one is the Retro Duo by Retro-Bit. Serves the same purpose as the Retron but it only supports NES/SNES games. It supports the Game Genie and Super Gameboy though. If one only wants to play these systems, this may be a more prudent choice. Also comes in portable handheld format. Another one that has become more popular over the last several years, possibly because there seems to be renewed interest in the Sega Genesis/Megadrive collecting market in the United States, is the Sega Firecore (unofficially AKA the Sega Genesis Model 4), which was officially licensed by Sega to AT Games to produce. This is a very cool SG/MD console that is about the size of a clenched fist. I've used it extensively and find it quite reliable. Some of its very cool features include: over 80 built-in games (including one or two that were never released in North America), wireless controllers (although I highly prefer using the original SG controllers) and composite/RCA support. It does have some drawbacks compared to the original system though: No Game Genie support, only mono audio (the original genesis was mono as well but could be modified to produce (I believe true) stereo audio, which would have been a nice touch with this), the console is very lightweight (maybe less than a pound), which can be a downside, especially with a controller plugged in and it also still suffers from the occasional slowdown that plagued retro consoles (presumably because it still contains the same CPU and amount of RAM). There is also several versions of the Firecore available in portable format. I have had one myself for years. It's essentially the shape of a PSP but thicker and it accepts SG/MD carts just like the console version. There are some versions without cartridge support though. This portable Firecore is far more preferable to a Sega Nomad for a whole myriad of reasons I won't go into. One reason though is that the Nomad was designed 11 years before the Firecore.
Another popular one is the Supaboy by Hyperkin. It is essentially a portable SNES. It's painted to look like a North American SNES which is pretty cool. It supports actual SNES/Famicom carts but also has a very cool additional feature usually absent on other repro portables: controller ports so one can use original SNES controllers. This has the added bonus/effect of being able to play games with two players at the same time! A quite useful and cool feature. It also has composite video/audio output so it can be hooked up to a TV and used just like an original SNES console. It works well and all of these things make it a worthwhile purchase at (usually) around $60-75 if you like the SNES.
Feel free to share any questions or comments. You can just reminisce about retro gaming in general too. I like to talk about that. For the usual tune I leave you all with, today I've chosen one of my favorite tracks from a PS1 classic, Tomb Raider 2. It's called 'Lara Plays in the Snow' from the Tibetan Foothills stage. It's composed by Nathan McCree.