If you're going to do any experimenting at all in electronics, you need one of these. They're really cheap and they will save you tons of irritation. Here's what a breadboard looks like (image stolen from LadyAda.net)
Using a breadboard you can quickly put circuits together and change them around in seconds.
Breadboard jumper wires
You can just cut bits of solid core wire, telephone wire (24 gauge solid core) works fine, but for very little money you can just buy a bunch of jumper wires with perfect little ends on them for working with breadboards. Here are some jumper wires (image from adafruit)
Arduino is a fantastic environment to get started with microcontrollers. There is a huge amount of help available online, the amount of stuff you can buy to interface with it to make your projects a reality quickly is almost hard to believe. For one-off or very small runs of hobby projects, the Arduino is probably the way to go. There are a lot of different Arduinos. If you will be buying peripherals for the Arduino (called "shields") then you will want a shield-compatible board such as the Arduino Uno. These can be found on ebay for < $15
If you intend to build your stuff on a breadboard and possibly eventually incorporate it into a small board, then the Arduino Nano is a good choice. You can plant it right onto a breadboard and wire directly to it. Arduino Nanos are < $10 on eBay. Be sure to specify "USB" - there are some versions without it. Later if you want to really cut costs, you can buy the Arduino Pro Mini which usually does not have a USB interface - you use an external USB adapter to program them then unplug it when you deploy your design.
|Arduino Nano USB|
The Arduino family is pretty huge. There's also the Arduino Mega which extends to much more I/O, more memory and a more powerful CPU.
There are other more powerful development environments such as the Raspberry Pi, BeagleBone Black and others which are basically small computers complete with networking, keyboard/mouse and monitor connections. If you need that kind of power, they're definitely worth looking at. My projects tend to be smaller so I stick to the smaller solutions.
If you're going to play with Arduino, you don't need this - normal Arduinos have a USB interface built in.
However, if you are working on a project that will have a bigger number of items produced and it is cost sensitive, you may wish to use a smaller chip - AVR processors can cost as little as 50 cents in quantity. If you're going to use microcontroller chips more directly, you'll need a programming device. All of my work is done with AVR chips, and these can be readily programmed with an "in-system programming" (ISP) device. I use the USBASP programmer, it's a little USB stick with a ribbon cable out the back. The cable goes to a 10 pin header, but I prefer the 6 pin, so be sure to get one with an adapter to go from 10 to 6 pin. You can find these programmers with adapters for < $7 on ebay.