Here is a picture of a typical power supply.
The box at the top has all the capacitors in it. You can use one big capacitor or many smaller capacitors wired in paralell. It is often cheaper to have many smaller capacitors than one big one that equals the same value. When designing your power supply it is recomended to have 2000uf capacitance for every amp your transformer can deliver. Example, a 30 amp transformer would require 60,000 uf capacitance.
You will note the heat sink at the top. This is to cool down the rectifiing diodes. A typical power supply requires 4 diodes arranged in such a way to convert a AC current to a DC current. The arrangement is called bridge rectification.
Other electronic parts you can add but are not absolutely essential are, a bleeder resister that will slowly disscharge the capacitors when the power supply is off. Also an electronic device called a MOV can be used to protect your circuit from power surges. A MOV will short out over a specified voltage and take the punishment instead of your circuit.