Yes you absolutely can use a cap as a battery in fact there are /were some computers where the cmos settings and real time clock is powered by a cap.
basically caps in electronics are used in a number of ways. just offhand:
all dc power supplies use caps, to spread and flatten out the waveform (essentially turning a sine wave into a square wave, with a bridge rectifier essentially being a circuit that turns both sides of the sine wave into a square wave, and a reverse square wave, that are then superimposed on each other, and smoothed out.
filters, caps can pass a signal to ground, effectively removing it from the circuit
charge/discharge power sources
stable or more stable power sources.
etc.
the electronic symbol for a cap looks like -|(- or -||- basically if you look at almost any schematic you will find that all over the place.
the charge /discharge rate for caps, has to do with the ratio between the size of the cap (measured in uf) and the resistance of the circuit measured in ohms.
I could go on... but I am still 1/2 asleep.