The fact that the alternator is charging shown by the voltage going up when running, shows that the thin wire, normally blue is connecting correctly, this is further shown by the fact it iluminates at all.
What you have is a defective DIODE pack with leaky diode's, which are like one way electrical valves, they are part of the regulator/rectifier part that converts AC current of the alternator to DC current which the car runs on.
A simple repair/replace will sort it, should be around £35/45, if removing fitting yourself which is easy on these.
tabetha