As XR4x4rs says, put the multimeter leads on the battery terminals, select to voltage (of a suitable range) make sure its DC volts!

With the engine off it should read ~12v, start it up, if the alternator is working well then it will read about 13.2-14.5ishv