how do you charge lithium betteries

i was hoping to find out a little more about how the charger actually charges the batteries.

is it as simple as setting the max voltage and then pumping that many volts into the battery untill the battery reaches that level.

for example if i have a lithium battery that i want to charge to 4v could i just pump 4v into the cell no matter what the voltage of the cell is. even if the cell is discharged to 3v.

or does the charge voltage need to be set at a slightly higher level then the battery voltage.
for example
cell voltage = 3v charge voltage =3.1v,
cell voltage = 3.2v charge voltage =3.3v
etc

there are two reasons i ask. Firstly I've seen a couple of people use dc/dc converters to charge cells individually and these can't adjust the voltage. Secondly i have been pondering the idea of putting in a permanent petrol generator, which I'll rectify to DC and then use that to charge the batteries on the go.

So will a constant charge voltage damage the batteries?

thanks you for all you help