Hey Alex,
I say you should use a controller. First, an "on/off" switch? You need a contactor. You switch 48 volts of battery across that motor and you'll draw hundreds of amps. That will fry a switch. Also, across the line starting like that will pull wheelies and break things. Motor controllers have a current limit and ramp-up features which are there for a reason.
Next, is that golf car motor series or SepEx? If it is a series motor and you run without a controller, try to shift on the fly, when you disengage the motor it will self destruct.
Next, does the motor have a drive end bearing? Most golf car motors pilot on the axle bearing.
Have fun with it, but be careful.
major