Your buddy was wrong.


As per the definitions section of the Rules of Golf, you have not made a stroke: But a stroke has not been made if the player... accidentally strikes the ball when making a practice swing or while preparing to make a stroke.

That basically means nothing has happened, i.e., no stroke and no penalty either. In fact, if you did play the ball where it landed on the ground and the ball landed in front of the tee markers THEN you would be penalized.