Sean Penn’s early brush with fame found him aggressively confronting paparazzi eager to snap his mug.

This week, Penn raged against a fan whose crime was stealthily snapping a picture of the Oscar winner with a smart phone.

Penn was at the Lobby Bar at the St. Regis Hotel in the Financial District … when he saw a flash go off from another patron’s cell phone and realized the guy was trying to take a photo. 

A witness at the bar tells TMZ … Penn bolted after the guy (who wasn’t a paparazzo) … and screamed at him, “Do we look like f**king zoo animals?”

Penn — who was in town to speak at a sales conference — also grabbed the man’s phone and slammed it to the ground.

Penn is caught on tape shouting, “I’ll make you eat that phone … get the fuck out of here.”