New Jersey native made a fortune from tourist shops but found he didn’t crave the billionaire lifestyle.