The celeb-studded coastal paradise where the average home is worth roughly $3 million has been grappling with rapid erosion.