Not having to update Follow Up Date

I was wondering if it would be possible to make it so when you add another connect to someone after you've set a follow up date that the date doesn't change unless you change it. Right now it takes it away unless you set a new one.