Bluetooth: Host: Fix wrong addr for PA term when using adv list

If using the advertiser list when syncing to a PA, then the
address address needs to be updated for the PA sync object.
This is a small optimization to avoid doing a copy when
the address and SID is already set.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>