net: connection: Use NET_CONN_*_PORT_SPEC flag when comparing ports

The NET_CONN_LOCAL/REMOTE_PORT_SPEC flags were set on the connection but
not really used. At the same time, when remote address was being
cleared, only the flag was unset, but the actual port number left
intact. This could lead to false port comparisons when remote address
was cleared from the connection context.

In order to avoid that, use the NET_CONN_*_PORT_SPEC flags to determine
whether to compare local/remote ports instead of checking the port value
right away (which may not be valid anymore if the flag is unset).

Also align with MISRA rules when comparing flags in other cases.

Signed-off-by: Robert Lubos <robert.lubos@nordicsemi.no>