Return error from boot_find_key instead of asserting

When the hash key length parameter is invalid (bigger than allowed) just
return an error for the caller routines; this will work like a not found
key and allows the removal of assert() code in this module.

Signed-off-by: Fabio Utzig <utzig@apache.org>