Abolitionists had something to hide.
Germans protecting jewish escapees had something to hide.
To say that “you have nothing to fear if you have nothing to hide” is inherently flawed, because it assumes that having something to hide is wrong.