When should I NOT worry about respawn outlines as spy?

Spy players on youtube don't seem to be disguising as a teammate/undisguising as often as I am so how do I know when I shouldn't worry about anyone seeing me through walls?