It would make it hard to detect if someone had been in the house. Like the opposite of when detectives figure out if someone had just been at the scene of a crime? Leaving a drink that's still warm...

I've thought about this too much.

Yes. A phone is a privilege and should not compromise a persons capabilities for other task's. Some people get tunnel vision and forget about other matters.