Having a server-log running down the screen for hours at a time is serious motivation to tighten things up. Just wrote a snippet to include in most of my server blocks so that they each deliver a valid robots.txt.