Skip opening and statting a file if readdir says it's not a dir or regular.

While readdir is notorious to return DT_UNKNOWN, it works well on btrfs and
devtmpfs, which are two filesystems which are most interesting for us.