gitignore: ignore binaries only in top level

This rules was ignoring also src/* directories.
7 jobs for fix-invalid-hash
in 5 minutes and 16 seconds and was queued for 7 seconds