[Driver] Remove Flags from `cl_Group`. NFC

These flags are redundant since everything in the `cl_Group`
explicitly sets flags anyway. Remove the Flags because they're
confusing and will cause problems with some later refactorings.