Experimental 'genvslite' WIP. (#11049)

* Capture all compile args from the first round of ninja backend generation for all languages used in building the targets so that these args, defines, and include paths can be applied to the .vcxproj's intellisense fields for all buildtypes/configurations.

Solution generation is now set up for mutiple build configurations (buildtypes) when using '--genvslite'.

All generated vcxprojs invoke the same high-level meson compile to build all targets;  there's no selective target building (could add this later).  Related to this, we skip pointlessly generating vcxprojs for targets that aren't buildable (BuildTarget-derived), which aren't of interest to the user anyway.

When using --genvslite, no longer inject '<ProjectReference ...>' dependencies on which a generated .vcxproj depends because that imposes a forced visual studio build dependency, which we don't want, since we're essentially bypassing VS's build in favour of running 'meson compile ...'.

When populating the vcxproj's shared intellisense defines, include paths, and compiler options fields, we choose the most frequent src file language, since this means more project src files can simply reference the project shared fields and fewer files of non-primary language types need to populate their full set of intellisense fields.  This makes for smaller .vcxproj files.

Paths for generated source/header/etc files, left alone, would be added to solution projects relative to the '..._vs' build directory, where they're never generated;  they're generated under the respective '..._[debug/opt/release]' ninja build directories that correspond to the solution build configuration. Although VS doesn't allow conditional src/header listings in vcxprojs (at least not in a simple way that I'm aware of), we can ensure these generated sources get adjusted to at least reference locations under one of the concrete build directories (I've chosen '..._debug') under which they will be generated.

Testing with --genvslite has revealed that, in some cases, the presence of 'c:\windows\system32;c:\windows' on the 'Path' environment variable (via the make-style project's ExecutablePath element) is critical to getting the 'meson compile ...' build to succeed.  Not sure whether this is some 'find and guess' implicit defaults behaviour within meson or within the MSVC compiler that some projects may rely on. Feels weird but not sure of a better solution than forcibly adding these to the Path environment variable (the Executable Path property of the project).

Added a new windows-only test to windowstests.py ('test_genvslite') to exercise the --genvslite option along with checking that the 'msbuild' command invokes the 'meson compile ...' of the build-type-appropriate-suffixed temporary build dir and checks expected program output.

Check and report error if user specifies a non-ninja backend with a 'genvslite' setup, since that conflicts with the stated behaviour of genvslite.  Also added this test case to 'WindowsTests.test_genvslite'

I had problems tracking down some problematic environment variable behaviour, which appears to need a work-around. See further notes on VSINSTALLDIR, in windowstests.py, test_genvslite.
'meson setup --help' clearly states that positional arguments are ... [builddir] [sourcedir].  However, BasePlatformTests.init(...) was passing these in the order [sourcedir] [builddir].  This was producing failures, saying, "ERROR: Neither directory contains a build file meson.build." but when using the correct ordering, setup now succeeds.

Changed regen, run_tests, and run_install utility projects to be simpler makefile projects instead, with commands to invoke the appropriate '...meson.py --internal regencheck ...' (or install/test) on the '[builddir]_[buildtype]' as appropriate for the curent VS configuration.  Also, since the 'regen.vcxproj' utility didn't work correctly with '--genvslite' setup build dirs, and getting it to fully work would require more non-trivial intrusion into new parts of meson (i.e. '--internal regencheck', '--internal regenerate', and perhaps also 'setup --reconfigure'), for now, the REGEN project is replaced with a simpler, lighter-weight RECONFIGURE utility proj, which is unlinked from any solution build dependencies and which simply runs 'meson setup --reconfigure [builddir]_[buildtype] [srcdir]' on each of the ninja-backend build dirs for each buildtype.
Yes, although this will enable the building/compiling to be correctly configured, it can leave the solution/vcxprojs stale and out-of-date, it's simple for the user to 'meson setup --genvslite ...' to fully regenerate an updated, correct solution again. However, I've noted this down as a 'fixme' to consider implementing the full regen behaviour for the genvslite case.

* Review feedback changes -
- Avoid use of 'captured_compile_args_per_buildtype_and_target' as an 'out' param.
- Factored a little msetup.py, 'run(...)' macro/looping setup steps, for genvslite, out into a 'run_genvslite_setup' func.

* Review feedback:  Fixed missing spaces between multi-line strings.

* 'backend_name' assignment gets immediately overwritten in 'genvslite' case so moved it into else/non-genvslite block.

* Had to bump up 'test cases/unit/113 genvslites/...' up to 114; it collided with a newly added test dir again.

* Changed validation of 'capture' and 'captured_compile_args_...' to use MesonBugException instead of MesonException.

* Changed some function param and closing brace indentation.

2 files changed, 91 insertions, 3 deletions

diff --git a/unittests/baseplatformtests.py b/unittests/baseplatformtests.py
index ea95b15..4b16e7d 100644
--- a/unittests/baseplatformtests.py
+++ b/unittests/baseplatformtests.py
@@ -207,7 +207,8 @@ class BasePlatformTests(TestCase):
             extra_args = []
         if not isinstance(extra_args, list):
             extra_args = [extra_args]
-        args = [srcdir, self.builddir]
+        build_and_src_dir_args = [self.builddir, srcdir]
+        args = []
         if default_args:
             args += ['--prefix', self.prefix]
             if self.libdir:
@@ -219,7 +220,7 @@ class BasePlatformTests(TestCase):
         self.privatedir = os.path.join(self.builddir, 'meson-private')
         if inprocess:
             try:
-                returncode, out, err = run_configure_inprocess(['setup'] + self.meson_args + args + extra_args, override_envvars)
+                returncode, out, err = run_configure_inprocess(['setup'] + self.meson_args + args + extra_args + build_and_src_dir_args, override_envvars)
             except Exception as e:
                 if not allow_fail:
                     self._print_meson_log()
@@ -245,7 +246,7 @@ class BasePlatformTests(TestCase):
                     raise RuntimeError('Configure failed')
         else:
             try:
-                out = self._run(self.setup_command + args + extra_args, override_envvars=override_envvars, workdir=workdir)
+                out = self._run(self.setup_command + args + extra_args + build_and_src_dir_args, override_envvars=override_envvars, workdir=workdir)
             except SkipTest:
                 raise SkipTest('Project requested skipping: ' + srcdir)
             except Exception:
diff --git a/unittests/windowstests.py b/unittests/windowstests.py
index c81d924..36a1f3f 100644
--- a/unittests/windowstests.py
+++ b/unittests/windowstests.py
@@ -184,6 +184,93 @@ class WindowsTests(BasePlatformTests):
             # to the right reason).
             return
         self.build()
+    
+    @skipIf(is_cygwin(), 'Test only applicable to Windows')
+    def test_genvslite(self):
+        # The test framework itself might be forcing a specific, non-ninja backend across a set of tests, which
+        # includes this test. E.g. -
+        #   > python.exe run_unittests.py --backend=vs WindowsTests
+        # Since that explicitly specifies a backend that's incompatible with (and essentially meaningless in
+        # conjunction with) 'genvslite', we should skip further genvslite testing.
+        if self.backend is not Backend.ninja:
+            raise SkipTest('Test only applies when using the Ninja backend')
+
+        testdir = os.path.join(self.unit_test_dir, '114 genvslite')
+
+        env = get_fake_env(testdir, self.builddir, self.prefix)
+        cc = detect_c_compiler(env, MachineChoice.HOST)
+        if cc.get_argument_syntax() != 'msvc':
+            raise SkipTest('Test only applies when MSVC tools are available.')
+
+        # We want to run the genvslite setup. I.e. -
+        #    meson setup --genvslite vs2022 ...
+        # which we should expect to generate the set of _debug/_debugoptimized/_release suffixed
+        # build directories.  Then we want to check that the solution/project build hooks (like clean,
+        # build, and rebuild) end up ultimately invoking the 'meson compile ...' of the appropriately
+        # suffixed build dir, for which we need to use 'msbuild.exe'
+
+        # Find 'msbuild.exe'
+        msbuildprog = ExternalProgram('msbuild.exe')
+        self.assertTrue(msbuildprog.found(), msg='msbuild.exe not found')
+
+        # Setup with '--genvslite ...'
+        self.new_builddir()
+
+        # Firstly, we'd like to check that meson errors if the user explicitly specifies a non-ninja backend
+        # during setup.
+        with self.assertRaises(subprocess.CalledProcessError) as cm:
+            self.init(testdir, extra_args=['--genvslite', 'vs2022', '--backend', 'vs'])
+        self.assertIn("specifying a non-ninja backend conflicts with a 'genvslite' setup", cm.exception.stdout)
+
+        # Wrap the following bulk of setup and msbuild invocation testing in a try-finally because any exception,
+        # failure, or success must always clean up any of the suffixed build dir folders that may have been generated.
+        try:
+            # Since this 
+            self.init(testdir, extra_args=['--genvslite', 'vs2022'])
+            # We need to bear in mind that the BasePlatformTests framework creates and cleans up its own temporary
+            # build directory.  However, 'genvslite' creates a set of suffixed build directories which we'll have
+            # to clean up ourselves. See 'finally' block below.
+
+            # We intentionally skip the -
+            #   self.build()
+            # step because we're wanting to test compilation/building through the solution/project's interface.
+
+            # Execute the debug and release builds through the projects 'Build' hooks
+            genvslite_vcxproj_path = str(os.path.join(self.builddir+'_vs', 'genvslite@exe.vcxproj'))
+            # This use-case of invoking the .sln/.vcxproj build hooks, not through Visual Studio itself, but through
+            # 'msbuild.exe', in a VS tools command prompt environment (e.g. "x64 Native Tools Command Prompt for VS 2022"), is a
+            # problem:  Such an environment sets the 'VSINSTALLDIR' variable which, mysteriously, has the side-effect of causing
+            # the spawned 'meson compile' command to fail to find 'ninja' (and even when ninja can be found elsewhere, all the
+            # compiler binaries that ninja wants to run also fail to be found).  The PATH environment variable in the child python
+            # (and ninja) processes are fundamentally stripped down of all the critical search paths required to run the ninja
+            # compile work ... ONLY when 'VSINSTALLDIR' is set;  without 'VSINSTALLDIR' set, the meson compile command does search
+            # for and find ninja (ironically, it finds it under the path where VSINSTALLDIR pointed!).
+            # For the above reason, this testing works around this bizarre behaviour by temporarily removing any 'VSINSTALLDIR'
+            # variable, prior to invoking the builds -
+            current_env = os.environ.copy()
+            current_env.pop('VSINSTALLDIR', None)
+            subprocess.check_call(
+                ['msbuild', '-target:Build', '-property:Configuration=debug', genvslite_vcxproj_path],
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+                env=current_env)
+            subprocess.check_call(
+                ['msbuild', '-target:Build', '-property:Configuration=release', genvslite_vcxproj_path],
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+                env=current_env)
+
+            # Check this has actually built the appropriate exes
+            output_debug = subprocess.check_output(str(os.path.join(self.builddir+'_debug', 'genvslite.exe')))
+            self.assertEqual( output_debug, b'Debug\r\n' )
+            output_release = subprocess.check_output(str(os.path.join(self.builddir+'_release', 'genvslite.exe')))
+            self.assertEqual( output_release, b'Non-debug\r\n' )
+
+        finally:
+            # Clean up our special suffixed temporary build dirs
+            suffixed_build_dirs = glob(self.builddir+'_*', recursive=False)
+            for build_dir in suffixed_build_dirs:
+                shutil.rmtree(build_dir)
 
     def test_install_pdb_introspection(self):
         testdir = os.path.join(self.platform_test_dir, '1 basic')