libstdc++: Fix unsigned wraparound in codecvt::do_length [PR105857]

When the max argument to std::codecvt<wchar_t, char, mbstate_t>::length
is SIZE_MAX/4+1 or greater the multiplication with sizeof(wchar_t) will
wrap to a small value, and the alloca call will have a buffer that's
smaller than requested. The call to mbsnrtowcs then has a buffer that is
smaller than the value passed as the buffer length. When libstdc++.so is
built with -D_FORTIFY_SOURCE=3 the mismatched buffer and length will get
detected and will abort inside Glibc.

When it doesn't abort, there's no buffer overflow because Glibc's
mbsnrtowcs has the same len * sizeof(wchar_t) calculation to determine
the size of the buffer in bytes, and that will wrap to the same small
number as the alloca argument. So luckily Glibc agrees with the caller
about the real size of the buffer, and won't overflow it.

Even when the max argument isn't large enough to wrap, it can still be
much too large to safely pass to alloca, so we should limit that. We
already have a loop that processes chunks so that we can handle null
characters in the middle of the input. If we limit the alloca buffer to
4kB then we'll just loop each time that buffer is filled.

libstdc++-v3/ChangeLog:

	PR libstdc++/105857
	* config/locale/dragonfly/codecvt_members.cc (do_length): Limit
	size of alloca buffer to 4k.
	* config/locale/gnu/codecvt_members.cc (do_length): Likewise.
	* testsuite/22_locale/codecvt/length/wchar_t/105857.cc: New
	test.

0 files changed, 0 insertions, 0 deletions