llvm-undname: Make demangling of MD5 names more robust

Demangler::parse() for MD5 names would:

1. Put all remaining text into the MD5 name sight unseen
2. Not modify MangledName

This meant that if the demangler recursively called parse() (e.g. in
demangleLocallyScopedNamePiece()), every recursive call that started on
an MD5 name would add all remaining bytes to the output buffer but
only advance the input by a byte.  For valid inputs, MD5 types are
never (well, see comments for 2 exceptions) nested, but for invalid
input this could cause memory use quadratic in the input size.

llvm-svn: 361744

1 files changed, 25 insertions, 3 deletions

diff --git a/llvm/lib/Demangle/MicrosoftDemangle.cpp b/llvm/lib/Demangle/MicrosoftDemangle.cpp
index 041d327..c1e6e14 100644
--- a/llvm/lib/Demangle/MicrosoftDemangle.cpp
+++ b/llvm/lib/Demangle/MicrosoftDemangle.cpp
@@ -747,16 +747,38 @@ SymbolNode *Demangler::demangleDeclarator(StringView &MangledName) {
 
 // Parser entry point.
 SymbolNode *Demangler::parse(StringView &MangledName) {
-  // We can't demangle MD5 names, just output them as-is.
-  // Also, MSVC-style mangled symbols must start with '?'.
   if (MangledName.startsWith("??@")) {
     // This is an MD5 mangled name.  We can't demangle it, just return the
     // mangled name.
+    // An MD5 mangled name is ??@ followed by 32 characters and a terminating @.
+    size_t MD5Last = MangledName.find('@', strlen("??@"));
+    if (MD5Last == StringView::npos) {
+      Error = true;
+      return nullptr;
+    }
+    const char* Start = MangledName.begin();
+    MangledName = MangledName.dropFront(MD5Last + 1);
+
+    // There are two additional special cases for MD5 names:
+    // 1. For complete object locators where the object name is long enough
+    //    for the object to have an MD5 name, the complete object locator is
+    //    called ??@...@??_R4@ (with a trailing "??_R4@" instead of the usual
+    //    leading "??_R4". This is handled here.
+    // 2. For catchable types, in versions of MSVC before 2015 (<1900) or after
+    //    2017.2 (>= 1914), the catchable type mangling is _CT??@...@??@...@8
+    //    instead of_CT??@...@8 with just one MD5 name. Since we don't yet
+    //    demangle catchable types anywhere, this isn't handled for MD5 names
+    //    either.
+    MangledName.consumeFront("??_R4@");
+
+    StringView MD5(Start, MangledName.begin());
     SymbolNode *S = Arena.alloc<SymbolNode>(NodeKind::Md5Symbol);
-    S->Name = synthesizeQualifiedName(Arena, MangledName);
+    S->Name = synthesizeQualifiedName(Arena, MD5);
+
     return S;
   }
 
+  // MSVC-style mangled symbols must start with '?'.
   if (!MangledName.startsWith('?')) {
     Error = true;
     return nullptr;