assert: fix deepEqual RangeError: Maximum call stack size exceeded

Fixes: https://github.com/nodejs/node/issues/13314
Refs: https://github.com/nodejs/node/issues/6416

This commit changes semantics of the memos cycles tracker. Before
the change it was used to track all currently wisited nodes of an
object tree, which is a bit shifted from its original intention of
tracking cycles. The change brings intended semantics, by tracking
only objects of the current branch of the object tree.

PR-URL: https://github.com/nodejs/node/pull/13318
Fixes: https://github.com/nodejs/node/issues/13314
Ref: https://github.com/nodejs/node/issues/6416


Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>