Commit e107c977 authored by Matthew Fernandez's avatar Matthew Fernandez
Browse files

fix: avoid reusing the pivot type during scalarset comparison

When sorting a state by scalarset, there is a point that should have been clear
to me from the beginning but was not: it is invalid to use a nested
scalarset-based member. That is, a scalarset field or scalarset-indexed array
that is within a scalarset-indexed array. The reason for this is that any value
derived from this comparison will be invalidated when the parent array is
reshuffled during sorting. This exhibited as the quicksort algorithm in the
checker traversing out of bounds.

We fix this by tracking when we have used the key scalarset (the pivot) as we
recurse. Whenever we encounter a scalarset-based member, we only use it if
either (a) it is a different scalarset to the pivot or (b) the pivot has not yet
been used.

Github: closes #12 "symmetry reduction"
parent cfd0da09
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment