Signed-off-by: Kevin Albertson <kevin.albertson@mongodb.com>

This reverts commit 6bacf922.

This reverts commit 95ed9344.

* Fix: mongoc_gridfs_bucket_find not respecting 'opts'

Fix: CDRIVER-3164 and CDRIVER-4049

GridFSBucket.find must forward the 'opts' to the underlying
collection query. Also permit 'opts' to be NULL.

* Sort by metadata instead of uploadDate

Synced with mongodb/specifications@44f83106033a609f2af40e3e4d99852a03096ba5

This brings the tests up-to-date with mongodb/specifications@b07c26dc.

* Sync command monitoring spec tests

This brings the spec tests up to date with mongodb/specifications#84ac002b

* Add support for 1.5 unified test format schema

This adds support for the "observeSensitiveCommands" monitoring option, as well as the new "auth" runOnRequirement.

* Redact sensitive commands for APM

* Test redaction of replies for sensitive commands

* Force redaction of replies for sensitive commands

Previously, the driver would not redact the reply to a hello command with speculative authentication unless the reply also was sensitive. As this makes the test completely useless, we've decided to always require redaction of replies when the command was redacted.

* Document is_redacted and force_redaction arguments to APM initialisers

* Make redaction helpers static

* Print correct error message when DNS resolution fails

* Drop duplicate case value

Synced with mongodb/specifications@fc21cb7330925647e7961c324f1594096025c17e

Note: this also pulls in test changes relevant to aggregate let support (CDRIVER-4010)

* CDRIVER-3991 Add versioned API connection examples

* Use mongoc_client_new in public examples

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>

* Test versioned API samples on 4.9+ only

* Free client and server_api object in tests

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>

* CDRIVER-3912 Run OCSP tests against Ubuntu 20.04
* CDRIVER-3946 Run Evg tasks on Ubuntu 18.04 and RHEL 8.1 (power8)
* CDRIVER-4022 Skip invalid_wc_server_error test on mongos

This implements the second prose test from the CRUD spec and relocates the first prose test, which was originally added in CDRIVER-3575.

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>

* Handle helloOk flag in server description

* Refactor handshake command naming

* Use hello command after seeing helloOk in legacy hello response

* Fix 5.0 test

* Update SDAM spec tests to latest version

* Reset hello_ok in mongoc_server_description_reset only

* Don't update initial server description in server monitor

* Always build legacy_hello_cmd with legacy hello command

* Test fallback to legacy hello after connection is closed

* Update wrong comment

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>

* Don't modify legacy_hello_cmd when using versioned API

* Lock topology mutex before obtaining server description

* Don't update hello_ok when declaring API version

This is not necessary since we don't rely on hello_ok if an API version was declared

This partially reverts commit 7d58f04773c53b78043cb4cb89f07b8cb64a17a5.

* Remove early return when reconciling scanner nodes

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>

* CDRIVER-4026 Specify 5.0 cursor behaviour

Revert "Fix batchSize behaviour for getMore (#793)"

This reverts commit 5cd215b2.

The spec tests in this commit are synced to commit 9da20b4b from the mongodb/specifications repository.

* Restore usage of assert_match_bson

* CDRIVER-3895: Update CRUD, transactions, and unified spec tests

Also replaces CRUD legacy/v2 tests with unified equivalents for CDRIVER-3960.

Synced with mongodb/specifications@a124e21be89cbfc78606725c6ed1fa00dc992777

* Skip unified CRUD tests related to CDRIVER-3630

* Ensure upsertedIds is always present in BulkWriteResult

* Allow array or document for updateOne/Many update arg

* Remove dot/dollar restrictions for insert/replace validation

This adds top-level key validation to _mongoc_validate_replace, similar to what existed in _mongoc_validate_update.

* Remove redundant replace doc validation for legacy updates

Replacement docs will already be validated by _mongoc_validate_replace before this function is reached.

This adds logic to ensure "q" and "u" documents are present, similar to what existed in _mongoc_write_command_delete_legacy.

* Test success and failure for legacy update code path

The failure code path is testing that arguments are still validated by mongoc-util.c, despite replacement validation being removed in a previous commit.

* Revert "Skip tests with dots and dollars in field names on 5.0+ (#795)"

This reverts commit bd014ee7

.

* Revise bulk and collection tests for allowing dots/dollars

Empty keys can be used as a reliable BSON validation error, since insert, replace, and update all specify BSON_VALIDATE_EMPTY_KEYS in their vflags. Error message expectations are adjusted accordingly.

An _id document with a dollar-prefixed key is also used as a reliable server-side validation error.

* Allow methods in bulk_op_append to fail so expectError can be evaluated

* Skip unified tests due to schema version and outstanding issues

Allow entire files to be skipped to preempt schema version errors.

* Ensure WC is set on database and collection entities

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>

* CDRIVER-3970: Remove DBRef validation from Extended JSON parser

Documents that resemble but do not conform to a DBRef should be left as-is.

Syncs BSON corpus tests with mongodb/specifications@a274fbfddfd39f5f3d00712c422e4f1b062d783c

Skips were added for additional corpus tests unrelated to this spec change.

Use hex escaping instead of unicode for MSVC compatibility

Co-authored-by: Kevin Albertson <kevin.albertson@10gen.com>
Co-authored-by: Fermín Galán Márquez <fgalan@users.noreply.github.com>

* Remove deprecated terminology for legacy hello

* Remove deprecated terminology for secondaryOk

* Remove deprecated terminology for allow/deny list

* Remove deprecated terminology for notPrimary

* Update terminology in spec tests

* Fix issues from code review

* Remove more language in docs

* Undo accidental change to hello

* Use secondaryOk when printing query flags

* Fix wrong variable name

* Remove deprecated terminology in tests

* Remove branch name from installation docs

* Clarify build examples in readme

* Use hello command for handshake with versioned API

* Support handling hello command responses

* Deprecate legacy hello terminology

* Test handling hello command responses

* Refactor internal methods to use hello terminology

* Change mock server to also handle hello

* Replace deprecated terminology in test expectations

* Use hello in tests

* Fix speculative auth responder

* Fix legacy hello command usage

* Add test for behaviour when server does not support hello command

* Use legacy hello in tests to avoid failures on old server versions

* Update SDAM spec tests

* Fix memory leak

* Fix wrong command expectations for handshake

* Refactor handshake autoresponders

* Fix wrong comment style

* Rename mongoc_server_description_hello_response

* Use method to retrieve hello response

* Improve deprecation declaration

* Rename methods for hello callbacks

* Fix various test issues

* Rename hello_response method

* CDRIVER-3917 route all test client and client_pool creation through the test framework

* small fixes

* add missing include

* rename pool method to match client method

* Refactor test framework client factories to accept API version

* Revert change in mongoc-cache.c

Despite the #include, the linker does not like this:
tests/test-mongoc-cache.c.o: In function `ping':
src/libmongoc/tests/test-mongoc-cache.c:45: undefined reference to `test_framework_client_new'

* Run all tests with requireApiVersion=true

* Fix large_ismaster test

* Don't set server api when client can't be created

* Don't use test clients in tests specifically testing server API behaviour

* Print default API version in test output

* Set requireApiVersion option earlier

* Consider apiVersion field when assembling large bulk operation

* Fix wrong comment wording

Co-authored-by: Benjamin Rewis <32186188+benjirewis@users.noreply.github.com>

* Fix inaccurate variable name

* Remove obsolete todo

* Document MONGODB_API_VERSION environment variable in contributing docs

* Document new test methods

* Fix tests with acceptApiVersion2

* Update newly added test

Co-authored-by: samantharitter <samantha.ritter@10gen.com>
Co-authored-by: Benjamin Rewis <32186188+benjirewis@users.noreply.github.com>

* CDRIVER-3934 Correctly handle network errors during speculative authentication

* Assert request before getting memory failure

* Fix code review feedback

* Fix wrong directory for package builds

* CDRIVER-3937 Remove usages of xtrace in build scripts

If an unacknowledged write concern is used as an option for a
mongoc_find_and_modify_with_opts, the output reply document is
always empty.

Syncs transaction spec tests with mongodb/specifications@ec326862be443d11f0233cee2de32a95751144df. One change was missed in 2d7938aa.