From 03805fc376239ddc56838d33b20a031734e3e3a1 Mon Sep 17 00:00:00 2001 From: Karyne Mayer Date: Tue, 26 Jan 2016 08:51:13 +0000 Subject: CryptoPkg: Fix function qsort for non 32-bit machines Although the function qsort receives as an argument a "compare" function which returns an "int", QuickSortWorker (the function used internally by qsort to do its job) receives as an argument a "CompareFunction" which returns an "INTN". In a 32-bit machine, "INTN" is defined as "INT32", which is defined as "int" and everything works well. However, when qsort is compiled for a 64-bit machine, "INTN" is defined as "INT64" and the return values of the compare functions become incompatible ("int" for qsort and "INT64" for QuickSortWorker), causing malfunction. For example, let's assume qsort is being compiled for a 64-bit machine. As stated before, the "compare" function will be returning an "int", and "CompareFunction" will be returning an "INT64". When, for example, the "compare" function (which was passed as an argument to qsort and, then, re-passed as an argument to QuickSortWorker) returns -1 (or 0xffffffff, in a 32-bit integer, its original return type) from inside a call to QuickSortWorker, its return value is interpreted as being an "INT64" value - which turns out to be 4294967295 (or 0x00000000ffffffff, in a 64-bit integer) -, making the function QuickSortWorker to behave unexpectedly. Note that this unexpected (or incorrect) conversion does not happen when casting an "INT32" to an "INT64" directly, but does happen when casting function types. The issue is fixed by changing the return type of SORT_COMPARE (the type of "CompareFunction", used by QuickSortWorker) from "INTN" to "int". This way, both qsort and QuickSortWorker use compatible definitions for their compare functions. Contributed-under: TianoCore Contribution Agreement 1.0 Acked-by: Paulo Alcantara Cavalcanti Signed-off-by: Karyne Mayer Signed-off-by: Rodrigo Dias Correa Signed-off-by: Arthur Crippa Burigo Reviewed-by: Qin Long git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@19748 6f19259b-4bc3-4df7-8a09-765794883524 --- CryptoPkg/Library/BaseCryptLib/SysCall/CrtWrapper.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'CryptoPkg/Library') diff --git a/CryptoPkg/Library/BaseCryptLib/SysCall/CrtWrapper.c b/CryptoPkg/Library/BaseCryptLib/SysCall/CrtWrapper.c index fb446b6..c0ccc0e 100644 --- a/CryptoPkg/Library/BaseCryptLib/SysCall/CrtWrapper.c +++ b/CryptoPkg/Library/BaseCryptLib/SysCall/CrtWrapper.c @@ -2,7 +2,7 @@ C Run-Time Libraries (CRT) Wrapper Implementation for OpenSSL-based Cryptographic Library. -Copyright (c) 2009 - 2011, Intel Corporation. All rights reserved.
+Copyright (c) 2009 - 2016, Intel Corporation. All rights reserved.
This program and the accompanying materials are licensed and made available under the terms and conditions of the BSD License which accompanies this distribution. The full text of the license may be found at @@ -22,7 +22,7 @@ FILE *stdin = NULL; FILE *stdout = NULL; typedef -INTN +int (*SORT_COMPARE)( IN VOID *Buffer1, IN VOID *Buffer2 -- cgit v1.1