diff options
author | Iain Buclaw <ibuclaw@gcc.gnu.org> | 2018-10-28 19:51:47 +0000 |
---|---|---|
committer | Iain Buclaw <ibuclaw@gcc.gnu.org> | 2018-10-28 19:51:47 +0000 |
commit | b4c522fabd0df7be08882d2207df8b2765026110 (patch) | |
tree | b5ffc312b0a441c1ba24323152aec463fdbe5e9f /libphobos/src/std/digest | |
parent | 01ce9e31a02c8039d88e90f983735104417bf034 (diff) | |
download | gcc-b4c522fabd0df7be08882d2207df8b2765026110.zip gcc-b4c522fabd0df7be08882d2207df8b2765026110.tar.gz gcc-b4c522fabd0df7be08882d2207df8b2765026110.tar.bz2 |
Add D front-end, libphobos library, and D2 testsuite.
ChangeLog:
* Makefile.def (target_modules): Add libphobos.
(flags_to_pass): Add GDC, GDCFLAGS, GDC_FOR_TARGET and
GDCFLAGS_FOR_TARGET.
(dependencies): Make libphobos depend on libatomic, libbacktrace
configure, and zlib configure.
(language): Add language d.
* Makefile.in: Rebuild.
* Makefile.tpl (BUILD_EXPORTS): Add GDC and GDCFLAGS.
(HOST_EXPORTS): Add GDC.
(POSTSTAGE1_HOST_EXPORTS): Add GDC and GDC_FOR_BUILD.
(BASE_TARGET_EXPORTS): Add GDC.
(GDC_FOR_BUILD, GDC, GDCFLAGS): New variables.
(GDC_FOR_TARGET, GDC_FLAGS_FOR_TARGET): New variables.
(EXTRA_HOST_FLAGS): Add GDC.
(STAGE1_FLAGS_TO_PASS): Add GDC.
(EXTRA_TARGET_FLAGS): Add GDC and GDCFLAGS.
* config-ml.in: Treat GDC and GDCFLAGS like other compiler/flag
environment variables.
* configure: Rebuild.
* configure.ac: Add target-libphobos to target_libraries. Set and
substitute GDC_FOR_BUILD and GDC_FOR_TARGET.
config/ChangeLog:
* multi.m4: Set GDC.
gcc/ChangeLog:
* Makefile.in (tm_d_file_list, tm_d_include_list): New variables.
(TM_D_H, D_TARGET_DEF, D_TARGET_H, D_TARGET_OBJS): New variables.
(tm_d.h, cs-tm_d.h, default-d.o): New rules.
(d/d-target-hooks-def.h, s-d-target-hooks-def-h): New rules.
(s-tm-texi): Also check timestamp on d-target.def.
(generated_files): Add TM_D_H and d-target-hooks-def.h.
(build/genhooks.o): Also depend on D_TARGET_DEF.
* config.gcc (tm_d_file, d_target_objs, target_has_targetdm): New
variables.
* config/aarch64/aarch64-d.c: New file.
* config/aarch64/aarch64-linux.h (GNU_USER_TARGET_D_CRITSEC_SIZE):
Define.
* config/aarch64/aarch64-protos.h (aarch64_d_target_versions): New
prototype.
* config/aarch64/aarch64.h (TARGET_D_CPU_VERSIONS): Define.
* config/aarch64/t-aarch64 (aarch64-d.o): New rule.
* config/arm/arm-d.c: New file.
* config/arm/arm-protos.h (arm_d_target_versions): New prototype.
* config/arm/arm.h (TARGET_D_CPU_VERSIONS): Define.
* config/arm/linux-eabi.h (EXTRA_TARGET_D_OS_VERSIONS): Define.
* config/arm/t-arm (arm-d.o): New rule.
* config/default-d.c: New file.
* config/glibc-d.c: New file.
* config/gnu.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/i386/i386-d.c: New file.
* config/i386/i386-protos.h (ix86_d_target_versions): New prototype.
* config/i386/i386.h (TARGET_D_CPU_VERSIONS): Define.
* config/i386/linux-common.h (EXTRA_TARGET_D_OS_VERSIONS): Define.
(GNU_USER_TARGET_D_CRITSEC_SIZE): Define.
* config/i386/t-i386 (i386-d.o): New rule.
* config/kfreebsd-gnu.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/kopensolaris-gnu.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/linux-android.h (ANDROID_TARGET_D_OS_VERSIONS): Define.
* config/linux.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/mips/linux-common.h (EXTRA_TARGET_D_OS_VERSIONS): Define.
* config/mips/mips-d.c: New file.
* config/mips/mips-protos.h (mips_d_target_versions): New prototype.
* config/mips/mips.h (TARGET_D_CPU_VERSIONS): Define.
* config/mips/t-mips (mips-d.o): New rule.
* config/powerpcspe/linux.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/powerpcspe/linux64.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/powerpcspe/powerpcspe-d.c: New file.
* config/powerpcspe/powerpcspe-protos.h (rs6000_d_target_versions):
New prototype.
* config/powerpcspe/powerpcspe.c (rs6000_output_function_epilogue):
Support GNU D by using 0 as the language type.
* config/powerpcspe/powerpcspe.h (TARGET_D_CPU_VERSIONS): Define.
* config/powerpcspe/t-powerpcspe (powerpcspe-d.o): New rule.
* config/riscv/riscv-d.c: New file.
* config/riscv/riscv-protos.h (riscv_d_target_versions): New
prototype.
* config/riscv/riscv.h (TARGET_D_CPU_VERSIONS): Define.
* config/riscv/t-riscv (riscv-d.o): New rule.
* config/rs6000/linux.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/rs6000/linux64.h (GNU_USER_TARGET_D_OS_VERSIONS): Define.
* config/rs6000/rs6000-d.c: New file.
* config/rs6000/rs6000-protos.h (rs6000_d_target_versions): New
prototype.
* config/rs6000/rs6000.c (rs6000_output_function_epilogue):
Support GNU D by using 0 as the language type.
* config/rs6000/rs6000.h (TARGET_D_CPU_VERSIONS): Define.
* config/rs6000/t-rs6000 (rs6000-d.o): New rule.
* config/s390/s390-d.c: New file.
* config/s390/s390-protos.h (s390_d_target_versions): New prototype.
* config/s390/s390.h (TARGET_D_CPU_VERSIONS): Define.
* config/s390/t-s390 (s390-d.o): New rule.
* config/sparc/sparc-d.c: New file.
* config/sparc/sparc-protos.h (sparc_d_target_versions): New
prototype.
* config/sparc/sparc.h (TARGET_D_CPU_VERSIONS): Define.
* config/sparc/t-sparc (sparc-d.o): New rule.
* config/t-glibc (glibc-d.o): New rule.
* configure: Regenerated.
* configure.ac (tm_d_file): New variable.
(tm_d_file_list, tm_d_include_list, d_target_objs): Add substitutes.
* doc/contrib.texi (Contributors): Add self for the D frontend.
* doc/frontends.texi (G++ and GCC): Mention D as a supported language.
* doc/install.texi (Configuration): Mention libphobos as an option for
--enable-shared. Mention d as an option for --enable-languages.
(Testing): Mention check-d as a target.
* doc/invoke.texi (Overall Options): Mention .d, .dd, and .di as file
name suffixes. Mention d as a -x option.
* doc/sourcebuild.texi (Top Level): Mention libphobos.
* doc/standards.texi (Standards): Add section on D language.
* doc/tm.texi: Regenerated.
* doc/tm.texi.in: Add @node for D language and ABI, and @hook for
TARGET_CPU_VERSIONS, TARGET_D_OS_VERSIONS, and TARGET_D_CRITSEC_SIZE.
* dwarf2out.c (is_dlang): New function.
(gen_compile_unit_die): Use DW_LANG_D for D.
(declare_in_namespace): Return module die for D, instead of adding
extra declarations into the namespace.
(gen_namespace_die): Generate DW_TAG_module for D.
(gen_decl_die): Handle CONST_DECLSs for D.
(dwarf2out_decl): Likewise.
(prune_unused_types_walk_local_classes): Handle DW_tag_interface_type.
(prune_unused_types_walk): Handle DW_tag_interface_type same as other
kinds of aggregates.
* gcc.c (default_compilers): Add entries for .d, .dd and .di.
* genhooks.c: Include d/d-target.def.
gcc/po/ChangeLog:
* EXCLUDES: Add sources from d/dmd.
gcc/testsuite/ChangeLog:
* gcc.misc-tests/help.exp: Add D to option descriptions check.
* gdc.dg/asan/asan.exp: New file.
* gdc.dg/asan/gdc272.d: New test.
* gdc.dg/compilable.d: New test.
* gdc.dg/dg.exp: New file.
* gdc.dg/gdc254.d: New test.
* gdc.dg/gdc260.d: New test.
* gdc.dg/gdc270a.d: New test.
* gdc.dg/gdc270b.d: New test.
* gdc.dg/gdc282.d: New test.
* gdc.dg/gdc283.d: New test.
* gdc.dg/imports/gdc170.d: New test.
* gdc.dg/imports/gdc231.d: New test.
* gdc.dg/imports/gdc239.d: New test.
* gdc.dg/imports/gdc241a.d: New test.
* gdc.dg/imports/gdc241b.d: New test.
* gdc.dg/imports/gdc251a.d: New test.
* gdc.dg/imports/gdc251b.d: New test.
* gdc.dg/imports/gdc253.d: New test.
* gdc.dg/imports/gdc254a.d: New test.
* gdc.dg/imports/gdc256.d: New test.
* gdc.dg/imports/gdc27.d: New test.
* gdc.dg/imports/gdcpkg256/package.d: New test.
* gdc.dg/imports/runnable.d: New test.
* gdc.dg/link.d: New test.
* gdc.dg/lto/lto.exp: New file.
* gdc.dg/lto/ltotests_0.d: New test.
* gdc.dg/lto/ltotests_1.d: New test.
* gdc.dg/runnable.d: New test.
* gdc.dg/simd.d: New test.
* gdc.test/gdc-test.exp: New file.
* lib/gdc-dg.exp: New file.
* lib/gdc.exp: New file.
libphobos/ChangeLog:
* Makefile.am: New file.
* Makefile.in: New file.
* acinclude.m4: New file.
* aclocal.m4: New file.
* config.h.in: New file.
* configure: New file.
* configure.ac: New file.
* d_rules.am: New file.
* libdruntime/Makefile.am: New file.
* libdruntime/Makefile.in: New file.
* libdruntime/__entrypoint.di: New file.
* libdruntime/__main.di: New file.
* libdruntime/gcc/attribute.d: New file.
* libdruntime/gcc/backtrace.d: New file.
* libdruntime/gcc/builtins.d: New file.
* libdruntime/gcc/config.d.in: New file.
* libdruntime/gcc/deh.d: New file.
* libdruntime/gcc/libbacktrace.d.in: New file.
* libdruntime/gcc/unwind/arm.d: New file.
* libdruntime/gcc/unwind/arm_common.d: New file.
* libdruntime/gcc/unwind/c6x.d: New file.
* libdruntime/gcc/unwind/generic.d: New file.
* libdruntime/gcc/unwind/package.d: New file.
* libdruntime/gcc/unwind/pe.d: New file.
* m4/autoconf.m4: New file.
* m4/druntime.m4: New file.
* m4/druntime/cpu.m4: New file.
* m4/druntime/libraries.m4: New file.
* m4/druntime/os.m4: New file.
* m4/gcc_support.m4: New file.
* m4/gdc.m4: New file.
* m4/libtool.m4: New file.
* src/Makefile.am: New file.
* src/Makefile.in: New file.
* src/libgphobos.spec.in: New file.
* testsuite/Makefile.am: New file.
* testsuite/Makefile.in: New file.
* testsuite/config/default.exp: New file.
* testsuite/lib/libphobos-dg.exp: New file.
* testsuite/lib/libphobos.exp: New file.
* testsuite/testsuite_flags.in: New file.
From-SVN: r265573
Diffstat (limited to 'libphobos/src/std/digest')
-rw-r--r-- | libphobos/src/std/digest/crc.d | 705 | ||||
-rw-r--r-- | libphobos/src/std/digest/digest.d | 21 | ||||
-rw-r--r-- | libphobos/src/std/digest/hmac.d | 336 | ||||
-rw-r--r-- | libphobos/src/std/digest/md.d | 590 | ||||
-rw-r--r-- | libphobos/src/std/digest/murmurhash.d | 755 | ||||
-rw-r--r-- | libphobos/src/std/digest/package.d | 1171 | ||||
-rw-r--r-- | libphobos/src/std/digest/ripemd.d | 762 | ||||
-rw-r--r-- | libphobos/src/std/digest/sha.d | 1291 |
8 files changed, 5631 insertions, 0 deletions
diff --git a/libphobos/src/std/digest/crc.d b/libphobos/src/std/digest/crc.d new file mode 100644 index 0000000..c606f4c --- /dev/null +++ b/libphobos/src/std/digest/crc.d @@ -0,0 +1,705 @@ +/** +Cyclic Redundancy Check (32-bit) implementation. + +$(SCRIPT inhibitQuickIndex = 1;) + +$(DIVC quickindex, +$(BOOKTABLE , +$(TR $(TH Category) $(TH Functions) +) +$(TR $(TDNW Template API) $(TD $(MYREF CRC) $(MYREF CRC32) $(MYREF CRC64ECMA) $(MYREF CRC64ISO) +) +) +$(TR $(TDNW OOP API) $(TD $(MYREF CRC32Digest) $(MYREF CRC64ECMADigest) $(MYREF CRC64ISODigest)) +) +$(TR $(TDNW Helpers) $(TD $(MYREF crcHexString) $(MYREF crc32Of) $(MYREF crc64ECMAOf) $(MYREF crc64ISOOf)) +) +) +) + + * + * This module conforms to the APIs defined in $(D std.digest). To understand the + * differences between the template and the OOP API, see $(MREF std, digest). + * + * This module publicly imports $(MREF std, digest) and can be used as a stand-alone + * module. + * + * Note: + * CRCs are usually printed with the MSB first. When using + * $(REF toHexString, std,digest) the result will be in an unexpected + * order. Use $(REF toHexString, std,digest)'s optional order parameter + * to specify decreasing order for the correct result. The $(LREF crcHexString) + * alias can also be used for this purpose. + * + * License: $(HTTP www.boost.org/LICENSE_1_0.txt, Boost License 1.0). + * + * Authors: Pavel "EvilOne" Minayev, Alex Rønne Petersen, Johannes Pfau + * + * References: + * $(LINK2 http://en.wikipedia.org/wiki/Cyclic_redundancy_check, Wikipedia on CRC) + * + * Source: $(PHOBOSSRC std/digest/_crc.d) + * + * Standards: + * Implements the 'common' IEEE CRC32 variant + * (LSB-first order, Initial value uint.max, complement result) + * + * CTFE: + * Digests do not work in CTFE + */ +/* + * Copyright (c) 2001 - 2002 + * Pavel "EvilOne" Minayev + * Copyright (c) 2012 + * Alex Rønne Petersen + * Distributed under the Boost Software License, Version 1.0. + * (See accompanying file LICENSE_1_0.txt or copy at + * http://www.boost.org/LICENSE_1_0.txt) + */ +module std.digest.crc; + +public import std.digest; + +version (unittest) + import std.exception; + + +/// +@safe unittest +{ + //Template API + import std.digest.crc; + + ubyte[4] hash = crc32Of("The quick brown fox jumps over the lazy dog"); + assert(crcHexString(hash) == "414FA339"); + + //Feeding data + ubyte[1024] data; + CRC32 crc; + crc.put(data[]); + crc.start(); //Start again + crc.put(data[]); + hash = crc.finish(); +} + +/// +@safe unittest +{ + //OOP API + import std.digest.crc; + + auto crc = new CRC32Digest(); + ubyte[] hash = crc.digest("The quick brown fox jumps over the lazy dog"); + assert(crcHexString(hash) == "414FA339"); //352441c2 + + //Feeding data + ubyte[1024] data; + crc.put(data[]); + crc.reset(); //Start again + crc.put(data[]); + hash = crc.finish(); +} + +private T[256][8] genTables(T)(T polynomial) +{ + T[256][8] res = void; + + foreach (i; 0 .. 0x100) + { + T crc = i; + foreach (_; 0 .. 8) + crc = (crc >> 1) ^ (-int(crc & 1) & polynomial); + res[0][i] = crc; + } + + foreach (i; 0 .. 0x100) + { + res[1][i] = (res[0][i] >> 8) ^ res[0][res[0][i] & 0xFF]; + res[2][i] = (res[1][i] >> 8) ^ res[0][res[1][i] & 0xFF]; + res[3][i] = (res[2][i] >> 8) ^ res[0][res[2][i] & 0xFF]; + res[4][i] = (res[3][i] >> 8) ^ res[0][res[3][i] & 0xFF]; + res[5][i] = (res[4][i] >> 8) ^ res[0][res[4][i] & 0xFF]; + res[6][i] = (res[5][i] >> 8) ^ res[0][res[5][i] & 0xFF]; + res[7][i] = (res[6][i] >> 8) ^ res[0][res[6][i] & 0xFF]; + } + return res; +} + +@system unittest +{ + auto tables = genTables(0xEDB88320); + assert(tables[0][0] == 0x00000000 && tables[0][$ - 1] == 0x2d02ef8d && tables[7][$ - 1] == 0x264b06e6); +} + +/** + * Template API CRC32 implementation. + * See $(D std.digest) for differences between template and OOP API. + */ +alias CRC32 = CRC!(32, 0xEDB88320); + +/** + * Template API CRC64-ECMA implementation. + * See $(D std.digest.digest) for differences between template and OOP API. + */ +alias CRC64ECMA = CRC!(64, 0xC96C5795D7870F42); + +/** + * Template API CRC64-ISO implementation. + * See $(D std.digest.digest) for differences between template and OOP API. + */ +alias CRC64ISO = CRC!(64, 0xD800000000000000); + +/** + * Generic Template API used for CRC32 and CRC64 implementations. + * + * The N parameter indicate the size of the hash in bits. + * The parameter P specify the polynomial to be used for reduction. + * + * You may want to use the CRC32, CRC65ECMA and CRC64ISO aliases + * for convenience. + * + * See $(D std.digest.digest) for differences between template and OOP API. + */ +struct CRC(uint N, ulong P) if (N == 32 || N == 64) +{ + private: + static if (N == 32) + { + alias T = uint; + } + else + { + alias T = ulong; + } + + static immutable T[256][8] tables = genTables!T(P); + + /** + * Type of the finished CRC hash. + * ubyte[4] if N is 32, ubyte[8] if N is 64. + */ + alias R = ubyte[T.sizeof]; + + // magic initialization constants + T _state = T.max; + + public: + /** + * Use this to feed the digest with data. + * Also implements the $(REF isOutputRange, std,range,primitives) + * interface for $(D ubyte) and $(D const(ubyte)[]). + */ + void put(scope const(ubyte)[] data...) @trusted pure nothrow @nogc + { + T crc = _state; + // process eight bytes at once + while (data.length >= 8) + { + // Use byte-wise reads to support architectures without HW support + // for unaligned reads. This can be optimized by compilers to a single + // 32-bit read if unaligned reads are supported. + // DMD is not able to do this optimization though, so explicitly + // do unaligned reads for DMD's architectures. + version (X86) + enum hasLittleEndianUnalignedReads = true; + else version (X86_64) + enum hasLittleEndianUnalignedReads = true; + else + enum hasLittleEndianUnalignedReads = false; // leave decision to optimizer + static if (hasLittleEndianUnalignedReads) + { + uint one = (cast(uint*) data.ptr)[0]; + uint two = (cast(uint*) data.ptr)[1]; + } + else + { + uint one = (data.ptr[3] << 24 | data.ptr[2] << 16 | data.ptr[1] << 8 | data.ptr[0]); + uint two = (data.ptr[7] << 24 | data.ptr[6] << 16 | data.ptr[5] << 8 | data.ptr[4]); + } + + static if (N == 32) + { + one ^= crc; + } + else + { + one ^= (crc & 0xffffffff); + two ^= (crc >> 32); + } + + crc = + tables[0][two >> 24] ^ + tables[1][(two >> 16) & 0xFF] ^ + tables[2][(two >> 8) & 0xFF] ^ + tables[3][two & 0xFF] ^ + tables[4][one >> 24] ^ + tables[5][(one >> 16) & 0xFF] ^ + tables[6][(one >> 8) & 0xFF] ^ + tables[7][one & 0xFF]; + + data = data[8 .. $]; + } + // remaining 1 to 7 bytes + foreach (d; data) + crc = (crc >> 8) ^ tables[0][(crc & 0xFF) ^ d]; + _state = crc; + } + + /** + * Used to initialize the CRC32 digest. + * + * Note: + * For this CRC32 Digest implementation calling start after default construction + * is not necessary. Calling start is only necessary to reset the Digest. + * + * Generic code which deals with different Digest types should always call start though. + */ + void start() @safe pure nothrow @nogc + { + this = CRC.init; + } + + /** + * Returns the finished CRC hash. This also calls $(LREF start) to + * reset the internal state. + */ + R finish() @safe pure nothrow @nogc + { + auto tmp = peek(); + start(); + return tmp; + } + + /** + * Works like $(D finish) but does not reset the internal state, so it's possible + * to continue putting data into this CRC after a call to peek. + */ + R peek() const @safe pure nothrow @nogc + { + import std.bitmanip : nativeToLittleEndian; + //Complement, LSB first / Little Endian, see http://rosettacode.org/wiki/CRC-32 + return nativeToLittleEndian(~_state); + } +} + +/// +@safe unittest +{ + //Simple example, hashing a string using crc32Of helper function + ubyte[4] hash32 = crc32Of("abc"); + //Let's get a hash string + assert(crcHexString(hash32) == "352441C2"); + // Repeat for CRC64 + ubyte[8] hash64ecma = crc64ECMAOf("abc"); + assert(crcHexString(hash64ecma) == "2CD8094A1A277627"); + ubyte[8] hash64iso = crc64ISOOf("abc"); + assert(crcHexString(hash64iso) == "3776C42000000000"); +} + +/// +@safe unittest +{ + ubyte[1024] data; + //Using the basic API + CRC32 hash32; + CRC64ECMA hash64ecma; + CRC64ISO hash64iso; + //Initialize data here... + hash32.put(data); + ubyte[4] result32 = hash32.finish(); + hash64ecma.put(data); + ubyte[8] result64ecma = hash64ecma.finish(); + hash64iso.put(data); + ubyte[8] result64iso = hash64iso.finish(); +} + +/// +@safe unittest +{ + //Let's use the template features: + //Note: When passing a CRC32 to a function, it must be passed by reference! + void doSomething(T)(ref T hash) + if (isDigest!T) + { + hash.put(cast(ubyte) 0); + } + CRC32 crc32; + crc32.start(); + doSomething(crc32); + assert(crcHexString(crc32.finish()) == "D202EF8D"); + // repeat for CRC64 + CRC64ECMA crc64ecma; + crc64ecma.start(); + doSomething(crc64ecma); + assert(crcHexString(crc64ecma.finish()) == "1FADA17364673F59"); + CRC64ISO crc64iso; + crc64iso.start(); + doSomething(crc64iso); + assert(crcHexString(crc64iso.finish()) == "6F90000000000000"); +} + +@safe unittest +{ + assert(isDigest!CRC32); + assert(isDigest!CRC64ECMA); + assert(isDigest!CRC64ISO); +} + +@system unittest +{ + ubyte[4] digest; + + CRC32 crc; + crc.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + assert(crc.peek() == cast(ubyte[]) x"bd50274c"); + crc.start(); + crc.put(cast(ubyte[])""); + assert(crc.finish() == cast(ubyte[]) x"00000000"); + + digest = crc32Of(""); + assert(digest == cast(ubyte[]) x"00000000"); + + //Test vector from http://rosettacode.org/wiki/CRC-32 + assert(crcHexString(crc32Of("The quick brown fox jumps over the lazy dog")) == "414FA339"); + + digest = crc32Of("a"); + assert(digest == cast(ubyte[]) x"43beb7e8"); + + digest = crc32Of("abc"); + assert(digest == cast(ubyte[]) x"c2412435"); + + digest = crc32Of("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + assert(digest == cast(ubyte[]) x"5f3f1a17"); + + digest = crc32Of("message digest"); + assert(digest == cast(ubyte[]) x"7f9d1520"); + + digest = crc32Of("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + assert(digest == cast(ubyte[]) x"d2e6c21f"); + + digest = crc32Of("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + assert(digest == cast(ubyte[]) x"724aa97c"); + + assert(crcHexString(cast(ubyte[4]) x"c3fcd3d7") == "D7D3FCC3"); +} + +@system unittest +{ + ubyte[8] digest; + + CRC64ECMA crc; + crc.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + assert(crc.peek() == cast(ubyte[]) x"2f121b7575789626"); + crc.start(); + crc.put(cast(ubyte[])""); + assert(crc.finish() == cast(ubyte[]) x"0000000000000000"); + digest = crc64ECMAOf(""); + assert(digest == cast(ubyte[]) x"0000000000000000"); + + //Test vector from http://rosettacode.org/wiki/CRC-32 + assert(crcHexString(crc64ECMAOf("The quick brown fox jumps over the lazy dog")) == "5B5EB8C2E54AA1C4"); + + digest = crc64ECMAOf("a"); + assert(digest == cast(ubyte[]) x"052b652e77840233"); + + digest = crc64ECMAOf("abc"); + assert(digest == cast(ubyte[]) x"2776271a4a09d82c"); + + digest = crc64ECMAOf("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + assert(digest == cast(ubyte[]) x"4b7cdce3746c449f"); + + digest = crc64ECMAOf("message digest"); + assert(digest == cast(ubyte[]) x"6f9b8a3156c9bc5d"); + + digest = crc64ECMAOf("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + assert(digest == cast(ubyte[]) x"2656b716e1bf0503"); + + digest = crc64ECMAOf("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + assert(digest == cast(ubyte[]) x"bd3eb7765d0a22ae"); + + assert(crcHexString(cast(ubyte[8]) x"c3fcd3d7efbeadde") == "DEADBEEFD7D3FCC3"); +} + +@system unittest +{ + ubyte[8] digest; + + CRC64ISO crc; + crc.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + assert(crc.peek() == cast(ubyte[]) x"f0494ab780989b42"); + crc.start(); + crc.put(cast(ubyte[])""); + assert(crc.finish() == cast(ubyte[]) x"0000000000000000"); + digest = crc64ISOOf(""); + assert(digest == cast(ubyte[]) x"0000000000000000"); + + //Test vector from http://rosettacode.org/wiki/CRC-32 + assert(crcHexString(crc64ISOOf("The quick brown fox jumps over the lazy dog")) == "4EF14E19F4C6E28E"); + + digest = crc64ISOOf("a"); + assert(digest == cast(ubyte[]) x"0000000000002034"); + + digest = crc64ISOOf("abc"); + assert(digest == cast(ubyte[]) x"0000000020c47637"); + + digest = crc64ISOOf("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + assert(digest == cast(ubyte[]) x"5173f717971365e5"); + + digest = crc64ISOOf("message digest"); + assert(digest == cast(ubyte[]) x"a2c355bbc0b93f86"); + + digest = crc64ISOOf("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + assert(digest == cast(ubyte[]) x"598B258292E40084"); + + digest = crc64ISOOf("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + assert(digest == cast(ubyte[]) x"760cd2d3588bf809"); + + assert(crcHexString(cast(ubyte[8]) x"c3fcd3d7efbeadde") == "DEADBEEFD7D3FCC3"); +} + +/** + * This is a convenience alias for $(REF digest, std,digest) using the + * CRC32 implementation. + * + * Params: + * data = $(D InputRange) of $(D ElementType) implicitly convertible to + * $(D ubyte), $(D ubyte[]) or $(D ubyte[num]) or one or more arrays + * of any type. + * + * Returns: + * CRC32 of data + */ +//simple alias doesn't work here, hope this gets inlined... +ubyte[4] crc32Of(T...)(T data) +{ + return digest!(CRC32, T)(data); +} + +/// +@system unittest +{ + ubyte[] data = [4,5,7,25]; + assert(data.crc32Of == [167, 180, 199, 131]); + + import std.utf : byChar; + assert("hello"d.byChar.crc32Of == [134, 166, 16, 54]); + + ubyte[4] hash = "abc".crc32Of(); + assert(hash == digest!CRC32("ab", "c")); + + import std.range : iota; + enum ubyte S = 5, F = 66; + assert(iota(S, F).crc32Of == [59, 140, 234, 154]); +} + +/** + * This is a convenience alias for $(REF digest, std,digest) using the + * CRC64-ECMA implementation. + * + * Params: + * data = $(D InputRange) of $(D ElementType) implicitly convertible to + * $(D ubyte), $(D ubyte[]) or $(D ubyte[num]) or one or more arrays + * of any type. + * + * Returns: + * CRC64-ECMA of data + */ +//simple alias doesn't work here, hope this gets inlined... +ubyte[8] crc64ECMAOf(T...)(T data) +{ + return digest!(CRC64ECMA, T)(data); +} + +/// +@system unittest +{ + ubyte[] data = [4,5,7,25]; + assert(data.crc64ECMAOf == [58, 142, 220, 214, 118, 98, 105, 69]); + + import std.utf : byChar; + assert("hello"d.byChar.crc64ECMAOf == [177, 55, 185, 219, 229, 218, 30, 155]); + + ubyte[8] hash = "abc".crc64ECMAOf(); + assert("abc".crc64ECMAOf == [39, 118, 39, 26, 74, 9, 216, 44]); + assert(hash == digest!CRC64ECMA("ab", "c")); + + import std.range : iota; + enum ubyte S = 5, F = 66; + assert(iota(S, F).crc64ECMAOf == [6, 184, 91, 238, 46, 213, 127, 188]); +} + +/** + * This is a convenience alias for $(REF digest, std,digest,digest) using the + * CRC64-ISO implementation. + * + * Params: + * data = $(D InputRange) of $(D ElementType) implicitly convertible to + * $(D ubyte), $(D ubyte[]) or $(D ubyte[num]) or one or more arrays + * of any type. + * + * Returns: + * CRC64-ISO of data + */ +//simple alias doesn't work here, hope this gets inlined... +ubyte[8] crc64ISOOf(T...)(T data) +{ + return digest!(CRC64ISO, T)(data); +} + +/// +@system unittest +{ + ubyte[] data = [4,5,7,25]; + assert(data.crc64ISOOf == [0, 0, 0, 80, 137, 232, 203, 120]); + + import std.utf : byChar; + assert("hello"d.byChar.crc64ISOOf == [0, 0, 16, 216, 226, 238, 62, 60]); + + ubyte[8] hash = "abc".crc64ISOOf(); + assert("abc".crc64ISOOf == [0, 0, 0, 0, 32, 196, 118, 55]); + assert(hash == digest!CRC64ISO("ab", "c")); + + import std.range : iota; + enum ubyte S = 5, F = 66; + + assert(iota(S, F).crc64ISOOf == [21, 185, 116, 95, 219, 11, 54, 7]); +} + +/** + * producing the usual CRC32 string output. + */ +public alias crcHexString = toHexString!(Order.decreasing); +///ditto +public alias crcHexString = toHexString!(Order.decreasing, 16); + +/** + * OOP API CRC32 implementation. + * See $(D std.digest) for differences between template and OOP API. + * + * This is an alias for $(D $(REF WrapperDigest, std,digest)!CRC32), see + * there for more information. + */ +alias CRC32Digest = WrapperDigest!CRC32; + +/** + * OOP API CRC64-ECMA implementation. + * See $(D std.digest.digest) for differences between template and OOP API. + * + * This is an alias for $(D $(REF WrapperDigest, std,digest,digest)!CRC64ECMA), + * see there for more information. + */ +alias CRC64ECMADigest = WrapperDigest!CRC64ECMA; + +/** + * OOP API CRC64-ISO implementation. + * See $(D std.digest.digest) for differences between template and OOP API. + * + * This is an alias for $(D $(REF WrapperDigest, std,digest,digest)!CRC64ISO), + * see there for more information. + */ +alias CRC64ISODigest = WrapperDigest!CRC64ISO; + +/// +@safe unittest +{ + //Simple example, hashing a string using Digest.digest helper function + auto crc = new CRC32Digest(); + ubyte[] hash = crc.digest("abc"); + //Let's get a hash string + assert(crcHexString(hash) == "352441C2"); +} + +/// +@system unittest +{ + //Let's use the OOP features: + void test(Digest dig) + { + dig.put(cast(ubyte) 0); + } + auto crc = new CRC32Digest(); + test(crc); + + //Let's use a custom buffer: + ubyte[4] buf; + ubyte[] result = crc.finish(buf[]); + assert(crcHexString(result) == "D202EF8D"); +} + +/// +@safe unittest +{ + //Simple example + auto hash = new CRC32Digest(); + hash.put(cast(ubyte) 0); + ubyte[] result = hash.finish(); +} + +/// +@system unittest +{ + //using a supplied buffer + ubyte[4] buf; + auto hash = new CRC32Digest(); + hash.put(cast(ubyte) 0); + ubyte[] result = hash.finish(buf[]); + //The result is now in result (and in buf. If you pass a buffer which is bigger than + //necessary, result will have the correct length, but buf will still have it's original + //length) +} + +@system unittest +{ + import std.range; + + auto crc = new CRC32Digest(); + + crc.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + assert(crc.peek() == cast(ubyte[]) x"bd50274c"); + crc.reset(); + crc.put(cast(ubyte[])""); + assert(crc.finish() == cast(ubyte[]) x"00000000"); + + crc.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + ubyte[20] result; + auto result2 = crc.finish(result[]); + assert(result[0 .. 4] == result2 && result2 == cast(ubyte[]) x"bd50274c"); + + debug + assertThrown!Error(crc.finish(result[0 .. 3])); + + assert(crc.length == 4); + + assert(crc.digest("") == cast(ubyte[]) x"00000000"); + + assert(crc.digest("a") == cast(ubyte[]) x"43beb7e8"); + + assert(crc.digest("abc") == cast(ubyte[]) x"c2412435"); + + assert(crc.digest("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq") + == cast(ubyte[]) x"5f3f1a17"); + + assert(crc.digest("message digest") == cast(ubyte[]) x"7f9d1520"); + + assert(crc.digest("abcdefghijklmnopqrstuvwxyz") + == cast(ubyte[]) x"bd50274c"); + + assert(crc.digest("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") + == cast(ubyte[]) x"d2e6c21f"); + + assert(crc.digest("1234567890123456789012345678901234567890", + "1234567890123456789012345678901234567890") + == cast(ubyte[]) x"724aa97c"); + + ubyte[] onemilliona = new ubyte[1000000]; + onemilliona[] = 'a'; + auto digest = crc32Of(onemilliona); + assert(digest == cast(ubyte[]) x"BCBF25DC"); + + auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); + digest = crc32Of(oneMillionRange); + assert(digest == cast(ubyte[]) x"BCBF25DC"); +} diff --git a/libphobos/src/std/digest/digest.d b/libphobos/src/std/digest/digest.d new file mode 100644 index 0000000..325afd4 --- /dev/null +++ b/libphobos/src/std/digest/digest.d @@ -0,0 +1,21 @@ +module std.digest.digest; + +import _newDigest = std.digest; + +// scheduled for deprecation in 2.077 +// See also: https://github.com/dlang/phobos/pull/5013#issuecomment-313987845 +alias isDigest = _newDigest.isDigest; +alias DigestType = _newDigest.DigestType; +alias hasPeek = _newDigest.hasPeek; +alias hasBlockSize = _newDigest.hasBlockSize; +alias digest = _newDigest.digest; +alias hexDigest = _newDigest.hexDigest; +alias makeDigest = _newDigest.makeDigest; +alias Digest = _newDigest.Digest; +alias Order = _newDigest.Order; +alias toHexString = _newDigest.toHexString; +alias asArray = _newDigest.asArray; +alias digestLength = _newDigest.digestLength; +alias WrapperDigest = _newDigest.WrapperDigest; +alias secureEqual = _newDigest.secureEqual; +alias LetterCase = _newDigest.LetterCase; diff --git a/libphobos/src/std/digest/hmac.d b/libphobos/src/std/digest/hmac.d new file mode 100644 index 0000000..bf0cbf3 --- /dev/null +++ b/libphobos/src/std/digest/hmac.d @@ -0,0 +1,336 @@ +// Written in the D programming language. + +/** +This package implements the hash-based message authentication code (_HMAC) +algorithm as defined in $(HTTP tools.ietf.org/html/rfc2104, RFC2104). See also +the corresponding $(HTTP en.wikipedia.org/wiki/Hash-based_message_authentication_code, Wikipedia article). + +$(SCRIPT inhibitQuickIndex = 1;) + +Macros: + +License: $(HTTP boost.org/LICENSE_1_0.txt, Boost License 1.0). + +Source: $(PHOBOSSRC std/digest/_hmac.d) + */ + +module std.digest.hmac; + +import std.digest : isDigest, hasBlockSize, isDigestibleRange, DigestType; +import std.meta : allSatisfy; + +@safe: + +/** + * Template API HMAC implementation. + * + * This implements an _HMAC over the digest H. If H doesn't provide + * information about the block size, it can be supplied explicitly using + * the second overload. + * + * This type conforms to $(REF isDigest, std,digest). + */ + +/// Compute HMAC over an input string +@safe unittest +{ + import std.ascii : LetterCase; + import std.digest : toHexString; + import std.digest.sha : SHA1; + import std.string : representation; + + auto secret = "secret".representation; + assert("The quick brown fox jumps over the lazy dog" + .representation + .hmac!SHA1(secret) + .toHexString!(LetterCase.lower) == "198ea1ea04c435c1246b586a06d5cf11c3ffcda6"); +} + +template HMAC(H) +if (isDigest!H && hasBlockSize!H) +{ + alias HMAC = HMAC!(H, H.blockSize); +} + +/** + * Overload of HMAC to be used if H doesn't provide information about its + * block size. + */ + +struct HMAC(H, size_t hashBlockSize) +if (hashBlockSize % 8 == 0) +{ + enum blockSize = hashBlockSize; + + private H digest; + private ubyte[blockSize / 8] key; + + /** + * Constructs the HMAC digest using the specified secret. + */ + + this(scope const(ubyte)[] secret) + { + // if secret is too long, shorten it by computing its hash + typeof(digest.finish()) buffer = void; + if (secret.length > blockSize / 8) + { + digest.start(); + digest.put(secret); + buffer = digest.finish(); + secret = buffer[]; + } + + // if secret is too short, it will be padded with zeroes + // (the key buffer is already zero-initialized) + import std.algorithm.mutation : copy; + secret.copy(key[]); + + start(); + } + + /// + @safe pure nothrow @nogc unittest + { + import std.digest.hmac, std.digest.sha; + import std.string : representation; + auto hmac = HMAC!SHA1("My s3cR3T keY".representation); + hmac.put("Hello, world".representation); + static immutable expected = [ + 130, 32, 235, 44, 208, 141, + 150, 232, 211, 214, 162, 195, + 188, 127, 52, 89, 100, 68, 90, 216]; + assert(hmac.finish() == expected); + } + + /** + * Reinitializes the digest, making it ready for reuse. + * + * Note: + * The constructor leaves the digest in an initialized state, so that this + * method only needs to be called if an unfinished digest is to be reused. + * + * Returns: + * A reference to the digest for convenient chaining. + */ + + ref HMAC!(H, blockSize) start() return + { + ubyte[blockSize / 8] ipad = void; + foreach (immutable i; 0 .. blockSize / 8) + ipad[i] = key[i] ^ 0x36; + + digest.start(); + digest.put(ipad[]); + + return this; + } + + /// + @safe pure nothrow @nogc unittest + { + import std.digest.hmac, std.digest.sha; + import std.string : representation; + string data1 = "Hello, world", data2 = "Hola mundo"; + auto hmac = HMAC!SHA1("My s3cR3T keY".representation); + hmac.put(data1.representation); + hmac.start(); // reset digest + hmac.put(data2.representation); // start over + static immutable expected = [ + 122, 151, 232, 240, 249, 80, + 19, 178, 186, 77, 110, 23, 208, + 52, 11, 88, 34, 151, 192, 255]; + assert(hmac.finish() == expected); + } + + /** + * Feeds a piece of data into the hash computation. This method allows the + * type to be used as an $(REF OutputRange, std,range). + * + * Returns: + * A reference to the digest for convenient chaining. + */ + + ref HMAC!(H, blockSize) put(in ubyte[] data...) return + { + digest.put(data); + return this; + } + + /// + @safe pure nothrow @nogc unittest + { + import std.digest.hmac, std.digest.sha; + import std.string : representation; + string data1 = "Hello, world", data2 = "Hola mundo"; + auto hmac = HMAC!SHA1("My s3cR3T keY".representation); + hmac.put(data1.representation) + .put(data2.representation); + static immutable expected = [ + 197, 57, 52, 3, 13, 194, 13, + 36, 117, 228, 8, 11, 111, 51, + 165, 3, 123, 31, 251, 113]; + assert(hmac.finish() == expected); + } + + /** + * Resets the digest and returns the finished hash. + */ + + DigestType!H finish() + { + ubyte[blockSize / 8] opad = void; + foreach (immutable i; 0 .. blockSize / 8) + opad[i] = key[i] ^ 0x5c; + + auto tmp = digest.finish(); + + digest.start(); + digest.put(opad[]); + digest.put(tmp); + auto result = digest.finish(); + start(); // reset the digest + return result; + } + + /// + @safe pure nothrow @nogc unittest + { + import std.digest.hmac, std.digest.sha; + import std.string : representation; + string data1 = "Hello, world", data2 = "Hola mundo"; + auto hmac = HMAC!SHA1("My s3cR3T keY".representation); + auto digest = hmac.put(data1.representation) + .put(data2.representation) + .finish(); + static immutable expected = [ + 197, 57, 52, 3, 13, 194, 13, + 36, 117, 228, 8, 11, 111, 51, + 165, 3, 123, 31, 251, 113]; + assert(digest == expected); + } +} + +/// Convenience constructor for $(LREF HMAC). +template hmac(H) +if (isDigest!H && hasBlockSize!H) +{ + alias hmac = hmac!(H, H.blockSize); +} + +/// ditto +template hmac(H, size_t blockSize) +if (isDigest!H) +{ + /** + * Constructs an HMAC digest with the specified secret. + * + * Returns: + * An instance of HMAC that can be fed data as desired, and finished + * to compute the final hash when done. + */ + auto hmac(scope const(ubyte)[] secret) + { + return HMAC!(H, blockSize)(secret); + } + + /// + @safe pure nothrow @nogc unittest + { + import std.digest.hmac, std.digest.sha; + import std.string : representation; + string data1 = "Hello, world", data2 = "Hola mundo"; + auto digest = hmac!SHA1("My s3cR3T keY".representation) + .put(data1.representation) + .put(data2.representation) + .finish(); + static immutable expected = [ + 197, 57, 52, 3, 13, 194, 13, 36, + 117, 228, 8, 11, 111, 51, 165, + 3, 123, 31, 251, 113]; + assert(digest == expected); + } + + /** + * Computes an _HMAC digest over the given range of data with the + * specified secret. + * + * Returns: + * The final _HMAC hash. + */ + DigestType!H hmac(T...)(scope T data, scope const(ubyte)[] secret) + if (allSatisfy!(isDigestibleRange, typeof(data))) + { + import std.range.primitives : put; + auto hash = HMAC!(H, blockSize)(secret); + foreach (datum; data) + put(hash, datum); + return hash.finish(); + } + + /// + @safe pure nothrow @nogc unittest + { + import std.algorithm.iteration : map; + import std.digest.hmac, std.digest.sha; + import std.string : representation; + string data = "Hello, world"; + auto digest = data.representation + .map!(a => cast(ubyte)(a+1)) + .hmac!SHA1("My s3cR3T keY".representation); + static assert(is(typeof(digest) == ubyte[20])); + static immutable expected = [ + 163, 208, 118, 179, 216, 93, + 17, 10, 84, 200, 87, 104, 244, + 111, 136, 214, 167, 210, 58, 10]; + assert(digest == expected); + } +} + +version (unittest) +{ + import std.digest : toHexString, LetterCase; + alias hex = toHexString!(LetterCase.lower); +} + +@safe pure nothrow @nogc +unittest +{ + import std.digest.md : MD5; + import std.range : isOutputRange; + static assert(isOutputRange!(HMAC!MD5, ubyte)); + static assert(isDigest!(HMAC!MD5)); + static assert(hasBlockSize!(HMAC!MD5) && HMAC!MD5.blockSize == MD5.blockSize); +} + +@safe pure nothrow +unittest +{ + import std.digest.md : MD5; + import std.digest.sha : SHA1, SHA256; + + ubyte[] nada; + assert(hmac!MD5 (nada, nada).hex == "74e6f7298a9c2d168935f58c001bad88"); + assert(hmac!SHA1 (nada, nada).hex == "fbdb1d1b18aa6c08324b7d64b71fb76370690e1d"); + assert(hmac!SHA256(nada, nada).hex == "b613679a0814d9ec772f95d778c35fc5ff1697c493715653c6c712144292c5ad"); + + import std.string : representation; + auto key = "key".representation, + long_key = ("012345678901234567890123456789012345678901" + ~"234567890123456789012345678901234567890123456789").representation, + data1 = "The quick brown fox ".representation, + data2 = "jumps over the lazy dog".representation, + data = data1 ~ data2; + + assert(data.hmac!MD5 (key).hex == "80070713463e7749b90c2dc24911e275"); + assert(data.hmac!SHA1 (key).hex == "de7c9b85b8b78aa6bc8a7a36f70a90701c9db4d9"); + assert(data.hmac!SHA256(key).hex == "f7bc83f430538424b13298e6aa6fb143ef4d59a14946175997479dbc2d1a3cd8"); + + assert(data.hmac!MD5 (long_key).hex == "e1728d68e05beae186ea768561963778"); + assert(data.hmac!SHA1 (long_key).hex == "560d3cd77316e57ab4bba0c186966200d2b37ba3"); + assert(data.hmac!SHA256(long_key).hex == "a1b0065a5d1edd93152c677e1bc1b1e3bc70d3a76619842e7f733f02b8135c04"); + + assert(hmac!MD5 (key).put(data1).put(data2).finish == data.hmac!MD5 (key)); + assert(hmac!SHA1 (key).put(data1).put(data2).finish == data.hmac!SHA1 (key)); + assert(hmac!SHA256(key).put(data1).put(data2).finish == data.hmac!SHA256(key)); +} diff --git a/libphobos/src/std/digest/md.d b/libphobos/src/std/digest/md.d new file mode 100644 index 0000000..1b621cf --- /dev/null +++ b/libphobos/src/std/digest/md.d @@ -0,0 +1,590 @@ +/** + * Computes MD5 hashes of arbitrary data. MD5 hashes are 16 byte quantities that are like a + * checksum or CRC, but are more robust. + * +$(SCRIPT inhibitQuickIndex = 1;) + +$(DIVC quickindex, +$(BOOKTABLE , +$(TR $(TH Category) $(TH Functions) +) +$(TR $(TDNW Template API) $(TD $(MYREF MD5) +) +) +$(TR $(TDNW OOP API) $(TD $(MYREF MD5Digest)) +) +$(TR $(TDNW Helpers) $(TD $(MYREF md5Of)) +) +) +) + + * This module conforms to the APIs defined in $(D std.digest). To understand the + * differences between the template and the OOP API, see $(MREF std, digest). + * + * This module publicly imports $(MREF std, digest) and can be used as a stand-alone + * module. + * + * License: $(HTTP www.boost.org/LICENSE_1_0.txt, Boost License 1.0). + * + * CTFE: + * Digests do not work in CTFE + * + * Authors: + * Piotr Szturmaj, Kai Nacke, Johannes Pfau $(BR) + * The routines and algorithms are derived from the $(I RSA Data Security, Inc. MD5 Message-Digest Algorithm). + * + * References: + * $(LINK2 http://en.wikipedia.org/wiki/Md5, Wikipedia on MD5) + * + * Source: $(PHOBOSSRC std/digest/_md.d) + * + */ + +/* md5.d - RSA Data Security, Inc., MD5 message-digest algorithm + * Derived from the RSA Data Security, Inc. MD5 Message-Digest Algorithm. + */ +module std.digest.md; + +public import std.digest; + +/// +@safe unittest +{ + //Template API + import std.digest.md; + + //Feeding data + ubyte[1024] data; + MD5 md5; + md5.start(); + md5.put(data[]); + md5.start(); //Start again + md5.put(data[]); + auto hash = md5.finish(); +} + +/// +@safe unittest +{ + //OOP API + import std.digest.md; + + auto md5 = new MD5Digest(); + ubyte[] hash = md5.digest("abc"); + assert(toHexString(hash) == "900150983CD24FB0D6963F7D28E17F72"); + + //Feeding data + ubyte[1024] data; + md5.put(data[]); + md5.reset(); //Start again + md5.put(data[]); + hash = md5.finish(); +} + +//rotateLeft rotates x left n bits +private uint rotateLeft(uint x, uint n) @safe pure nothrow @nogc +{ + // With recently added optimization to DMD (commit 32ea0206 at 07/28/11), this is translated to rol. + // No assembler required. + return (x << n) | (x >> (32-n)); +} + +/** + * Template API MD5 implementation. + * See $(D std.digest) for differences between template and OOP API. + */ +struct MD5 +{ + private: + // magic initialization constants + uint[4] _state = [0x67452301,0xefcdab89,0x98badcfe,0x10325476]; // state (ABCD) + ulong _count; //number of bits, modulo 2^64 + ubyte[64] _buffer; // input buffer + + static immutable ubyte[64] _padding = + [ + 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 + ]; + + // F, G, H and I are basic MD5 functions + static @safe pure nothrow @nogc + { + uint F(uint x, uint y, uint z) { return (x & y) | (~x & z); } + uint G(uint x, uint y, uint z) { return (x & z) | (y & ~z); } + uint H(uint x, uint y, uint z) { return x ^ y ^ z; } + uint I(uint x, uint y, uint z) { return y ^ (x | ~z); } + } + + + /* + * FF, GG, HH, and II transformations for rounds 1, 2, 3, and 4. + * Rotation is separate from addition to prevent recomputation. + */ + static void FF(ref uint a, uint b, uint c, uint d, uint x, uint s, uint ac) + @safe pure nothrow @nogc + { + a += F (b, c, d) + x + ac; + a = rotateLeft(a, s); + a += b; + } + + static void GG(ref uint a, uint b, uint c, uint d, uint x, uint s, uint ac) + @safe pure nothrow @nogc + { + a += G (b, c, d) + x + ac; + a = rotateLeft(a, s); + a += b; + } + + static void HH(ref uint a, uint b, uint c, uint d, uint x, uint s, uint ac) + @safe pure nothrow @nogc + { + a += H (b, c, d) + x + ac; + a = rotateLeft(a, s); + a += b; + } + + static void II(ref uint a, uint b, uint c, uint d, uint x, uint s, uint ac) + @safe pure nothrow @nogc + { + a += I (b, c, d) + x + ac; + a = rotateLeft(a, s); + a += b; + } + + /* + * MD5 basic transformation. Transforms state based on block. + */ + + //Constants for MD5Transform routine. + enum + { + S11 = 7, + S12 = 12, + S13 = 17, + S14 = 22, + S21 = 5, + S22 = 9, + S23 = 14, + S24 = 20, + S31 = 4, + S32 = 11, + S33 = 16, + S34 = 23, + S41 = 6, + S42 = 10, + S43 = 15, + S44 = 21, + } + + private void transform(const(ubyte[64])* block) pure nothrow @nogc + { + uint a = _state[0], + b = _state[1], + c = _state[2], + d = _state[3]; + + uint[16] x = void; + + version (BigEndian) + { + import std.bitmanip : littleEndianToNative; + + for (size_t i = 0; i < 16; i++) + { + x[i] = littleEndianToNative!uint(*cast(ubyte[4]*)&(*block)[i*4]); + } + } + else + { + (cast(ubyte*) x.ptr)[0 .. 64] = (cast(ubyte*) block)[0 .. 64]; + } + + //Round 1 + FF (a, b, c, d, x[ 0], S11, 0xd76aa478); /* 1 */ + FF (d, a, b, c, x[ 1], S12, 0xe8c7b756); /* 2 */ + FF (c, d, a, b, x[ 2], S13, 0x242070db); /* 3 */ + FF (b, c, d, a, x[ 3], S14, 0xc1bdceee); /* 4 */ + FF (a, b, c, d, x[ 4], S11, 0xf57c0faf); /* 5 */ + FF (d, a, b, c, x[ 5], S12, 0x4787c62a); /* 6 */ + FF (c, d, a, b, x[ 6], S13, 0xa8304613); /* 7 */ + FF (b, c, d, a, x[ 7], S14, 0xfd469501); /* 8 */ + FF (a, b, c, d, x[ 8], S11, 0x698098d8); /* 9 */ + FF (d, a, b, c, x[ 9], S12, 0x8b44f7af); /* 10 */ + FF (c, d, a, b, x[10], S13, 0xffff5bb1); /* 11 */ + FF (b, c, d, a, x[11], S14, 0x895cd7be); /* 12 */ + FF (a, b, c, d, x[12], S11, 0x6b901122); /* 13 */ + FF (d, a, b, c, x[13], S12, 0xfd987193); /* 14 */ + FF (c, d, a, b, x[14], S13, 0xa679438e); /* 15 */ + FF (b, c, d, a, x[15], S14, 0x49b40821); /* 16 */ + + //Round 2 + GG (a, b, c, d, x[ 1], S21, 0xf61e2562); /* 17 */ + GG (d, a, b, c, x[ 6], S22, 0xc040b340); /* 18 */ + GG (c, d, a, b, x[11], S23, 0x265e5a51); /* 19 */ + GG (b, c, d, a, x[ 0], S24, 0xe9b6c7aa); /* 20 */ + GG (a, b, c, d, x[ 5], S21, 0xd62f105d); /* 21 */ + GG (d, a, b, c, x[10], S22, 0x2441453); /* 22 */ + GG (c, d, a, b, x[15], S23, 0xd8a1e681); /* 23 */ + GG (b, c, d, a, x[ 4], S24, 0xe7d3fbc8); /* 24 */ + GG (a, b, c, d, x[ 9], S21, 0x21e1cde6); /* 25 */ + GG (d, a, b, c, x[14], S22, 0xc33707d6); /* 26 */ + GG (c, d, a, b, x[ 3], S23, 0xf4d50d87); /* 27 */ + GG (b, c, d, a, x[ 8], S24, 0x455a14ed); /* 28 */ + GG (a, b, c, d, x[13], S21, 0xa9e3e905); /* 29 */ + GG (d, a, b, c, x[ 2], S22, 0xfcefa3f8); /* 30 */ + GG (c, d, a, b, x[ 7], S23, 0x676f02d9); /* 31 */ + GG (b, c, d, a, x[12], S24, 0x8d2a4c8a); /* 32 */ + + //Round 3 + HH (a, b, c, d, x[ 5], S31, 0xfffa3942); /* 33 */ + HH (d, a, b, c, x[ 8], S32, 0x8771f681); /* 34 */ + HH (c, d, a, b, x[11], S33, 0x6d9d6122); /* 35 */ + HH (b, c, d, a, x[14], S34, 0xfde5380c); /* 36 */ + HH (a, b, c, d, x[ 1], S31, 0xa4beea44); /* 37 */ + HH (d, a, b, c, x[ 4], S32, 0x4bdecfa9); /* 38 */ + HH (c, d, a, b, x[ 7], S33, 0xf6bb4b60); /* 39 */ + HH (b, c, d, a, x[10], S34, 0xbebfbc70); /* 40 */ + HH (a, b, c, d, x[13], S31, 0x289b7ec6); /* 41 */ + HH (d, a, b, c, x[ 0], S32, 0xeaa127fa); /* 42 */ + HH (c, d, a, b, x[ 3], S33, 0xd4ef3085); /* 43 */ + HH (b, c, d, a, x[ 6], S34, 0x4881d05); /* 44 */ + HH (a, b, c, d, x[ 9], S31, 0xd9d4d039); /* 45 */ + HH (d, a, b, c, x[12], S32, 0xe6db99e5); /* 46 */ + HH (c, d, a, b, x[15], S33, 0x1fa27cf8); /* 47 */ + HH (b, c, d, a, x[ 2], S34, 0xc4ac5665); /* 48 */ + + //Round 4 + II (a, b, c, d, x[ 0], S41, 0xf4292244); /* 49 */ + II (d, a, b, c, x[ 7], S42, 0x432aff97); /* 50 */ + II (c, d, a, b, x[14], S43, 0xab9423a7); /* 51 */ + II (b, c, d, a, x[ 5], S44, 0xfc93a039); /* 52 */ + II (a, b, c, d, x[12], S41, 0x655b59c3); /* 53 */ + II (d, a, b, c, x[ 3], S42, 0x8f0ccc92); /* 54 */ + II (c, d, a, b, x[10], S43, 0xffeff47d); /* 55 */ + II (b, c, d, a, x[ 1], S44, 0x85845dd1); /* 56 */ + II (a, b, c, d, x[ 8], S41, 0x6fa87e4f); /* 57 */ + II (d, a, b, c, x[15], S42, 0xfe2ce6e0); /* 58 */ + II (c, d, a, b, x[ 6], S43, 0xa3014314); /* 59 */ + II (b, c, d, a, x[13], S44, 0x4e0811a1); /* 60 */ + II (a, b, c, d, x[ 4], S41, 0xf7537e82); /* 61 */ + II (d, a, b, c, x[11], S42, 0xbd3af235); /* 62 */ + II (c, d, a, b, x[ 2], S43, 0x2ad7d2bb); /* 63 */ + II (b, c, d, a, x[ 9], S44, 0xeb86d391); /* 64 */ + + _state[0] += a; + _state[1] += b; + _state[2] += c; + _state[3] += d; + + //Zeroize sensitive information. + x[] = 0; + } + + public: + enum blockSize = 512; + + /** + * Use this to feed the digest with data. + * Also implements the $(REF isOutputRange, std,range,primitives) + * interface for $(D ubyte) and $(D const(ubyte)[]). + * + * Example: + * ---- + * MD5 dig; + * dig.put(cast(ubyte) 0); //single ubyte + * dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic + * ubyte[10] buf; + * dig.put(buf); //buffer + * ---- + */ + void put(scope const(ubyte)[] data...) @trusted pure nothrow @nogc + { + uint i, index, partLen; + auto inputLen = data.length; + + //Compute number of bytes mod 64 + index = (cast(uint)_count >> 3) & (64 - 1); + + //Update number of bits + _count += inputLen * 8; + + partLen = 64 - index; + + //Transform as many times as possible + if (inputLen >= partLen) + { + (&_buffer[index])[0 .. partLen] = data.ptr[0 .. partLen]; + transform(&_buffer); + + for (i = partLen; i + 63 < inputLen; i += 64) + { + transform(cast(const(ubyte[64])*)(data[i .. i + 64].ptr)); + } + + index = 0; + } + else + { + i = 0; + } + + /* Buffer remaining input */ + if (inputLen - i) + (&_buffer[index])[0 .. inputLen-i] = (&data[i])[0 .. inputLen-i]; + } + + /** + * Used to (re)initialize the MD5 digest. + * + * Note: + * For this MD5 Digest implementation calling start after default construction + * is not necessary. Calling start is only necessary to reset the Digest. + * + * Generic code which deals with different Digest types should always call start though. + * + * Example: + * -------- + * MD5 digest; + * //digest.start(); //Not necessary + * digest.put(0); + * -------- + */ + void start() @safe pure nothrow @nogc + { + this = MD5.init; + } + + /** + * Returns the finished MD5 hash. This also calls $(LREF start) to + * reset the internal state. + */ + ubyte[16] finish() @trusted pure nothrow @nogc + { + import std.bitmanip : nativeToLittleEndian; + + ubyte[16] data = void; + ubyte[8] bits = void; + uint index, padLen; + + //Save number of bits + bits[0 .. 8] = nativeToLittleEndian(_count)[]; + + //Pad out to 56 mod 64 + index = (cast(uint)_count >> 3) & (64 - 1); + padLen = (index < 56) ? (56 - index) : (120 - index); + put(_padding[0 .. padLen]); + + //Append length (before padding) + put(bits); + + //Store state in digest + data[0 .. 4] = nativeToLittleEndian(_state[0])[]; + data[4 .. 8] = nativeToLittleEndian(_state[1])[]; + data[8 .. 12] = nativeToLittleEndian(_state[2])[]; + data[12 .. 16] = nativeToLittleEndian(_state[3])[]; + + /* Zeroize sensitive information. */ + start(); + return data; + } + /// + @safe unittest + { + //Simple example + MD5 hash; + hash.start(); + hash.put(cast(ubyte) 0); + ubyte[16] result = hash.finish(); + } +} + +/// +@safe unittest +{ + //Simple example, hashing a string using md5Of helper function + ubyte[16] hash = md5Of("abc"); + //Let's get a hash string + assert(toHexString(hash) == "900150983CD24FB0D6963F7D28E17F72"); +} + +/// +@safe unittest +{ + //Using the basic API + MD5 hash; + hash.start(); + ubyte[1024] data; + //Initialize data here... + hash.put(data); + ubyte[16] result = hash.finish(); +} + +/// +@safe unittest +{ + //Let's use the template features: + void doSomething(T)(ref T hash) + if (isDigest!T) + { + hash.put(cast(ubyte) 0); + } + MD5 md5; + md5.start(); + doSomething(md5); + assert(toHexString(md5.finish()) == "93B885ADFE0DA089CDF634904FD59F71"); +} + +@safe unittest +{ + assert(isDigest!MD5); +} + +@system unittest +{ + import std.range; + + ubyte[16] digest; + + MD5 md5; + md5.put(cast(ubyte[])"abcdef"); + md5.start(); + md5.put(cast(ubyte[])""); + assert(md5.finish() == cast(ubyte[]) x"d41d8cd98f00b204e9800998ecf8427e"); + + digest = md5Of(""); + assert(digest == cast(ubyte[]) x"d41d8cd98f00b204e9800998ecf8427e"); + + digest = md5Of("a"); + assert(digest == cast(ubyte[]) x"0cc175b9c0f1b6a831c399e269772661"); + + digest = md5Of("abc"); + assert(digest == cast(ubyte[]) x"900150983cd24fb0d6963f7d28e17f72"); + + digest = md5Of("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + assert(digest == cast(ubyte[]) x"8215ef0796a20bcaaae116d3876c664a"); + + digest = md5Of("message digest"); + assert(digest == cast(ubyte[]) x"f96b697d7cb7938d525a2f31aaf161d0"); + + digest = md5Of("abcdefghijklmnopqrstuvwxyz"); + assert(digest == cast(ubyte[]) x"c3fcd3d76192e4007dfb496cca67e13b"); + + digest = md5Of("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + assert(digest == cast(ubyte[]) x"d174ab98d277d9f5a5611c2c9f419d9f"); + + digest = md5Of("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + assert(digest == cast(ubyte[]) x"57edf4a22be3c955ac49da2e2107b67a"); + + assert(toHexString(cast(ubyte[16]) x"c3fcd3d76192e4007dfb496cca67e13b") + == "C3FCD3D76192E4007DFB496CCA67E13B"); + + ubyte[] onemilliona = new ubyte[1000000]; + onemilliona[] = 'a'; + digest = md5Of(onemilliona); + assert(digest == cast(ubyte[]) x"7707D6AE4E027C70EEA2A935C2296F21"); + + auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); + digest = md5Of(oneMillionRange); + assert(digest == cast(ubyte[]) x"7707D6AE4E027C70EEA2A935C2296F21"); +} + +/** + * This is a convenience alias for $(REF digest, std,digest) using the + * MD5 implementation. + */ +//simple alias doesn't work here, hope this gets inlined... +auto md5Of(T...)(T data) +{ + return digest!(MD5, T)(data); +} + +/// +@safe unittest +{ + ubyte[16] hash = md5Of("abc"); + assert(hash == digest!MD5("abc")); +} + +/** + * OOP API MD5 implementation. + * See $(D std.digest) for differences between template and OOP API. + * + * This is an alias for $(D $(REF WrapperDigest, std,digest)!MD5), see + * there for more information. + */ +alias MD5Digest = WrapperDigest!MD5; + +/// +@safe unittest +{ + //Simple example, hashing a string using Digest.digest helper function + auto md5 = new MD5Digest(); + ubyte[] hash = md5.digest("abc"); + //Let's get a hash string + assert(toHexString(hash) == "900150983CD24FB0D6963F7D28E17F72"); +} + +/// +@system unittest +{ + //Let's use the OOP features: + void test(Digest dig) + { + dig.put(cast(ubyte) 0); + } + auto md5 = new MD5Digest(); + test(md5); + + //Let's use a custom buffer: + ubyte[16] buf; + ubyte[] result = md5.finish(buf[]); + assert(toHexString(result) == "93B885ADFE0DA089CDF634904FD59F71"); +} + +@system unittest +{ + auto md5 = new MD5Digest(); + + md5.put(cast(ubyte[])"abcdef"); + md5.reset(); + md5.put(cast(ubyte[])""); + assert(md5.finish() == cast(ubyte[]) x"d41d8cd98f00b204e9800998ecf8427e"); + + md5.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + ubyte[20] result; + auto result2 = md5.finish(result[]); + assert(result[0 .. 16] == result2 && result2 == cast(ubyte[]) x"c3fcd3d76192e4007dfb496cca67e13b"); + + debug + { + import std.exception; + assertThrown!Error(md5.finish(result[0 .. 15])); + } + + assert(md5.length == 16); + + assert(md5.digest("") == cast(ubyte[]) x"d41d8cd98f00b204e9800998ecf8427e"); + + assert(md5.digest("a") == cast(ubyte[]) x"0cc175b9c0f1b6a831c399e269772661"); + + assert(md5.digest("abc") == cast(ubyte[]) x"900150983cd24fb0d6963f7d28e17f72"); + + assert(md5.digest("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq") + == cast(ubyte[]) x"8215ef0796a20bcaaae116d3876c664a"); + + assert(md5.digest("message digest") == cast(ubyte[]) x"f96b697d7cb7938d525a2f31aaf161d0"); + + assert(md5.digest("abcdefghijklmnopqrstuvwxyz") + == cast(ubyte[]) x"c3fcd3d76192e4007dfb496cca67e13b"); + + assert(md5.digest("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") + == cast(ubyte[]) x"d174ab98d277d9f5a5611c2c9f419d9f"); + + assert(md5.digest("1234567890123456789012345678901234567890", + "1234567890123456789012345678901234567890") + == cast(ubyte[]) x"57edf4a22be3c955ac49da2e2107b67a"); +} diff --git a/libphobos/src/std/digest/murmurhash.d b/libphobos/src/std/digest/murmurhash.d new file mode 100644 index 0000000..74efed5 --- /dev/null +++ b/libphobos/src/std/digest/murmurhash.d @@ -0,0 +1,755 @@ +/** +Computes $(LINK2 https://en.wikipedia.org/wiki/MurmurHash, MurmurHash) hashes +of arbitrary data. MurmurHash is a non-cryptographic hash function suitable +for general hash-based lookup. It is optimized for x86 but can be used on +all architectures. + +The current version is MurmurHash3, which yields a 32-bit or 128-bit hash value. +The older MurmurHash 1 and 2 are currently not supported. + +MurmurHash3 comes in three flavors, listed in increasing order of throughput: +$(UL +$(LI $(D MurmurHash3!32) produces a 32-bit value and is optimized for 32-bit architectures) +$(LI $(D MurmurHash3!(128, 32)) produces a 128-bit value and is optimized for 32-bit architectures) +$(LI $(D MurmurHash3!(128, 64)) produces a 128-bit value and is optimized for 64-bit architectures) +) + +Note: +$(UL +$(LI $(D MurmurHash3!(128, 32)) and $(D MurmurHash3!(128, 64)) produce different values.) +$(LI The current implementation is optimized for little endian architectures. + It will exhibit different results on big endian architectures and a slightly + less uniform distribution.) +) + +This module conforms to the APIs defined in $(MREF std, digest). + +This module publicly imports $(MREF std, digest) and can be used as a stand-alone module. + +Source: $(PHOBOSSRC std/digest/_murmurhash.d) +License: $(HTTP www.boost.org/LICENSE_1_0.txt, Boost License 1.0). +Authors: Guillaume Chatelet +References: $(LINK2 https://github.com/aappleby/smhasher, Reference implementation) +$(BR) $(LINK2 https://en.wikipedia.org/wiki/MurmurHash, Wikipedia) +*/ +/* Copyright Guillaume Chatelet 2016. + * Distributed under the Boost Software License, Version 1.0. + * (See LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) + */ +module std.digest.murmurhash; + +/// +@safe unittest +{ + // MurmurHash3!32, MurmurHash3!(128, 32) and MurmurHash3!(128, 64) implement + // the std.digest Template API. + static assert(isDigest!(MurmurHash3!32)); + // The convenient digest template allows for quick hashing of any data. + ubyte[4] hashed = digest!(MurmurHash3!32)([1, 2, 3, 4]); + assert(hashed == [0, 173, 69, 68]); +} + +/// +@safe unittest +{ + // One can also hash ubyte data piecewise by instanciating a hasher and call + // the 'put' method. + const(ubyte)[] data1 = [1, 2, 3]; + const(ubyte)[] data2 = [4, 5, 6, 7]; + // The incoming data will be buffered and hashed element by element. + MurmurHash3!32 hasher; + hasher.put(data1); + hasher.put(data2); + // The call to 'finish' ensures: + // - the remaining bits are processed + // - the hash gets finalized + auto hashed = hasher.finish(); + assert(hashed == [181, 151, 88, 252]); +} + +/// +@safe unittest +{ + // Using `putElements`, `putRemainder` and `finalize` you gain full + // control over which part of the algorithm to run. + // This allows for maximum throughput but needs extra care. + + // Data type must be the same as the hasher's element type: + // - uint for MurmurHash3!32 + // - uint[4] for MurmurHash3!(128, 32) + // - ulong[2] for MurmurHash3!(128, 64) + const(uint)[] data = [1, 2, 3, 4]; + // Note the hasher starts with 'Fast'. + MurmurHash3!32 hasher; + // Push as many array of elements as you need. The less calls the better. + hasher.putElements(data); + // Put remainder bytes if needed. This method can be called only once. + hasher.putRemainder(ubyte(1), ubyte(1), ubyte(1)); + // Call finalize to incorporate data length in the hash. + hasher.finalize(); + // Finally get the hashed value. + auto hashed = hasher.getBytes(); + assert(hashed == [188, 165, 108, 2]); +} + +public import std.digest; + +@safe: + +/* +Performance notes: + - To help a bit with the performance when compiling with DMD some + functions have been rewritten to pass by value instead of by reference. + - GDC and LDC are on par with their C++ counterpart. + - DMD is typically between 20% to 50% of the GCC version. +*/ + +/++ + + Implements the MurmurHash3 functions. You can specify the `size` of the + + hash in bit. For 128 bit hashes you can specify whether to optimize for 32 + + or 64 bit architectures. If you don't specify the `opt` value it will select + + the fastest version of the host platform. + + + + This hasher is compatible with the `Digest` API: + + $(UL + + $(LI `void start()`) + + $(LI `void put(scope const(ubyte)[] data...)`) + + $(LI `ubyte[Element.sizeof] finish()`) + + ) + + + + It also provides a faster, low level API working with data of size + + `Element.sizeof`: + + $(UL + + $(LI `void putElements(scope const(Element[]) elements...)`) + + $(LI `void putRemainder(scope const(ubyte[]) data...)`) + + $(LI `void finalize()`) + + $(LI `Element get()`) + + $(LI `ubyte[Element.sizeof] getBytes()`) + + ) + +/ +struct MurmurHash3(uint size /* 32 or 128 */ , uint opt = size_t.sizeof == 8 ? 64 : 32) +{ + enum blockSize = size; // Number of bits of the hashed value. + size_t element_count; // The number of full elements pushed, this is used for finalization. + + static if (size == 32) + { + private enum uint c1 = 0xcc9e2d51; + private enum uint c2 = 0x1b873593; + private uint h1; + alias Element = uint; /// The element type for 32-bit implementation. + + this(uint seed) + { + h1 = seed; + } + /++ + Adds a single Element of data without increasing `element_count`. + Make sure to increase `element_count` by `Element.sizeof` for each call to `putElement`. + +/ + void putElement(uint block) pure nothrow @nogc + { + h1 = update(h1, block, 0, c1, c2, 15, 13, 0xe6546b64U); + } + + /// Put remainder bytes. This must be called only once after `putElement` and before `finalize`. + void putRemainder(scope const(ubyte[]) data...) pure nothrow @nogc + { + assert(data.length < Element.sizeof); + assert(data.length >= 0); + element_count += data.length; + uint k1 = 0; + final switch (data.length & 3) + { + case 3: + k1 ^= data[2] << 16; + goto case; + case 2: + k1 ^= data[1] << 8; + goto case; + case 1: + k1 ^= data[0]; + h1 ^= shuffle(k1, c1, c2, 15); + goto case; + case 0: + } + } + + /// Incorporate `element_count` and finalizes the hash. + void finalize() pure nothrow @nogc + { + h1 ^= element_count; + h1 = fmix(h1); + } + + /// Returns the hash as an uint value. + Element get() pure nothrow @nogc + { + return h1; + } + + /// Returns the current hashed value as an ubyte array. + ubyte[4] getBytes() pure nothrow @nogc + { + return cast(typeof(return)) cast(uint[1])[get()]; + } + } + else static if (size == 128 && opt == 32) + { + private enum uint c1 = 0x239b961b; + private enum uint c2 = 0xab0e9789; + private enum uint c3 = 0x38b34ae5; + private enum uint c4 = 0xa1e38b93; + private uint h4, h3, h2, h1; + + alias Element = uint[4]; /// The element type for 128-bit implementation. + + this(uint seed4, uint seed3, uint seed2, uint seed1) pure nothrow @nogc + { + h4 = seed4; + h3 = seed3; + h2 = seed2; + h1 = seed1; + } + + this(uint seed) pure nothrow @nogc + { + h4 = h3 = h2 = h1 = seed; + } + + /++ + Adds a single Element of data without increasing element_count. + Make sure to increase `element_count` by `Element.sizeof` for each call to `putElement`. + +/ + void putElement(Element block) pure nothrow @nogc + { + h1 = update(h1, block[0], h2, c1, c2, 15, 19, 0x561ccd1bU); + h2 = update(h2, block[1], h3, c2, c3, 16, 17, 0x0bcaa747U); + h3 = update(h3, block[2], h4, c3, c4, 17, 15, 0x96cd1c35U); + h4 = update(h4, block[3], h1, c4, c1, 18, 13, 0x32ac3b17U); + } + + /// Put remainder bytes. This must be called only once after `putElement` and before `finalize`. + void putRemainder(scope const(ubyte[]) data...) pure nothrow @nogc + { + assert(data.length < Element.sizeof); + assert(data.length >= 0); + element_count += data.length; + uint k1 = 0; + uint k2 = 0; + uint k3 = 0; + uint k4 = 0; + + final switch (data.length & 15) + { + case 15: + k4 ^= data[14] << 16; + goto case; + case 14: + k4 ^= data[13] << 8; + goto case; + case 13: + k4 ^= data[12] << 0; + h4 ^= shuffle(k4, c4, c1, 18); + goto case; + case 12: + k3 ^= data[11] << 24; + goto case; + case 11: + k3 ^= data[10] << 16; + goto case; + case 10: + k3 ^= data[9] << 8; + goto case; + case 9: + k3 ^= data[8] << 0; + h3 ^= shuffle(k3, c3, c4, 17); + goto case; + case 8: + k2 ^= data[7] << 24; + goto case; + case 7: + k2 ^= data[6] << 16; + goto case; + case 6: + k2 ^= data[5] << 8; + goto case; + case 5: + k2 ^= data[4] << 0; + h2 ^= shuffle(k2, c2, c3, 16); + goto case; + case 4: + k1 ^= data[3] << 24; + goto case; + case 3: + k1 ^= data[2] << 16; + goto case; + case 2: + k1 ^= data[1] << 8; + goto case; + case 1: + k1 ^= data[0] << 0; + h1 ^= shuffle(k1, c1, c2, 15); + goto case; + case 0: + } + } + + /// Incorporate `element_count` and finalizes the hash. + void finalize() pure nothrow @nogc + { + h1 ^= element_count; + h2 ^= element_count; + h3 ^= element_count; + h4 ^= element_count; + + h1 += h2; + h1 += h3; + h1 += h4; + h2 += h1; + h3 += h1; + h4 += h1; + + h1 = fmix(h1); + h2 = fmix(h2); + h3 = fmix(h3); + h4 = fmix(h4); + + h1 += h2; + h1 += h3; + h1 += h4; + h2 += h1; + h3 += h1; + h4 += h1; + } + + /// Returns the hash as an uint[4] value. + Element get() pure nothrow @nogc + { + return [h1, h2, h3, h4]; + } + + /// Returns the current hashed value as an ubyte array. + ubyte[16] getBytes() pure nothrow @nogc + { + return cast(typeof(return)) get(); + } + } + else static if (size == 128 && opt == 64) + { + private enum ulong c1 = 0x87c37b91114253d5; + private enum ulong c2 = 0x4cf5ad432745937f; + private ulong h2, h1; + + alias Element = ulong[2]; /// The element type for 128-bit implementation. + + this(ulong seed) pure nothrow @nogc + { + h2 = h1 = seed; + } + + this(ulong seed2, ulong seed1) pure nothrow @nogc + { + h2 = seed2; + h1 = seed1; + } + + /++ + Adds a single Element of data without increasing `element_count`. + Make sure to increase `element_count` by `Element.sizeof` for each call to `putElement`. + +/ + void putElement(Element block) pure nothrow @nogc + { + h1 = update(h1, block[0], h2, c1, c2, 31, 27, 0x52dce729U); + h2 = update(h2, block[1], h1, c2, c1, 33, 31, 0x38495ab5U); + } + + /// Put remainder bytes. This must be called only once after `putElement` and before `finalize`. + void putRemainder(scope const(ubyte[]) data...) pure nothrow @nogc + { + assert(data.length < Element.sizeof); + assert(data.length >= 0); + element_count += data.length; + ulong k1 = 0; + ulong k2 = 0; + final switch (data.length & 15) + { + case 15: + k2 ^= ulong(data[14]) << 48; + goto case; + case 14: + k2 ^= ulong(data[13]) << 40; + goto case; + case 13: + k2 ^= ulong(data[12]) << 32; + goto case; + case 12: + k2 ^= ulong(data[11]) << 24; + goto case; + case 11: + k2 ^= ulong(data[10]) << 16; + goto case; + case 10: + k2 ^= ulong(data[9]) << 8; + goto case; + case 9: + k2 ^= ulong(data[8]) << 0; + h2 ^= shuffle(k2, c2, c1, 33); + goto case; + case 8: + k1 ^= ulong(data[7]) << 56; + goto case; + case 7: + k1 ^= ulong(data[6]) << 48; + goto case; + case 6: + k1 ^= ulong(data[5]) << 40; + goto case; + case 5: + k1 ^= ulong(data[4]) << 32; + goto case; + case 4: + k1 ^= ulong(data[3]) << 24; + goto case; + case 3: + k1 ^= ulong(data[2]) << 16; + goto case; + case 2: + k1 ^= ulong(data[1]) << 8; + goto case; + case 1: + k1 ^= ulong(data[0]) << 0; + h1 ^= shuffle(k1, c1, c2, 31); + goto case; + case 0: + } + } + + /// Incorporate `element_count` and finalizes the hash. + void finalize() pure nothrow @nogc + { + h1 ^= element_count; + h2 ^= element_count; + + h1 += h2; + h2 += h1; + h1 = fmix(h1); + h2 = fmix(h2); + h1 += h2; + h2 += h1; + } + + /// Returns the hash as an ulong[2] value. + Element get() pure nothrow @nogc + { + return [h1, h2]; + } + + /// Returns the current hashed value as an ubyte array. + ubyte[16] getBytes() pure nothrow @nogc + { + return cast(typeof(return)) get(); + } + } + else + { + alias Element = char; // This is needed to trigger the following error message. + static assert(false, "MurmurHash3(" ~ size.stringof ~ ", " ~ opt.stringof ~ ") is not implemented"); + } + + /++ + Pushes an array of elements at once. It is more efficient to push as much data as possible in a single call. + On platforms that do not support unaligned reads (MIPS or old ARM chips), the compiler may produce slower code to ensure correctness. + +/ + void putElements(scope const(Element[]) elements...) pure nothrow @nogc + { + foreach (const block; elements) + { + putElement(block); + } + element_count += elements.length * Element.sizeof; + } + + //------------------------------------------------------------------------- + // Implementation of the Digest API. + //------------------------------------------------------------------------- + + private union BufferUnion + { + Element block; + ubyte[Element.sizeof] data; + } + + private BufferUnion buffer; + private size_t bufferSize; + + @disable this(this); + + // Initialize + void start() + { + this = this.init; + } + + /++ + Adds data to the digester. This function can be called many times in a row + after start but before finish. + +/ + void put(scope const(ubyte)[] data...) pure nothrow + { + // Buffer should never be full while entering this function. + assert(bufferSize < Element.sizeof); + + // Check if we have some leftover data in the buffer. Then fill the first block buffer. + if (bufferSize + data.length < Element.sizeof) + { + buffer.data[bufferSize .. bufferSize + data.length] = data[]; + bufferSize += data.length; + return; + } + const bufferLeeway = Element.sizeof - bufferSize; + assert(bufferLeeway <= Element.sizeof); + buffer.data[bufferSize .. $] = data[0 .. bufferLeeway]; + putElement(buffer.block); + data = data[bufferLeeway .. $]; + + // Do main work: process chunks of `Element.sizeof` bytes. + const numElements = data.length / Element.sizeof; + const remainderStart = numElements * Element.sizeof; + foreach (ref const Element block; cast(const(Element[]))(data[0 .. remainderStart])) + { + putElement(block); + } + // +1 for bufferLeeway Element. + element_count += (numElements + 1) * Element.sizeof; + data = data[remainderStart .. $]; + + // Now add remaining data to buffer. + assert(data.length < Element.sizeof); + bufferSize = data.length; + buffer.data[0 .. data.length] = data[]; + } + + /++ + Finalizes the computation of the hash and returns the computed value. + Note that $(D finish) can be called only once and that no subsequent calls + to $(D put) is allowed. + +/ + ubyte[Element.sizeof] finish() pure nothrow + { + auto tail = buffer.data[0 .. bufferSize]; + if (tail.length > 0) + { + putRemainder(tail); + } + finalize(); + return getBytes(); + } + + //------------------------------------------------------------------------- + // MurmurHash3 utils + //------------------------------------------------------------------------- + + private T rotl(T)(T x, uint y) + in + { + import std.traits : isUnsigned; + + static assert(isUnsigned!T); + debug assert(y >= 0 && y <= (T.sizeof * 8)); + } + body + { + return ((x << y) | (x >> ((T.sizeof * 8) - y))); + } + + private T shuffle(T)(T k, T c1, T c2, ubyte r1) + { + import std.traits : isUnsigned; + + static assert(isUnsigned!T); + k *= c1; + k = rotl(k, r1); + k *= c2; + return k; + } + + private T update(T)(ref T h, T k, T mixWith, T c1, T c2, ubyte r1, ubyte r2, T n) + { + import std.traits : isUnsigned; + + static assert(isUnsigned!T); + h ^= shuffle(k, c1, c2, r1); + h = rotl(h, r2); + h += mixWith; + return h * 5 + n; + } + + private uint fmix(uint h) pure nothrow @nogc + { + h ^= h >> 16; + h *= 0x85ebca6b; + h ^= h >> 13; + h *= 0xc2b2ae35; + h ^= h >> 16; + return h; + } + + private ulong fmix(ulong k) pure nothrow @nogc + { + k ^= k >> 33; + k *= 0xff51afd7ed558ccd; + k ^= k >> 33; + k *= 0xc4ceb9fe1a85ec53; + k ^= k >> 33; + return k; + } +} + +version (unittest) +{ + import std.string : representation; + + private auto hash(H, Element = H.Element)(string data) + { + H hasher; + immutable elements = data.length / Element.sizeof; + hasher.putElements(cast(const(Element)[]) data[0 .. elements * Element.sizeof]); + hasher.putRemainder(cast(const(ubyte)[]) data[elements * Element.sizeof .. $]); + hasher.finalize(); + return hasher.getBytes(); + } + + private void checkResult(H)(in string[string] groundtruth) + { + foreach (data, expectedHash; groundtruth) + { + assert(data.digest!H.toHexString() == expectedHash); + assert(data.hash!H.toHexString() == expectedHash); + H hasher; + foreach (element; data) + { + hasher.put(element); + } + assert(hasher.finish.toHexString() == expectedHash); + } + } +} + +@safe unittest +{ + // dfmt off + checkResult!(MurmurHash3!32)([ + "" : "00000000", + "a" : "B269253C", + "ab" : "5FD7BF9B", + "abc" : "FA93DDB3", + "abcd" : "6A67ED43", + "abcde" : "F69A9BE8", + "abcdef" : "85C08161", + "abcdefg" : "069B3C88", + "abcdefgh" : "C4CCDD49", + "abcdefghi" : "F0061442", + "abcdefghij" : "91779288", + "abcdefghijk" : "DF253B5F", + "abcdefghijkl" : "273D6FA3", + "abcdefghijklm" : "1B1612F2", + "abcdefghijklmn" : "F06D52F8", + "abcdefghijklmno" : "D2F7099D", + "abcdefghijklmnop" : "ED9162E7", + "abcdefghijklmnopq" : "4A5E65B6", + "abcdefghijklmnopqr" : "94A819C2", + "abcdefghijklmnopqrs" : "C15BBF85", + "abcdefghijklmnopqrst" : "9A711CBE", + "abcdefghijklmnopqrstu" : "ABE7195A", + "abcdefghijklmnopqrstuv" : "C73CB670", + "abcdefghijklmnopqrstuvw" : "1C4D1EA5", + "abcdefghijklmnopqrstuvwx" : "3939F9B0", + "abcdefghijklmnopqrstuvwxy" : "1A568338", + "abcdefghijklmnopqrstuvwxyz" : "6D034EA3"]); + // dfmt on +} + +@safe unittest +{ + // dfmt off + checkResult!(MurmurHash3!(128,32))([ + "" : "00000000000000000000000000000000", + "a" : "3C9394A71BB056551BB056551BB05655", + "ab" : "DF5184151030BE251030BE251030BE25", + "abc" : "D1C6CD75A506B0A2A506B0A2A506B0A2", + "abcd" : "AACCB6962EC6AF452EC6AF452EC6AF45", + "abcde" : "FB2E40C5BCC5245D7701725A7701725A", + "abcdef" : "0AB97CE12127AFA1F9DFBEA9F9DFBEA9", + "abcdefg" : "D941B590DE3A86092869774A2869774A", + "abcdefgh" : "3611F4AE8714B1AD92806CFA92806CFA", + "abcdefghi" : "1C8C05AD6F590622107DD2147C4194DD", + "abcdefghij" : "A72ED9F50E90379A2AAA92C77FF12F69", + "abcdefghijk" : "DDC9C8A01E111FCA2DF1FE8257975EBD", + "abcdefghijkl" : "FE038573C02482F4ADDFD42753E58CD2", + "abcdefghijklm" : "15A23AC1ECA1AEDB66351CF470DE2CD9", + "abcdefghijklmn" : "8E11EC75D71F5D60F4456F944D89D4F1", + "abcdefghijklmno" : "691D6DEEAED51A4A5714CE84A861A7AD", + "abcdefghijklmnop" : "2776D29F5612B990218BCEE445BA93D1", + "abcdefghijklmnopq" : "D3A445046F5C51642ADC6DD99D07111D", + "abcdefghijklmnopqr" : "AA5493A0DA291D966A9E7128585841D9", + "abcdefghijklmnopqrs" : "281B6A4F9C45B9BFC3B77850930F2C20", + "abcdefghijklmnopqrst" : "19342546A8216DB62873B49E545DCB1F", + "abcdefghijklmnopqrstu" : "A6C0F30D6C738620E7B9590D2E088D99", + "abcdefghijklmnopqrstuv" : "A7D421D9095CDCEA393CBBA908342384", + "abcdefghijklmnopqrstuvw" : "C3A93D572B014949317BAD7EE809158F", + "abcdefghijklmnopqrstuvwx" : "802381D77956833791F87149326E4801", + "abcdefghijklmnopqrstuvwxy" : "0AC619A5302315755A80D74ADEFAA842", + "abcdefghijklmnopqrstuvwxyz" : "1306343E662F6F666E56F6172C3DE344"]); + // dfmt on +} + +@safe unittest +{ + // dfmt off + checkResult!(MurmurHash3!(128,64))([ + "" : "00000000000000000000000000000000", + "a" : "897859F6655555855A890E51483AB5E6", + "ab" : "2E1BED16EA118B93ADD4529B01A75EE6", + "abc" : "6778AD3F3F3F96B4522DCA264174A23B", + "abcd" : "4FCD5646D6B77BB875E87360883E00F2", + "abcde" : "B8BB96F491D036208CECCF4BA0EEC7C5", + "abcdef" : "55BFA3ACBF867DE45C842133990971B0", + "abcdefg" : "99E49EC09F2FCDA6B6BB55B13AA23A1C", + "abcdefgh" : "028CEF37B00A8ACCA14069EB600D8948", + "abcdefghi" : "64793CF1CFC0470533E041B7F53DB579", + "abcdefghij" : "998C2F770D5BC1B6C91A658CDC854DA2", + "abcdefghijk" : "029D78DFB8D095A871E75A45E2317CBB", + "abcdefghijkl" : "94E17AE6B19BF38E1C62FF7232309E1F", + "abcdefghijklm" : "73FAC0A78D2848167FCCE70DFF7B652E", + "abcdefghijklmn" : "E075C3F5A794D09124336AD2276009EE", + "abcdefghijklmno" : "FB2F0C895124BE8A612A969C2D8C546A", + "abcdefghijklmnop" : "23B74C22A33CCAC41AEB31B395D63343", + "abcdefghijklmnopq" : "57A6BD887F746475E40D11A19D49DAEC", + "abcdefghijklmnopqr" : "508A7F90EC8CF0776BC7005A29A8D471", + "abcdefghijklmnopqrs" : "886D9EDE23BC901574946FB62A4D8AA6", + "abcdefghijklmnopqrst" : "F1E237F926370B314BD016572AF40996", + "abcdefghijklmnopqrstu" : "3CC9FF79E268D5C9FB3C9BE9C148CCD7", + "abcdefghijklmnopqrstuv" : "56F8ABF430E388956DA9F4A8741FDB46", + "abcdefghijklmnopqrstuvw" : "8E234F9DBA0A4840FFE9541CEBB7BE83", + "abcdefghijklmnopqrstuvwx" : "F72CDED40F96946408F22153A3CF0F79", + "abcdefghijklmnopqrstuvwxy" : "0F96072FA4CBE771DBBD9E398115EEED", + "abcdefghijklmnopqrstuvwxyz" : "A94A6F517E9D9C7429D5A7B6899CADE9"]); + // dfmt on +} + +@safe unittest +{ + // Pushing unaligned data and making sure the result is still coherent. + void testUnalignedHash(H)() + { + immutable ubyte[1025] data = 0xAC; + immutable alignedHash = digest!H(data[0 .. $ - 1]); // 0 .. 1023 + immutable unalignedHash = digest!H(data[1 .. $]); // 1 .. 1024 + assert(alignedHash == unalignedHash); + } + + testUnalignedHash!(MurmurHash3!32)(); + testUnalignedHash!(MurmurHash3!(128, 32))(); + testUnalignedHash!(MurmurHash3!(128, 64))(); +} diff --git a/libphobos/src/std/digest/package.d b/libphobos/src/std/digest/package.d new file mode 100644 index 0000000..f4646ae --- /dev/null +++ b/libphobos/src/std/digest/package.d @@ -0,0 +1,1171 @@ +/** + * This module describes the _digest APIs used in Phobos. All digests follow + * these APIs. Additionally, this module contains useful helper methods which + * can be used with every _digest type. + * +$(SCRIPT inhibitQuickIndex = 1;) + +$(DIVC quickindex, +$(BOOKTABLE , +$(TR $(TH Category) $(TH Functions) +) +$(TR $(TDNW Template API) $(TD $(MYREF isDigest) $(MYREF DigestType) $(MYREF hasPeek) + $(MYREF hasBlockSize) + $(MYREF ExampleDigest) $(MYREF _digest) $(MYREF hexDigest) $(MYREF makeDigest) +) +) +$(TR $(TDNW OOP API) $(TD $(MYREF Digest) +) +) +$(TR $(TDNW Helper functions) $(TD $(MYREF toHexString)) +) +$(TR $(TDNW Implementation helpers) $(TD $(MYREF digestLength) $(MYREF WrapperDigest)) +) +) +) + + * APIs: + * There are two APIs for digests: The template API and the OOP API. The template API uses structs + * and template helpers like $(LREF isDigest). The OOP API implements digests as classes inheriting + * the $(LREF Digest) interface. All digests are named so that the template API struct is called "$(B x)" + * and the OOP API class is called "$(B x)Digest". For example we have $(D MD5) <--> $(D MD5Digest), + * $(D CRC32) <--> $(D CRC32Digest), etc. + * + * The template API is slightly more efficient. It does not have to allocate memory dynamically, + * all memory is allocated on the stack. The OOP API has to allocate in the finish method if no + * buffer was provided. If you provide a buffer to the OOP APIs finish function, it doesn't allocate, + * but the $(LREF Digest) classes still have to be created using $(D new) which allocates them using the GC. + * + * The OOP API is useful to change the _digest function and/or _digest backend at 'runtime'. The benefit here + * is that switching e.g. Phobos MD5Digest and an OpenSSLMD5Digest implementation is ABI compatible. + * + * If just one specific _digest type and backend is needed, the template API is usually a good fit. + * In this simplest case, the template API can even be used without templates: Just use the "$(B x)" structs + * directly. + * + * License: $(HTTP www.boost.org/LICENSE_1_0.txt, Boost License 1.0). + * Authors: + * Johannes Pfau + * + * Source: $(PHOBOSSRC std/_digest/_package.d) + * + * CTFE: + * Digests do not work in CTFE + * + * TODO: + * Digesting single bits (as opposed to bytes) is not implemented. This will be done as another + * template constraint helper (hasBitDigesting!T) and an additional interface (BitDigest) + */ +/* Copyright Johannes Pfau 2012. + * Distributed under the Boost Software License, Version 1.0. + * (See accompanying file LICENSE_1_0.txt or copy at + * http://www.boost.org/LICENSE_1_0.txt) + */ +module std.digest; + +public import std.ascii : LetterCase; +import std.meta : allSatisfy; +import std.range.primitives; +import std.traits; + + +/// +@system unittest +{ + import std.digest.crc; + + //Simple example + char[8] hexHash = hexDigest!CRC32("The quick brown fox jumps over the lazy dog"); + assert(hexHash == "39A34F41"); + + //Simple example, using the API manually + CRC32 context = makeDigest!CRC32(); + context.put(cast(ubyte[])"The quick brown fox jumps over the lazy dog"); + ubyte[4] hash = context.finish(); + assert(toHexString(hash) == "39A34F41"); +} + +/// +@system unittest +{ + //Generating the hashes of a file, idiomatic D way + import std.digest.crc, std.digest.md, std.digest.sha; + import std.stdio; + + // Digests a file and prints the result. + void digestFile(Hash)(string filename) + if (isDigest!Hash) + { + auto file = File(filename); + auto result = digest!Hash(file.byChunk(4096 * 1024)); + writefln("%s (%s) = %s", Hash.stringof, filename, toHexString(result)); + } + + void main(string[] args) + { + foreach (name; args[1 .. $]) + { + digestFile!MD5(name); + digestFile!SHA1(name); + digestFile!CRC32(name); + } + } +} +/// +@system unittest +{ + //Generating the hashes of a file using the template API + import std.digest.crc, std.digest.md, std.digest.sha; + import std.stdio; + // Digests a file and prints the result. + void digestFile(Hash)(ref Hash hash, string filename) + if (isDigest!Hash) + { + File file = File(filename); + + //As digests imlement OutputRange, we could use std.algorithm.copy + //Let's do it manually for now + foreach (buffer; file.byChunk(4096 * 1024)) + hash.put(buffer); + + auto result = hash.finish(); + writefln("%s (%s) = %s", Hash.stringof, filename, toHexString(result)); + } + + void uMain(string[] args) + { + MD5 md5; + SHA1 sha1; + CRC32 crc32; + + md5.start(); + sha1.start(); + crc32.start(); + + foreach (arg; args[1 .. $]) + { + digestFile(md5, arg); + digestFile(sha1, arg); + digestFile(crc32, arg); + } + } +} + +/// +@system unittest +{ + import std.digest.crc, std.digest.md, std.digest.sha; + import std.stdio; + + // Digests a file and prints the result. + void digestFile(Digest hash, string filename) + { + File file = File(filename); + + //As digests implement OutputRange, we could use std.algorithm.copy + //Let's do it manually for now + foreach (buffer; file.byChunk(4096 * 1024)) + hash.put(buffer); + + ubyte[] result = hash.finish(); + writefln("%s (%s) = %s", typeid(hash).toString(), filename, toHexString(result)); + } + + void umain(string[] args) + { + auto md5 = new MD5Digest(); + auto sha1 = new SHA1Digest(); + auto crc32 = new CRC32Digest(); + + foreach (arg; args[1 .. $]) + { + digestFile(md5, arg); + digestFile(sha1, arg); + digestFile(crc32, arg); + } + } +} + +version (StdDdoc) + version = ExampleDigest; + +version (ExampleDigest) +{ + /** + * This documents the general structure of a Digest in the template API. + * All digest implementations should implement the following members and therefore pass + * the $(LREF isDigest) test. + * + * Note: + * $(UL + * $(LI A digest must be a struct (value type) to pass the $(LREF isDigest) test.) + * $(LI A digest passing the $(LREF isDigest) test is always an $(D OutputRange)) + * ) + */ + struct ExampleDigest + { + public: + /** + * Use this to feed the digest with data. + * Also implements the $(REF isOutputRange, std,range,primitives) + * interface for $(D ubyte) and $(D const(ubyte)[]). + * The following usages of $(D put) must work for any type which + * passes $(LREF isDigest): + * Example: + * ---- + * ExampleDigest dig; + * dig.put(cast(ubyte) 0); //single ubyte + * dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic + * ubyte[10] buf; + * dig.put(buf); //buffer + * ---- + */ + @trusted void put(scope const(ubyte)[] data...) + { + + } + + /** + * This function is used to (re)initialize the digest. + * It must be called before using the digest and it also works as a 'reset' function + * if the digest has already processed data. + */ + @trusted void start() + { + + } + + /** + * The finish function returns the final hash sum and resets the Digest. + * + * Note: + * The actual type returned by finish depends on the digest implementation. + * $(D ubyte[16]) is just used as an example. It is guaranteed that the type is a + * static array of ubytes. + * + * $(UL + * $(LI Use $(LREF DigestType) to obtain the actual return type.) + * $(LI Use $(LREF digestLength) to obtain the length of the ubyte array.) + * ) + */ + @trusted ubyte[16] finish() + { + return (ubyte[16]).init; + } + } +} + +/// +@system unittest +{ + //Using the OutputRange feature + import std.algorithm.mutation : copy; + import std.digest.md; + import std.range : repeat; + + auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); + auto ctx = makeDigest!MD5(); + copy(oneMillionRange, &ctx); //Note: You must pass a pointer to copy! + assert(ctx.finish().toHexString() == "7707D6AE4E027C70EEA2A935C2296F21"); +} + +/** + * Use this to check if a type is a digest. See $(LREF ExampleDigest) to see what + * a type must provide to pass this check. + * + * Note: + * This is very useful as a template constraint (see examples) + * + * BUGS: + * $(UL + * $(LI Does not yet verify that put takes scope parameters.) + * $(LI Should check that finish() returns a ubyte[num] array) + * ) + */ +template isDigest(T) +{ + import std.range : isOutputRange; + enum bool isDigest = isOutputRange!(T, const(ubyte)[]) && isOutputRange!(T, ubyte) && + is(T == struct) && + is(typeof( + { + T dig = void; //Can define + dig.put(cast(ubyte) 0, cast(ubyte) 0); //varags + dig.start(); //has start + auto value = dig.finish(); //has finish + })); +} + +/// +@system unittest +{ + import std.digest.crc; + static assert(isDigest!CRC32); +} +/// +@system unittest +{ + import std.digest.crc; + void myFunction(T)() + if (isDigest!T) + { + T dig; + dig.start(); + auto result = dig.finish(); + } + myFunction!CRC32(); +} + +/** + * Use this template to get the type which is returned by a digest's $(LREF finish) method. + */ +template DigestType(T) +{ + static if (isDigest!T) + { + alias DigestType = + ReturnType!(typeof( + { + T dig = void; + return dig.finish(); + })); + } + else + static assert(false, T.stringof ~ " is not a digest! (fails isDigest!T)"); +} + +/// +@system unittest +{ + import std.digest.crc; + assert(is(DigestType!(CRC32) == ubyte[4])); +} +/// +@system unittest +{ + import std.digest.crc; + CRC32 dig; + dig.start(); + DigestType!CRC32 result = dig.finish(); +} + +/** + * Used to check if a digest supports the $(D peek) method. + * Peek has exactly the same function signatures as finish, but it doesn't reset + * the digest's internal state. + * + * Note: + * $(UL + * $(LI This is very useful as a template constraint (see examples)) + * $(LI This also checks if T passes $(LREF isDigest)) + * ) + */ +template hasPeek(T) +{ + enum bool hasPeek = isDigest!T && + is(typeof( + { + T dig = void; //Can define + DigestType!T val = dig.peek(); + })); +} + +/// +@system unittest +{ + import std.digest.crc, std.digest.md; + assert(!hasPeek!(MD5)); + assert(hasPeek!CRC32); +} +/// +@system unittest +{ + import std.digest.crc; + void myFunction(T)() + if (hasPeek!T) + { + T dig; + dig.start(); + auto result = dig.peek(); + } + myFunction!CRC32(); +} + +/** + * Checks whether the digest has a $(D blockSize) member, which contains the + * digest's internal block size in bits. It is primarily used by $(REF HMAC, std,digest,hmac). + */ + +template hasBlockSize(T) +if (isDigest!T) +{ + enum bool hasBlockSize = __traits(compiles, { size_t blockSize = T.blockSize; }); +} + +/// +@system unittest +{ + import std.digest.hmac, std.digest.md; + static assert(hasBlockSize!MD5 && MD5.blockSize == 512); + static assert(hasBlockSize!(HMAC!MD5) && HMAC!MD5.blockSize == 512); +} + +package template isDigestibleRange(Range) +{ + import std.digest.md; + import std.range : isInputRange, ElementType; + enum bool isDigestibleRange = isInputRange!Range && is(typeof( + { + MD5 ha; //Could use any conformant hash + ElementType!Range val; + ha.put(val); + })); +} + +/** + * This is a convenience function to calculate a hash using the template API. + * Every digest passing the $(LREF isDigest) test can be used with this function. + * + * Params: + * range= an $(D InputRange) with $(D ElementType) $(D ubyte), $(D ubyte[]) or $(D ubyte[num]) + */ +DigestType!Hash digest(Hash, Range)(auto ref Range range) +if (!isArray!Range + && isDigestibleRange!Range) +{ + import std.algorithm.mutation : copy; + Hash hash; + hash.start(); + copy(range, &hash); + return hash.finish(); +} + +/// +@system unittest +{ + import std.digest.md; + import std.range : repeat; + auto testRange = repeat!ubyte(cast(ubyte)'a', 100); + auto md5 = digest!MD5(testRange); +} + +/** + * This overload of the digest function handles arrays. + * + * Params: + * data= one or more arrays of any type + */ +DigestType!Hash digest(Hash, T...)(scope const T data) +if (allSatisfy!(isArray, typeof(data))) +{ + Hash hash; + hash.start(); + foreach (datum; data) + hash.put(cast(const(ubyte[]))datum); + return hash.finish(); +} + +/// +@system unittest +{ + import std.digest.crc, std.digest.md, std.digest.sha; + auto md5 = digest!MD5( "The quick brown fox jumps over the lazy dog"); + auto sha1 = digest!SHA1( "The quick brown fox jumps over the lazy dog"); + auto crc32 = digest!CRC32("The quick brown fox jumps over the lazy dog"); + assert(toHexString(crc32) == "39A34F41"); +} + +/// +@system unittest +{ + import std.digest.crc; + auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog"); + assert(toHexString(crc32) == "39A34F41"); +} + +/** + * This is a convenience function similar to $(LREF digest), but it returns the string + * representation of the hash. Every digest passing the $(LREF isDigest) test can be used with this + * function. + * + * Params: + * order= the order in which the bytes are processed (see $(LREF toHexString)) + * range= an $(D InputRange) with $(D ElementType) $(D ubyte), $(D ubyte[]) or $(D ubyte[num]) + */ +char[digestLength!(Hash)*2] hexDigest(Hash, Order order = Order.increasing, Range)(ref Range range) +if (!isArray!Range && isDigestibleRange!Range) +{ + return toHexString!order(digest!Hash(range)); +} + +/// +@system unittest +{ + import std.digest.md; + import std.range : repeat; + auto testRange = repeat!ubyte(cast(ubyte)'a', 100); + assert(hexDigest!MD5(testRange) == "36A92CC94A9E0FA21F625F8BFB007ADF"); +} + +/** + * This overload of the hexDigest function handles arrays. + * + * Params: + * order= the order in which the bytes are processed (see $(LREF toHexString)) + * data= one or more arrays of any type + */ +char[digestLength!(Hash)*2] hexDigest(Hash, Order order = Order.increasing, T...)(scope const T data) +if (allSatisfy!(isArray, typeof(data))) +{ + return toHexString!order(digest!Hash(data)); +} + +/// +@system unittest +{ + import std.digest.crc; + assert(hexDigest!(CRC32, Order.decreasing)("The quick brown fox jumps over the lazy dog") == "414FA339"); +} +/// +@system unittest +{ + import std.digest.crc; + assert(hexDigest!(CRC32, Order.decreasing)("The quick ", "brown ", "fox jumps over the lazy dog") == "414FA339"); +} + +/** + * This is a convenience function which returns an initialized digest, so it's not necessary to call + * start manually. + */ +Hash makeDigest(Hash)() +{ + Hash hash; + hash.start(); + return hash; +} + +/// +@system unittest +{ + import std.digest.md; + auto md5 = makeDigest!MD5(); + md5.put(0); + assert(toHexString(md5.finish()) == "93B885ADFE0DA089CDF634904FD59F71"); +} + +/*+*************************** End of template part, welcome to OOP land **************************/ + +/** + * This describes the OOP API. To understand when to use the template API and when to use the OOP API, + * see the module documentation at the top of this page. + * + * The Digest interface is the base interface which is implemented by all digests. + * + * Note: + * A Digest implementation is always an $(D OutputRange) + */ +interface Digest +{ + public: + /** + * Use this to feed the digest with data. + * Also implements the $(REF isOutputRange, std,range,primitives) + * interface for $(D ubyte) and $(D const(ubyte)[]). + * + * Example: + * ---- + * void test(Digest dig) + * { + * dig.put(cast(ubyte) 0); //single ubyte + * dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic + * ubyte[10] buf; + * dig.put(buf); //buffer + * } + * ---- + */ + @trusted nothrow void put(scope const(ubyte)[] data...); + + /** + * Resets the internal state of the digest. + * Note: + * $(LREF finish) calls this internally, so it's not necessary to call + * $(D reset) manually after a call to $(LREF finish). + */ + @trusted nothrow void reset(); + + /** + * This is the length in bytes of the hash value which is returned by $(LREF finish). + * It's also the required size of a buffer passed to $(LREF finish). + */ + @trusted nothrow @property size_t length() const; + + /** + * The finish function returns the hash value. It takes an optional buffer to copy the data + * into. If a buffer is passed, it must be at least $(LREF length) bytes big. + */ + @trusted nothrow ubyte[] finish(); + ///ditto + nothrow ubyte[] finish(ubyte[] buf); + //@@@BUG@@@ http://d.puremagic.com/issues/show_bug.cgi?id=6549 + /*in + { + assert(buf.length >= this.length); + }*/ + + /** + * This is a convenience function to calculate the hash of a value using the OOP API. + */ + final @trusted nothrow ubyte[] digest(scope const(void[])[] data...) + { + this.reset(); + foreach (datum; data) + this.put(cast(ubyte[]) datum); + return this.finish(); + } +} + +/// +@system unittest +{ + //Using the OutputRange feature + import std.algorithm.mutation : copy; + import std.digest.md; + import std.range : repeat; + + auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); + auto ctx = new MD5Digest(); + copy(oneMillionRange, ctx); + assert(ctx.finish().toHexString() == "7707D6AE4E027C70EEA2A935C2296F21"); +} + +/// +@system unittest +{ + import std.digest.crc, std.digest.md, std.digest.sha; + ubyte[] md5 = (new MD5Digest()).digest("The quick brown fox jumps over the lazy dog"); + ubyte[] sha1 = (new SHA1Digest()).digest("The quick brown fox jumps over the lazy dog"); + ubyte[] crc32 = (new CRC32Digest()).digest("The quick brown fox jumps over the lazy dog"); + assert(crcHexString(crc32) == "414FA339"); +} + +/// +@system unittest +{ + import std.digest.crc; + ubyte[] crc32 = (new CRC32Digest()).digest("The quick ", "brown ", "fox jumps over the lazy dog"); + assert(crcHexString(crc32) == "414FA339"); +} + +@system unittest +{ + import std.range : isOutputRange; + assert(!isDigest!(Digest)); + assert(isOutputRange!(Digest, ubyte)); +} + +/// +@system unittest +{ + void test(Digest dig) + { + dig.put(cast(ubyte) 0); //single ubyte + dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic + ubyte[10] buf; + dig.put(buf); //buffer + } +} + +/*+*************************** End of OOP part, helper functions follow ***************************/ + +/** + * See $(LREF toHexString) + */ +enum Order : bool +{ + increasing, /// + decreasing /// +} + + +/** + * Used to convert a hash value (a static or dynamic array of ubytes) to a string. + * Can be used with the OOP and with the template API. + * + * The additional order parameter can be used to specify the order of the input data. + * By default the data is processed in increasing order, starting at index 0. To process it in the + * opposite order, pass Order.decreasing as a parameter. + * + * The additional letterCase parameter can be used to specify the case of the output data. + * By default the output is in upper case. To change it to the lower case + * pass LetterCase.lower as a parameter. + * + * Note: + * The function overloads returning a string allocate their return values + * using the GC. The versions returning static arrays use pass-by-value for + * the return value, effectively avoiding dynamic allocation. + */ +char[num*2] toHexString(Order order = Order.increasing, size_t num, LetterCase letterCase = LetterCase.upper) +(in ubyte[num] digest) +{ + static if (letterCase == LetterCase.upper) + { + import std.ascii : hexDigits = hexDigits; + } + else + { + import std.ascii : hexDigits = lowerHexDigits; + } + + + char[num*2] result; + size_t i; + + static if (order == Order.increasing) + { + foreach (u; digest) + { + result[i++] = hexDigits[u >> 4]; + result[i++] = hexDigits[u & 15]; + } + } + else + { + size_t j = num - 1; + while (i < num*2) + { + result[i++] = hexDigits[digest[j] >> 4]; + result[i++] = hexDigits[digest[j] & 15]; + j--; + } + } + return result; +} + +///ditto +char[num*2] toHexString(LetterCase letterCase, Order order = Order.increasing, size_t num)(in ubyte[num] digest) +{ + return toHexString!(order, num, letterCase)(digest); +} + +///ditto +string toHexString(Order order = Order.increasing, LetterCase letterCase = LetterCase.upper) +(in ubyte[] digest) +{ + static if (letterCase == LetterCase.upper) + { + import std.ascii : hexDigits = hexDigits; + } + else + { + import std.ascii : hexDigits = lowerHexDigits; + } + + auto result = new char[digest.length*2]; + size_t i; + + static if (order == Order.increasing) + { + foreach (u; digest) + { + result[i++] = hexDigits[u >> 4]; + result[i++] = hexDigits[u & 15]; + } + } + else + { + import std.range : retro; + foreach (u; retro(digest)) + { + result[i++] = hexDigits[u >> 4]; + result[i++] = hexDigits[u & 15]; + } + } + import std.exception : assumeUnique; + // memory was just created, so casting to immutable is safe + return () @trusted { return assumeUnique(result); }(); +} + +///ditto +string toHexString(LetterCase letterCase, Order order = Order.increasing)(in ubyte[] digest) +{ + return toHexString!(order, letterCase)(digest); +} + +//For more example unittests, see Digest.digest, digest + +/// +@safe unittest +{ + import std.digest.crc; + //Test with template API: + auto crc32 = digest!CRC32("The quick ", "brown ", "fox jumps over the lazy dog"); + //Lower case variant: + assert(toHexString!(LetterCase.lower)(crc32) == "39a34f41"); + //Usually CRCs are printed in this order, though: + assert(toHexString!(Order.decreasing)(crc32) == "414FA339"); + assert(toHexString!(LetterCase.lower, Order.decreasing)(crc32) == "414fa339"); +} + +/// +@safe unittest +{ + import std.digest.crc; + // With OOP API + auto crc32 = (new CRC32Digest()).digest("The quick ", "brown ", "fox jumps over the lazy dog"); + //Usually CRCs are printed in this order, though: + assert(toHexString!(Order.decreasing)(crc32) == "414FA339"); +} + +@safe unittest +{ + ubyte[16] data; + assert(toHexString(data) == "00000000000000000000000000000000"); + + assert(toHexString(cast(ubyte[4])[42, 43, 44, 45]) == "2A2B2C2D"); + assert(toHexString(cast(ubyte[])[42, 43, 44, 45]) == "2A2B2C2D"); + assert(toHexString!(Order.decreasing)(cast(ubyte[4])[42, 43, 44, 45]) == "2D2C2B2A"); + assert(toHexString!(Order.decreasing, LetterCase.lower)(cast(ubyte[4])[42, 43, 44, 45]) == "2d2c2b2a"); + assert(toHexString!(Order.decreasing)(cast(ubyte[])[42, 43, 44, 45]) == "2D2C2B2A"); +} + +/*+*********************** End of public helper part, private helpers follow ***********************/ + +/* + * Used to convert from a ubyte[] slice to a ref ubyte[N]. + * This helper is used internally in the WrapperDigest template to wrap the template API's + * finish function. + */ +ref T[N] asArray(size_t N, T)(ref T[] source, string errorMsg = "") +{ + assert(source.length >= N, errorMsg); + return *cast(T[N]*) source.ptr; +} + +/* + * Returns the length (in bytes) of the hash value produced by T. + */ +template digestLength(T) +if (isDigest!T) +{ + enum size_t digestLength = (ReturnType!(T.finish)).length; +} + +@safe pure nothrow @nogc +unittest +{ + import std.digest.md : MD5; + import std.digest.sha : SHA1, SHA256, SHA512; + assert(digestLength!MD5 == 16); + assert(digestLength!SHA1 == 20); + assert(digestLength!SHA256 == 32); + assert(digestLength!SHA512 == 64); +} + +/** + * Wraps a template API hash struct into a Digest interface. + * Modules providing digest implementations will usually provide + * an alias for this template (e.g. MD5Digest, SHA1Digest, ...). + */ +class WrapperDigest(T) +if (isDigest!T) : Digest +{ + protected: + T _digest; + + public final: + /** + * Initializes the digest. + */ + this() + { + _digest.start(); + } + + /** + * Use this to feed the digest with data. + * Also implements the $(REF isOutputRange, std,range,primitives) + * interface for $(D ubyte) and $(D const(ubyte)[]). + */ + @trusted nothrow void put(scope const(ubyte)[] data...) + { + _digest.put(data); + } + + /** + * Resets the internal state of the digest. + * Note: + * $(LREF finish) calls this internally, so it's not necessary to call + * $(D reset) manually after a call to $(LREF finish). + */ + @trusted nothrow void reset() + { + _digest.start(); + } + + /** + * This is the length in bytes of the hash value which is returned by $(LREF finish). + * It's also the required size of a buffer passed to $(LREF finish). + */ + @trusted nothrow @property size_t length() const pure + { + return digestLength!T; + } + + /** + * The finish function returns the hash value. It takes an optional buffer to copy the data + * into. If a buffer is passed, it must have a length at least $(LREF length) bytes. + * + * Example: + * -------- + * + * import std.digest.md; + * ubyte[16] buf; + * auto hash = new WrapperDigest!MD5(); + * hash.put(cast(ubyte) 0); + * auto result = hash.finish(buf[]); + * //The result is now in result (and in buf). If you pass a buffer which is bigger than + * //necessary, result will have the correct length, but buf will still have it's original + * //length + * -------- + */ + nothrow ubyte[] finish(ubyte[] buf) + in + { + assert(buf.length >= this.length); + } + body + { + enum string msg = "Buffer needs to be at least " ~ digestLength!(T).stringof ~ " bytes " ~ + "big, check " ~ typeof(this).stringof ~ ".length!"; + asArray!(digestLength!T)(buf, msg) = _digest.finish(); + return buf[0 .. digestLength!T]; + } + + ///ditto + @trusted nothrow ubyte[] finish() + { + enum len = digestLength!T; + auto buf = new ubyte[len]; + asArray!(digestLength!T)(buf) = _digest.finish(); + return buf; + } + + version (StdDdoc) + { + /** + * Works like $(D finish) but does not reset the internal state, so it's possible + * to continue putting data into this WrapperDigest after a call to peek. + * + * These functions are only available if $(D hasPeek!T) is true. + */ + @trusted ubyte[] peek(ubyte[] buf) const; + ///ditto + @trusted ubyte[] peek() const; + } + else static if (hasPeek!T) + { + @trusted ubyte[] peek(ubyte[] buf) const + in + { + assert(buf.length >= this.length); + } + body + { + enum string msg = "Buffer needs to be at least " ~ digestLength!(T).stringof ~ " bytes " ~ + "big, check " ~ typeof(this).stringof ~ ".length!"; + asArray!(digestLength!T)(buf, msg) = _digest.peek(); + return buf[0 .. digestLength!T]; + } + + @trusted ubyte[] peek() const + { + enum len = digestLength!T; + auto buf = new ubyte[len]; + asArray!(digestLength!T)(buf) = _digest.peek(); + return buf; + } + } +} + +/// +@system unittest +{ + import std.digest.md; + //Simple example + auto hash = new WrapperDigest!MD5(); + hash.put(cast(ubyte) 0); + auto result = hash.finish(); +} + +/// +@system unittest +{ + //using a supplied buffer + import std.digest.md; + ubyte[16] buf; + auto hash = new WrapperDigest!MD5(); + hash.put(cast(ubyte) 0); + auto result = hash.finish(buf[]); + //The result is now in result (and in buf). If you pass a buffer which is bigger than + //necessary, result will have the correct length, but buf will still have it's original + //length +} + +@safe unittest +{ + // Test peek & length + import std.digest.crc; + auto hash = new WrapperDigest!CRC32(); + assert(hash.length == 4); + hash.put(cast(const(ubyte[]))"The quick brown fox jumps over the lazy dog"); + assert(hash.peek().toHexString() == "39A34F41"); + ubyte[5] buf; + assert(hash.peek(buf).toHexString() == "39A34F41"); +} + +/** + * Securely compares two digest representations while protecting against timing + * attacks. Do not use `==` to compare digest representations. + * + * The attack happens as follows: + * + * $(OL + * $(LI An attacker wants to send harmful data to your server, which + * requires a integrity HMAC SHA1 token signed with a secret.) + * $(LI The length of the token is known to be 40 characters long due to its format, + * so the attacker first sends `"0000000000000000000000000000000000000000"`, + * then `"1000000000000000000000000000000000000000"`, and so on.) + * $(LI The given HMAC token is compared with the expected token using the + * `==` string comparison, which returns `false` as soon as the first wrong + * element is found. If a wrong element is found, then a rejection is sent + * back to the sender.) + * $(LI Eventually, the attacker is able to determine the first character in + * the correct token because the sever takes slightly longer to return a + * rejection. This is due to the comparison moving on to second item in + * the two arrays, seeing they are different, and then sending the rejection.) + * $(LI It may seem like too small of a difference in time for the attacker + * to notice, but security researchers have shown that differences as + * small as $(LINK2 http://www.cs.rice.edu/~dwallach/pub/crosby-timing2009.pdf, + * 20µs can be reliably distinguished) even with network inconsistencies.) + * $(LI Repeat the process for each character until the attacker has the whole + * correct token and the server accepts the harmful data. This can be done + * in a week with the attacker pacing the attack to 10 requests per second + * with only one client.) + * ) + * + * This function defends against this attack by always comparing every single + * item in the array if the two arrays are the same length. Therefore, this + * function is always $(BIGOH n) for ranges of the same length. + * + * This attack can also be mitigated via rate limiting and banning IPs which have too + * many rejected requests. However, this does not completely solve the problem, + * as the attacker could be in control of a bot net. To fully defend against + * the timing attack, rate limiting, banning IPs, and using this function + * should be used together. + * + * Params: + * r1 = A digest representation + * r2 = A digest representation + * Returns: + * `true` if both representations are equal, `false` otherwise + * See_Also: + * $(LINK2 https://en.wikipedia.org/wiki/Timing_attack, The Wikipedia article + * on timing attacks). + */ +bool secureEqual(R1, R2)(R1 r1, R2 r2) +if (isInputRange!R1 && isInputRange!R2 && !isInfinite!R1 && !isInfinite!R2 && + (isIntegral!(ElementEncodingType!R1) || isSomeChar!(ElementEncodingType!R1)) && + !is(CommonType!(ElementEncodingType!R1, ElementEncodingType!R2) == void)) +{ + static if (hasLength!R1 && hasLength!R2) + if (r1.length != r2.length) + return false; + + int result; + + static if (isRandomAccessRange!R1 && isRandomAccessRange!R2 && + hasLength!R1 && hasLength!R2) + { + foreach (i; 0 .. r1.length) + result |= r1[i] ^ r2[i]; + } + else static if (hasLength!R1 && hasLength!R2) + { + // Lengths are the same so we can squeeze out a bit of performance + // by not checking if r2 is empty + for (; !r1.empty; r1.popFront(), r2.popFront()) + { + result |= r1.front ^ r2.front; + } + } + else + { + // Generic case, walk both ranges + for (; !r1.empty; r1.popFront(), r2.popFront()) + { + if (r2.empty) return false; + result |= r1.front ^ r2.front; + } + if (!r2.empty) return false; + } + + return result == 0; +} + +/// +@system pure unittest +{ + import std.digest.hmac : hmac; + import std.digest.sha : SHA1; + import std.string : representation; + + // a typical HMAC data integrity verification + auto secret = "A7GZIP6TAQA6OHM7KZ42KB9303CEY0MOV5DD6NTV".representation; + auto data = "data".representation; + + string hex1 = data.hmac!SHA1(secret).toHexString; + string hex2 = data.hmac!SHA1(secret).toHexString; + string hex3 = "data1".representation.hmac!SHA1(secret).toHexString; + + assert( secureEqual(hex1, hex2)); + assert(!secureEqual(hex1, hex3)); +} + +@system pure unittest +{ + import std.internal.test.dummyrange : ReferenceInputRange; + import std.range : takeExactly; + import std.string : representation; + import std.utf : byWchar, byDchar; + + { + auto hex1 = "02CA3484C375EDD3C0F08D3F50D119E61077".representation; + auto hex2 = "02CA3484C375EDD3C0F08D3F50D119E610779018".representation; + assert(!secureEqual(hex1, hex2)); + } + { + auto hex1 = "02CA3484C375EDD3C0F08D3F50D119E610779018"w.representation; + auto hex2 = "02CA3484C375EDD3C0F08D3F50D119E610779018"d.representation; + assert(secureEqual(hex1, hex2)); + } + { + auto hex1 = "02CA3484C375EDD3C0F08D3F50D119E610779018".byWchar; + auto hex2 = "02CA3484C375EDD3C0F08D3F50D119E610779018".byDchar; + assert(secureEqual(hex1, hex2)); + } + { + auto hex1 = "02CA3484C375EDD3C0F08D3F50D119E61077".byWchar; + auto hex2 = "02CA3484C375EDD3C0F08D3F50D119E610779018".byDchar; + assert(!secureEqual(hex1, hex2)); + } + { + auto hex1 = new ReferenceInputRange!int([0, 1, 2, 3, 4, 5, 6, 7, 8]).takeExactly(9); + auto hex2 = new ReferenceInputRange!int([0, 1, 2, 3, 4, 5, 6, 7, 8]).takeExactly(9); + assert(secureEqual(hex1, hex2)); + } + { + auto hex1 = new ReferenceInputRange!int([0, 1, 2, 3, 4, 5, 6, 7, 8]).takeExactly(9); + auto hex2 = new ReferenceInputRange!int([0, 1, 2, 3, 4, 5, 6, 7, 9]).takeExactly(9); + assert(!secureEqual(hex1, hex2)); + } +} diff --git a/libphobos/src/std/digest/ripemd.d b/libphobos/src/std/digest/ripemd.d new file mode 100644 index 0000000..c47a741 --- /dev/null +++ b/libphobos/src/std/digest/ripemd.d @@ -0,0 +1,762 @@ +/** + * Computes RIPEMD-160 hashes of arbitrary data. RIPEMD-160 hashes are 20 byte quantities + * that are like a checksum or CRC, but are more robust. + * +$(SCRIPT inhibitQuickIndex = 1;) + +$(DIVC quickindex, +$(BOOKTABLE , +$(TR $(TH Category) $(TH Functions) +) +$(TR $(TDNW Template API) $(TD $(MYREF RIPEMD160) +) +) +$(TR $(TDNW OOP API) $(TD $(MYREF RIPEMD160Digest)) +) +$(TR $(TDNW Helpers) $(TD $(MYREF ripemd160Of)) +) +) +) + + * This module conforms to the APIs defined in $(MREF std, digest). To understand the + * differences between the template and the OOP API, see $(MREF std, digest). + * + * This module publicly imports $(D std.digest) and can be used as a stand-alone + * module. + * + * License: $(HTTP www.boost.org/LICENSE_1_0.txt, Boost License 1.0). + * + * CTFE: + * Digests do not work in CTFE + * + * Authors: + * Kai Nacke $(BR) + * The algorithm was designed by Hans Dobbertin, Antoon Bosselaers, and Bart Preneel. $(BR) + * The D implementation is a direct translation of the ANSI C implementation by Antoon Bosselaers. + * + * References: + * $(UL + * $(LI $(LINK2 http://homes.esat.kuleuven.be/~bosselae/ripemd160.html, The hash function RIPEMD-160)) + * $(LI $(LINK2 http://en.wikipedia.org/wiki/RIPEMD-160, Wikipedia on RIPEMD-160)) + * ) + * + * Source: $(PHOBOSSRC std/digest/_ripemd.d) + * + */ + +module std.digest.ripemd; + +public import std.digest; + +/// +@safe unittest +{ + //Template API + import std.digest.md; + + ubyte[20] hash = ripemd160Of("abc"); + assert(toHexString(hash) == "8EB208F7E05D987A9B044A8E98C6B087F15A0BFC"); + + //Feeding data + ubyte[1024] data; + RIPEMD160 md; + md.start(); + md.put(data[]); + md.start(); //Start again + md.put(data[]); + hash = md.finish(); +} + +/// +@safe unittest +{ + //OOP API + import std.digest.md; + + auto md = new RIPEMD160Digest(); + ubyte[] hash = md.digest("abc"); + assert(toHexString(hash) == "8EB208F7E05D987A9B044A8E98C6B087F15A0BFC"); + + //Feeding data + ubyte[1024] data; + md.put(data[]); + md.reset(); //Start again + md.put(data[]); + hash = md.finish(); +} + +//rotateLeft rotates x left n bits +private uint rotateLeft(uint x, uint n) @safe pure nothrow @nogc +{ + // With recently added optimization to DMD (commit 32ea0206 at 07/28/11), this is translated to rol. + // No assembler required. + return (x << n) | (x >> (32-n)); +} + +/** + * Template API RIPEMD160 implementation. + * See $(D std.digest) for differences between template and OOP API. + */ +struct RIPEMD160 +{ + private: + // magic initialization constants + uint[5] _state = [0x67452301,0xefcdab89,0x98badcfe,0x10325476,0xc3d2e1f0]; // state (ABCDE) + ulong _count; //number of bits, modulo 2^64 + ubyte[64] _buffer; // input buffer + + static immutable ubyte[64] _padding = + [ + 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 + ]; + + // F, G, H, I and J are basic RIPEMD160 functions + static @safe pure nothrow @nogc + { + uint F(uint x, uint y, uint z) { return x ^ y ^ z; } + uint G(uint x, uint y, uint z) { return (x & y) | (~x & z); } + uint H(uint x, uint y, uint z) { return (x | ~y) ^ z; } + uint I(uint x, uint y, uint z) { return (x & z) | (y & ~z); } + uint J(uint x, uint y, uint z) { return x ^ (y | ~z); } + } + + /* + * FF, GG, HH, and II transformations for rounds 1, 2, 3, and 4. + * Rotation is separate from addition to prevent recomputation. + */ + + /* the ten basic operations FF() through III() */ + static void FF(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += F(b, c, d) + x; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void GG(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += G(b, c, d) + x + 0x5a827999UL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void HH(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += H(b, c, d) + x + 0x6ed9eba1UL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void II(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += I(b, c, d) + x + 0x8f1bbcdcUL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void JJ(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += J(b, c, d) + x + 0xa953fd4eUL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + /* + * FFF, GGG, HHH, and III transformations for parallel rounds 1, 2, 3, and 4. + * Rotation is separate from addition to prevent recomputation. + */ + + static void FFF(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += F(b, c, d) + x; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void GGG(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += G(b, c, d) + x + 0x7a6d76e9UL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void HHH(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += H(b, c, d) + x + 0x6d703ef3UL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void III(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += I(b, c, d) + x + 0x5c4dd124UL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + static void JJJ(ref uint a, uint b, ref uint c, uint d, uint e, uint x, uint s) + @safe pure nothrow @nogc + { + a += J(b, c, d) + x + 0x50a28be6UL; + a = rotateLeft(a, s) + e; + c = rotateLeft(c, 10); + } + + /* + * RIPEMD160 basic transformation. Transforms state based on block. + */ + + private void transform(const(ubyte[64])* block) + pure nothrow @nogc + { + uint aa = _state[0], + bb = _state[1], + cc = _state[2], + dd = _state[3], + ee = _state[4]; + uint aaa = _state[0], + bbb = _state[1], + ccc = _state[2], + ddd = _state[3], + eee = _state[4]; + + uint[16] x = void; + + version (BigEndian) + { + import std.bitmanip : littleEndianToNative; + + for (size_t i = 0; i < 16; i++) + { + x[i] = littleEndianToNative!uint(*cast(ubyte[4]*)&(*block)[i*4]); + } + } + else + { + (cast(ubyte*) x.ptr)[0 .. 64] = (cast(ubyte*) block)[0 .. 64]; + } + + /* round 1 */ + FF(aa, bb, cc, dd, ee, x[ 0], 11); + FF(ee, aa, bb, cc, dd, x[ 1], 14); + FF(dd, ee, aa, bb, cc, x[ 2], 15); + FF(cc, dd, ee, aa, bb, x[ 3], 12); + FF(bb, cc, dd, ee, aa, x[ 4], 5); + FF(aa, bb, cc, dd, ee, x[ 5], 8); + FF(ee, aa, bb, cc, dd, x[ 6], 7); + FF(dd, ee, aa, bb, cc, x[ 7], 9); + FF(cc, dd, ee, aa, bb, x[ 8], 11); + FF(bb, cc, dd, ee, aa, x[ 9], 13); + FF(aa, bb, cc, dd, ee, x[10], 14); + FF(ee, aa, bb, cc, dd, x[11], 15); + FF(dd, ee, aa, bb, cc, x[12], 6); + FF(cc, dd, ee, aa, bb, x[13], 7); + FF(bb, cc, dd, ee, aa, x[14], 9); + FF(aa, bb, cc, dd, ee, x[15], 8); + + /* round 2 */ + GG(ee, aa, bb, cc, dd, x[ 7], 7); + GG(dd, ee, aa, bb, cc, x[ 4], 6); + GG(cc, dd, ee, aa, bb, x[13], 8); + GG(bb, cc, dd, ee, aa, x[ 1], 13); + GG(aa, bb, cc, dd, ee, x[10], 11); + GG(ee, aa, bb, cc, dd, x[ 6], 9); + GG(dd, ee, aa, bb, cc, x[15], 7); + GG(cc, dd, ee, aa, bb, x[ 3], 15); + GG(bb, cc, dd, ee, aa, x[12], 7); + GG(aa, bb, cc, dd, ee, x[ 0], 12); + GG(ee, aa, bb, cc, dd, x[ 9], 15); + GG(dd, ee, aa, bb, cc, x[ 5], 9); + GG(cc, dd, ee, aa, bb, x[ 2], 11); + GG(bb, cc, dd, ee, aa, x[14], 7); + GG(aa, bb, cc, dd, ee, x[11], 13); + GG(ee, aa, bb, cc, dd, x[ 8], 12); + + /* round 3 */ + HH(dd, ee, aa, bb, cc, x[ 3], 11); + HH(cc, dd, ee, aa, bb, x[10], 13); + HH(bb, cc, dd, ee, aa, x[14], 6); + HH(aa, bb, cc, dd, ee, x[ 4], 7); + HH(ee, aa, bb, cc, dd, x[ 9], 14); + HH(dd, ee, aa, bb, cc, x[15], 9); + HH(cc, dd, ee, aa, bb, x[ 8], 13); + HH(bb, cc, dd, ee, aa, x[ 1], 15); + HH(aa, bb, cc, dd, ee, x[ 2], 14); + HH(ee, aa, bb, cc, dd, x[ 7], 8); + HH(dd, ee, aa, bb, cc, x[ 0], 13); + HH(cc, dd, ee, aa, bb, x[ 6], 6); + HH(bb, cc, dd, ee, aa, x[13], 5); + HH(aa, bb, cc, dd, ee, x[11], 12); + HH(ee, aa, bb, cc, dd, x[ 5], 7); + HH(dd, ee, aa, bb, cc, x[12], 5); + + /* round 4 */ + II(cc, dd, ee, aa, bb, x[ 1], 11); + II(bb, cc, dd, ee, aa, x[ 9], 12); + II(aa, bb, cc, dd, ee, x[11], 14); + II(ee, aa, bb, cc, dd, x[10], 15); + II(dd, ee, aa, bb, cc, x[ 0], 14); + II(cc, dd, ee, aa, bb, x[ 8], 15); + II(bb, cc, dd, ee, aa, x[12], 9); + II(aa, bb, cc, dd, ee, x[ 4], 8); + II(ee, aa, bb, cc, dd, x[13], 9); + II(dd, ee, aa, bb, cc, x[ 3], 14); + II(cc, dd, ee, aa, bb, x[ 7], 5); + II(bb, cc, dd, ee, aa, x[15], 6); + II(aa, bb, cc, dd, ee, x[14], 8); + II(ee, aa, bb, cc, dd, x[ 5], 6); + II(dd, ee, aa, bb, cc, x[ 6], 5); + II(cc, dd, ee, aa, bb, x[ 2], 12); + + /* round 5 */ + JJ(bb, cc, dd, ee, aa, x[ 4], 9); + JJ(aa, bb, cc, dd, ee, x[ 0], 15); + JJ(ee, aa, bb, cc, dd, x[ 5], 5); + JJ(dd, ee, aa, bb, cc, x[ 9], 11); + JJ(cc, dd, ee, aa, bb, x[ 7], 6); + JJ(bb, cc, dd, ee, aa, x[12], 8); + JJ(aa, bb, cc, dd, ee, x[ 2], 13); + JJ(ee, aa, bb, cc, dd, x[10], 12); + JJ(dd, ee, aa, bb, cc, x[14], 5); + JJ(cc, dd, ee, aa, bb, x[ 1], 12); + JJ(bb, cc, dd, ee, aa, x[ 3], 13); + JJ(aa, bb, cc, dd, ee, x[ 8], 14); + JJ(ee, aa, bb, cc, dd, x[11], 11); + JJ(dd, ee, aa, bb, cc, x[ 6], 8); + JJ(cc, dd, ee, aa, bb, x[15], 5); + JJ(bb, cc, dd, ee, aa, x[13], 6); + + /* parallel round 1 */ + JJJ(aaa, bbb, ccc, ddd, eee, x[ 5], 8); + JJJ(eee, aaa, bbb, ccc, ddd, x[14], 9); + JJJ(ddd, eee, aaa, bbb, ccc, x[ 7], 9); + JJJ(ccc, ddd, eee, aaa, bbb, x[ 0], 11); + JJJ(bbb, ccc, ddd, eee, aaa, x[ 9], 13); + JJJ(aaa, bbb, ccc, ddd, eee, x[ 2], 15); + JJJ(eee, aaa, bbb, ccc, ddd, x[11], 15); + JJJ(ddd, eee, aaa, bbb, ccc, x[ 4], 5); + JJJ(ccc, ddd, eee, aaa, bbb, x[13], 7); + JJJ(bbb, ccc, ddd, eee, aaa, x[ 6], 7); + JJJ(aaa, bbb, ccc, ddd, eee, x[15], 8); + JJJ(eee, aaa, bbb, ccc, ddd, x[ 8], 11); + JJJ(ddd, eee, aaa, bbb, ccc, x[ 1], 14); + JJJ(ccc, ddd, eee, aaa, bbb, x[10], 14); + JJJ(bbb, ccc, ddd, eee, aaa, x[ 3], 12); + JJJ(aaa, bbb, ccc, ddd, eee, x[12], 6); + + /* parallel round 2 */ + III(eee, aaa, bbb, ccc, ddd, x[ 6], 9); + III(ddd, eee, aaa, bbb, ccc, x[11], 13); + III(ccc, ddd, eee, aaa, bbb, x[ 3], 15); + III(bbb, ccc, ddd, eee, aaa, x[ 7], 7); + III(aaa, bbb, ccc, ddd, eee, x[ 0], 12); + III(eee, aaa, bbb, ccc, ddd, x[13], 8); + III(ddd, eee, aaa, bbb, ccc, x[ 5], 9); + III(ccc, ddd, eee, aaa, bbb, x[10], 11); + III(bbb, ccc, ddd, eee, aaa, x[14], 7); + III(aaa, bbb, ccc, ddd, eee, x[15], 7); + III(eee, aaa, bbb, ccc, ddd, x[ 8], 12); + III(ddd, eee, aaa, bbb, ccc, x[12], 7); + III(ccc, ddd, eee, aaa, bbb, x[ 4], 6); + III(bbb, ccc, ddd, eee, aaa, x[ 9], 15); + III(aaa, bbb, ccc, ddd, eee, x[ 1], 13); + III(eee, aaa, bbb, ccc, ddd, x[ 2], 11); + + /* parallel round 3 */ + HHH(ddd, eee, aaa, bbb, ccc, x[15], 9); + HHH(ccc, ddd, eee, aaa, bbb, x[ 5], 7); + HHH(bbb, ccc, ddd, eee, aaa, x[ 1], 15); + HHH(aaa, bbb, ccc, ddd, eee, x[ 3], 11); + HHH(eee, aaa, bbb, ccc, ddd, x[ 7], 8); + HHH(ddd, eee, aaa, bbb, ccc, x[14], 6); + HHH(ccc, ddd, eee, aaa, bbb, x[ 6], 6); + HHH(bbb, ccc, ddd, eee, aaa, x[ 9], 14); + HHH(aaa, bbb, ccc, ddd, eee, x[11], 12); + HHH(eee, aaa, bbb, ccc, ddd, x[ 8], 13); + HHH(ddd, eee, aaa, bbb, ccc, x[12], 5); + HHH(ccc, ddd, eee, aaa, bbb, x[ 2], 14); + HHH(bbb, ccc, ddd, eee, aaa, x[10], 13); + HHH(aaa, bbb, ccc, ddd, eee, x[ 0], 13); + HHH(eee, aaa, bbb, ccc, ddd, x[ 4], 7); + HHH(ddd, eee, aaa, bbb, ccc, x[13], 5); + + /* parallel round 4 */ + GGG(ccc, ddd, eee, aaa, bbb, x[ 8], 15); + GGG(bbb, ccc, ddd, eee, aaa, x[ 6], 5); + GGG(aaa, bbb, ccc, ddd, eee, x[ 4], 8); + GGG(eee, aaa, bbb, ccc, ddd, x[ 1], 11); + GGG(ddd, eee, aaa, bbb, ccc, x[ 3], 14); + GGG(ccc, ddd, eee, aaa, bbb, x[11], 14); + GGG(bbb, ccc, ddd, eee, aaa, x[15], 6); + GGG(aaa, bbb, ccc, ddd, eee, x[ 0], 14); + GGG(eee, aaa, bbb, ccc, ddd, x[ 5], 6); + GGG(ddd, eee, aaa, bbb, ccc, x[12], 9); + GGG(ccc, ddd, eee, aaa, bbb, x[ 2], 12); + GGG(bbb, ccc, ddd, eee, aaa, x[13], 9); + GGG(aaa, bbb, ccc, ddd, eee, x[ 9], 12); + GGG(eee, aaa, bbb, ccc, ddd, x[ 7], 5); + GGG(ddd, eee, aaa, bbb, ccc, x[10], 15); + GGG(ccc, ddd, eee, aaa, bbb, x[14], 8); + + /* parallel round 5 */ + FFF(bbb, ccc, ddd, eee, aaa, x[12] , 8); + FFF(aaa, bbb, ccc, ddd, eee, x[15] , 5); + FFF(eee, aaa, bbb, ccc, ddd, x[10] , 12); + FFF(ddd, eee, aaa, bbb, ccc, x[ 4] , 9); + FFF(ccc, ddd, eee, aaa, bbb, x[ 1] , 12); + FFF(bbb, ccc, ddd, eee, aaa, x[ 5] , 5); + FFF(aaa, bbb, ccc, ddd, eee, x[ 8] , 14); + FFF(eee, aaa, bbb, ccc, ddd, x[ 7] , 6); + FFF(ddd, eee, aaa, bbb, ccc, x[ 6] , 8); + FFF(ccc, ddd, eee, aaa, bbb, x[ 2] , 13); + FFF(bbb, ccc, ddd, eee, aaa, x[13] , 6); + FFF(aaa, bbb, ccc, ddd, eee, x[14] , 5); + FFF(eee, aaa, bbb, ccc, ddd, x[ 0] , 15); + FFF(ddd, eee, aaa, bbb, ccc, x[ 3] , 13); + FFF(ccc, ddd, eee, aaa, bbb, x[ 9] , 11); + FFF(bbb, ccc, ddd, eee, aaa, x[11] , 11); + + /* combine results */ + ddd += cc + _state[1]; /* final result for _state[0] */ + _state[1] = _state[2] + dd + eee; + _state[2] = _state[3] + ee + aaa; + _state[3] = _state[4] + aa + bbb; + _state[4] = _state[0] + bb + ccc; + _state[0] = ddd; + + //Zeroize sensitive information. + x[] = 0; + } + + public: + enum blockSize = 512; + + /** + * Use this to feed the digest with data. + * Also implements the $(REF isOutputRange, std,range,primitives) + * interface for $(D ubyte) and $(D const(ubyte)[]). + * + * Example: + * ---- + * RIPEMD160 dig; + * dig.put(cast(ubyte) 0); //single ubyte + * dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic + * ubyte[10] buf; + * dig.put(buf); //buffer + * ---- + */ + void put(scope const(ubyte)[] data...) @trusted pure nothrow @nogc + { + uint i, index, partLen; + auto inputLen = data.length; + + //Compute number of bytes mod 64 + index = (cast(uint)_count >> 3) & (64 - 1); + + //Update number of bits + _count += inputLen * 8; + + partLen = 64 - index; + + //Transform as many times as possible + if (inputLen >= partLen) + { + (&_buffer[index])[0 .. partLen] = data.ptr[0 .. partLen]; + transform(&_buffer); + + for (i = partLen; i + 63 < inputLen; i += 64) + { + transform(cast(const(ubyte[64])*)(data[i .. i + 64].ptr)); + } + + index = 0; + } + else + { + i = 0; + } + + /* Buffer remaining input */ + if (inputLen - i) + (&_buffer[index])[0 .. inputLen-i] = (&data[i])[0 .. inputLen-i]; + } + + /** + * Used to (re)initialize the RIPEMD160 digest. + * + * Note: + * For this RIPEMD160 Digest implementation calling start after default construction + * is not necessary. Calling start is only necessary to reset the Digest. + * + * Generic code which deals with different Digest types should always call start though. + * + * Example: + * -------- + * RIPEMD160 digest; + * //digest.start(); //Not necessary + * digest.put(0); + * -------- + */ + void start() @safe pure nothrow @nogc + { + this = RIPEMD160.init; + } + + /** + * Returns the finished RIPEMD160 hash. This also calls $(LREF start) to + * reset the internal state. + * + * Example: + * -------- + * //Simple example + * RIPEMD160 hash; + * hash.start(); + * hash.put(cast(ubyte) 0); + * ubyte[20] result = hash.finish(); + * assert(toHexString(result) == "C81B94933420221A7AC004A90242D8B1D3E5070D"); + * -------- + */ + ubyte[20] finish() @trusted pure nothrow @nogc + { + import std.bitmanip : nativeToLittleEndian; + + ubyte[20] data = void; + ubyte[8] bits = void; + uint index, padLen; + + //Save number of bits + bits[0 .. 8] = nativeToLittleEndian(_count)[]; + + //Pad out to 56 mod 64 + index = (cast(uint)_count >> 3) & (64 - 1); + padLen = (index < 56) ? (56 - index) : (120 - index); + put(_padding[0 .. padLen]); + + //Append length (before padding) + put(bits); + + //Store state in digest + data[0 .. 4] = nativeToLittleEndian(_state[0])[]; + data[4 .. 8] = nativeToLittleEndian(_state[1])[]; + data[8 .. 12] = nativeToLittleEndian(_state[2])[]; + data[12 .. 16] = nativeToLittleEndian(_state[3])[]; + data[16 .. 20] = nativeToLittleEndian(_state[4])[]; + + /* Zeroize sensitive information. */ + start(); + return data; + } +} + +/// +@safe unittest +{ + //Simple example, hashing a string using ripemd160Of helper function + ubyte[20] hash = ripemd160Of("abc"); + //Let's get a hash string + assert(toHexString(hash) == "8EB208F7E05D987A9B044A8E98C6B087F15A0BFC"); +} + +/// +@safe unittest +{ + //Using the basic API + RIPEMD160 hash; + hash.start(); + ubyte[1024] data; + //Initialize data here... + hash.put(data); + ubyte[20] result = hash.finish(); +} + +/// +@safe unittest +{ + //Let's use the template features: + void doSomething(T)(ref T hash) + if (isDigest!T) + { + hash.put(cast(ubyte) 0); + } + RIPEMD160 md; + md.start(); + doSomething(md); + assert(toHexString(md.finish()) == "C81B94933420221A7AC004A90242D8B1D3E5070D"); +} + +/// +@safe unittest +{ + //Simple example + RIPEMD160 hash; + hash.start(); + hash.put(cast(ubyte) 0); + ubyte[20] result = hash.finish(); + assert(toHexString(result) == "C81B94933420221A7AC004A90242D8B1D3E5070D"); +} + +@safe unittest +{ + assert(isDigest!RIPEMD160); +} + +@system unittest +{ + import std.range; + + ubyte[20] digest; + + RIPEMD160 md; + md.put(cast(ubyte[])"abcdef"); + md.start(); + md.put(cast(ubyte[])""); + assert(md.finish() == cast(ubyte[]) x"9c1185a5c5e9fc54612808977ee8f548b2258d31"); + + digest = ripemd160Of(""); + assert(digest == cast(ubyte[]) x"9c1185a5c5e9fc54612808977ee8f548b2258d31"); + + digest = ripemd160Of("a"); + assert(digest == cast(ubyte[]) x"0bdc9d2d256b3ee9daae347be6f4dc835a467ffe"); + + digest = ripemd160Of("abc"); + assert(digest == cast(ubyte[]) x"8eb208f7e05d987a9b044a8e98c6b087f15a0bfc"); + + digest = ripemd160Of("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + assert(digest == cast(ubyte[]) x"12a053384a9c0c88e405a06c27dcf49ada62eb2b"); + + digest = ripemd160Of("message digest"); + assert(digest == cast(ubyte[]) x"5d0689ef49d2fae572b881b123a85ffa21595f36"); + + digest = ripemd160Of("abcdefghijklmnopqrstuvwxyz"); + assert(digest == cast(ubyte[]) x"f71c27109c692c1b56bbdceb5b9d2865b3708dbc"); + + digest = ripemd160Of("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + assert(digest == cast(ubyte[]) x"b0e20b6e3116640286ed3a87a5713079b21f5189"); + + digest = ripemd160Of("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + assert(digest == cast(ubyte[]) x"9b752e45573d4b39f4dbd3323cab82bf63326bfb"); + + assert(toHexString(cast(ubyte[20]) x"f71c27109c692c1b56bbdceb5b9d2865b3708dbc") + == "F71C27109C692C1B56BBDCEB5B9D2865B3708DBC"); + + ubyte[] onemilliona = new ubyte[1000000]; + onemilliona[] = 'a'; + digest = ripemd160Of(onemilliona); + assert(digest == cast(ubyte[]) x"52783243c1697bdbe16d37f97f68f08325dc1528"); + + auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); + digest = ripemd160Of(oneMillionRange); + assert(digest == cast(ubyte[]) x"52783243c1697bdbe16d37f97f68f08325dc1528"); +} + +/** + * This is a convenience alias for $(REF digest, std,digest) using the + * RIPEMD160 implementation. + */ +//simple alias doesn't work here, hope this gets inlined... +auto ripemd160Of(T...)(T data) +{ + return digest!(RIPEMD160, T)(data); +} + +/// +@safe unittest +{ + ubyte[20] hash = ripemd160Of("abc"); + assert(hash == digest!RIPEMD160("abc")); +} + +/** + * OOP API RIPEMD160 implementation. + * See $(D std.digest) for differences between template and OOP API. + * + * This is an alias for $(D $(REF WrapperDigest, std,digest)!RIPEMD160), + * see there for more information. + */ +alias RIPEMD160Digest = WrapperDigest!RIPEMD160; + +/// +@safe unittest +{ + //Simple example, hashing a string using Digest.digest helper function + auto md = new RIPEMD160Digest(); + ubyte[] hash = md.digest("abc"); + //Let's get a hash string + assert(toHexString(hash) == "8EB208F7E05D987A9B044A8E98C6B087F15A0BFC"); +} + +/// +@system unittest +{ + //Let's use the OOP features: + void test(Digest dig) + { + dig.put(cast(ubyte) 0); + } + auto md = new RIPEMD160Digest(); + test(md); + + //Let's use a custom buffer: + ubyte[20] buf; + ubyte[] result = md.finish(buf[]); + assert(toHexString(result) == "C81B94933420221A7AC004A90242D8B1D3E5070D"); +} + +@system unittest +{ + auto md = new RIPEMD160Digest(); + + md.put(cast(ubyte[])"abcdef"); + md.reset(); + md.put(cast(ubyte[])""); + assert(md.finish() == cast(ubyte[]) x"9c1185a5c5e9fc54612808977ee8f548b2258d31"); + + md.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + ubyte[20] result; + auto result2 = md.finish(result[]); + assert(result[0 .. 20] == result2 && result2 == cast(ubyte[]) x"f71c27109c692c1b56bbdceb5b9d2865b3708dbc"); + + debug + { + import std.exception; + assertThrown!Error(md.finish(result[0 .. 19])); + } + + assert(md.length == 20); + + assert(md.digest("") == cast(ubyte[]) x"9c1185a5c5e9fc54612808977ee8f548b2258d31"); + + assert(md.digest("a") == cast(ubyte[]) x"0bdc9d2d256b3ee9daae347be6f4dc835a467ffe"); + + assert(md.digest("abc") == cast(ubyte[]) x"8eb208f7e05d987a9b044a8e98c6b087f15a0bfc"); + + assert(md.digest("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq") + == cast(ubyte[]) x"12a053384a9c0c88e405a06c27dcf49ada62eb2b"); + + assert(md.digest("message digest") == cast(ubyte[]) x"5d0689ef49d2fae572b881b123a85ffa21595f36"); + + assert(md.digest("abcdefghijklmnopqrstuvwxyz") + == cast(ubyte[]) x"f71c27109c692c1b56bbdceb5b9d2865b3708dbc"); + + assert(md.digest("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") + == cast(ubyte[]) x"b0e20b6e3116640286ed3a87a5713079b21f5189"); + + assert(md.digest("1234567890123456789012345678901234567890", + "1234567890123456789012345678901234567890") + == cast(ubyte[]) x"9b752e45573d4b39f4dbd3323cab82bf63326bfb"); + + assert(md.digest(new ubyte[160/8]) // 160 zero bits + == cast(ubyte[]) x"5c00bd4aca04a9057c09b20b05f723f2e23deb65"); +} diff --git a/libphobos/src/std/digest/sha.d b/libphobos/src/std/digest/sha.d new file mode 100644 index 0000000..671e07b --- /dev/null +++ b/libphobos/src/std/digest/sha.d @@ -0,0 +1,1291 @@ +// Written in the D programming language. +/** + * Computes SHA1 and SHA2 hashes of arbitrary data. SHA hashes are 20 to 64 byte + * quantities (depending on the SHA algorithm) that are like a checksum or CRC, + * but are more robust. + * +$(SCRIPT inhibitQuickIndex = 1;) + +$(DIVC quickindex, +$(BOOKTABLE , +$(TR $(TH Category) $(TH Functions) +) +$(TR $(TDNW Template API) $(TD $(MYREF SHA1) +) +) +$(TR $(TDNW OOP API) $(TD $(MYREF SHA1Digest)) +) +$(TR $(TDNW Helpers) $(TD $(MYREF sha1Of)) +) +) +) + + * SHA2 comes in several different versions, all supported by this module: + * SHA-224, SHA-256, SHA-384, SHA-512, SHA-512/224 and SHA-512/256. + * + * This module conforms to the APIs defined in $(MREF std, digest). To understand the + * differences between the template and the OOP API, see $(MREF std, digest). + * + * This module publicly imports $(D std.digest) and can be used as a stand-alone + * module. + * + * License: $(HTTP www.boost.org/LICENSE_1_0.txt, Boost License 1.0). + * + * CTFE: + * Digests do not work in CTFE + * + * Authors: + * The routines and algorithms are derived from the + * $(I Secure Hash Signature Standard (SHS) (FIPS PUB 180-2)). $(BR ) + * Kai Nacke, Johannes Pfau, Nick Sabalausky + * + * References: + * $(UL + * $(LI $(LINK2 http://csrc.nist.gov/publications/fips/fips180-2/fips180-2withchangenotice.pdf, FIPS PUB180-2)) + * $(LI $(LINK2 http://software.intel.com/en-us/articles/improving-the-performance-of-the-secure-hash-algorithm-1/, Fast implementation of SHA1)) + * $(LI $(LINK2 http://en.wikipedia.org/wiki/Secure_Hash_Algorithm, Wikipedia article about SHA)) + * ) + * + * Source: $(PHOBOSSRC std/digest/_sha.d) + * + */ + +/* Copyright Kai Nacke 2012. + * Distributed under the Boost Software License, Version 1.0. + * (See accompanying file LICENSE_1_0.txt or copy at + * http://www.boost.org/LICENSE_1_0.txt) + */ +module std.digest.sha; + +/// +@safe unittest +{ + //Template API + import std.digest.sha; + + ubyte[20] hash1 = sha1Of("abc"); + assert(toHexString(hash1) == "A9993E364706816ABA3E25717850C26C9CD0D89D"); + + ubyte[28] hash224 = sha224Of("abc"); + assert(toHexString(hash224) == "23097D223405D8228642A477BDA255B32AADBCE4BDA0B3F7E36C9DA7"); + + //Feeding data + ubyte[1024] data; + SHA1 sha1; + sha1.start(); + sha1.put(data[]); + sha1.start(); //Start again + sha1.put(data[]); + hash1 = sha1.finish(); +} + +/// +@safe unittest +{ + //OOP API + import std.digest.sha; + + auto sha1 = new SHA1Digest(); + ubyte[] hash1 = sha1.digest("abc"); + assert(toHexString(hash1) == "A9993E364706816ABA3E25717850C26C9CD0D89D"); + + auto sha224 = new SHA224Digest(); + ubyte[] hash224 = sha224.digest("abc"); + assert(toHexString(hash224) == "23097D223405D8228642A477BDA255B32AADBCE4BDA0B3F7E36C9DA7"); + + //Feeding data + ubyte[1024] data; + sha1.put(data[]); + sha1.reset(); //Start again + sha1.put(data[]); + hash1 = sha1.finish(); +} + +version (Win64) +{ + // wrong calling convention +} +else version (D_InlineAsm_X86) +{ + version (D_PIC) {} // Bugzilla 9378 + else private version = USE_SSSE3; +} +else version (D_InlineAsm_X86_64) +{ + private version = USE_SSSE3; +} + +version (LittleEndian) import core.bitop : bswap; + + +version (unittest) +{ + import std.exception; +} + + +public import std.digest; + +/* + * Helper methods for encoding the buffer. + * Can be removed if the optimizer can inline the methods from std.bitmanip. + */ +private ubyte[8] nativeToBigEndian(ulong val) @trusted pure nothrow @nogc +{ + version (LittleEndian) + immutable ulong res = (cast(ulong) bswap(cast(uint) val)) << 32 | bswap(cast(uint) (val >> 32)); + else + immutable ulong res = val; + return *cast(ubyte[8]*) &res; +} + +private ubyte[4] nativeToBigEndian(uint val) @trusted pure nothrow @nogc +{ + version (LittleEndian) + immutable uint res = bswap(val); + else + immutable uint res = val; + return *cast(ubyte[4]*) &res; +} + +private ulong bigEndianToNative(ubyte[8] val) @trusted pure nothrow @nogc +{ + version (LittleEndian) + { + import std.bitmanip : bigEndianToNative; + return bigEndianToNative!ulong(val); + } + else + return *cast(ulong*) &val; +} + +private uint bigEndianToNative(ubyte[4] val) @trusted pure nothrow @nogc +{ + version (LittleEndian) + return bswap(*cast(uint*) &val); + else + return *cast(uint*) &val; +} + +//rotateLeft rotates x left n bits +private uint rotateLeft(uint x, uint n) @safe pure nothrow @nogc +{ + // With recently added optimization to DMD (commit 32ea0206 at 07/28/11), this is translated to rol. + // No assembler required. + return (x << n) | (x >> (32-n)); +} + +//rotateRight rotates x right n bits +private uint rotateRight(uint x, uint n) @safe pure nothrow @nogc +{ + return (x >> n) | (x << (32-n)); +} +private ulong rotateRight(ulong x, uint n) @safe pure nothrow @nogc +{ + return (x >> n) | (x << (64-n)); +} + +/** + * Template API SHA1/SHA2 implementation. Supports: SHA-1, SHA-224, SHA-256, + * SHA-384, SHA-512, SHA-512/224 and SHA-512/256. + * + * The hashBlockSize and digestSize are in bits. However, it's likely easier to + * simply use the convenience aliases: SHA1, SHA224, SHA256, SHA384, SHA512, + * SHA512_224 and SHA512_256. + * + * See $(D std.digest) for differences between template and OOP API. + */ +struct SHA(uint hashBlockSize, uint digestSize) +{ + enum blockSize = hashBlockSize; + + static assert(blockSize == 512 || blockSize == 1024, + "Invalid SHA blockSize, must be 512 or 1024"); + static assert(digestSize == 160 || digestSize == 224 || digestSize == 256 || digestSize == 384 || digestSize == 512, + "Invalid SHA digestSize, must be 224, 256, 384 or 512"); + static assert(!(blockSize == 512 && digestSize > 256), + "Invalid SHA digestSize for a blockSize of 512. The digestSize must be 160, 224 or 256."); + static assert(!(blockSize == 1024 && digestSize < 224), + "Invalid SHA digestSize for a blockSize of 1024. The digestSize must be 224, 256, 384 or 512."); + + static if (digestSize == 160) /* SHA-1 */ + { + version (USE_SSSE3) + { + import core.cpuid : ssse3; + import std.internal.digest.sha_SSSE3 : sse3_constants=constants, transformSSSE3; + + static void transform(uint[5]* state, const(ubyte[64])* block) pure nothrow @nogc + { + if (ssse3) + { + version (D_InlineAsm_X86_64) + // constants as extra argument for PIC, see Bugzilla 9378 + transformSSSE3(state, block, &sse3_constants); + else + transformSSSE3(state, block); + } + else + transformX86(state, block); + } + } + else + { + alias transform = transformX86; + } + } + else static if (blockSize == 512) /* SHA-224, SHA-256 */ + alias transform = transformSHA2!uint; + else static if (blockSize == 1024) /* SHA-384, SHA-512, SHA-512/224, SHA-512/256 */ + alias transform = transformSHA2!ulong; + else + static assert(0); + + private: + /* magic initialization constants - state (ABCDEFGH) */ + static if (blockSize == 512 && digestSize == 160) /* SHA-1 */ + { + uint[5] state = + [0x67452301,0xefcdab89,0x98badcfe,0x10325476,0xc3d2e1f0]; + } + else static if (blockSize == 512 && digestSize == 224) /* SHA-224 */ + { + uint[8] state = [ + 0xc1059ed8, 0x367cd507, 0x3070dd17, 0xf70e5939, + 0xffc00b31, 0x68581511, 0x64f98fa7, 0xbefa4fa4, + ]; + } + else static if (blockSize == 512 && digestSize == 256) /* SHA-256 */ + { + uint[8] state = [ + 0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, + 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19, + ]; + } + else static if (blockSize == 1024 && digestSize == 224) /* SHA-512/224 */ + { + ulong[8] state = [ + 0x8C3D37C8_19544DA2, 0x73E19966_89DCD4D6, + 0x1DFAB7AE_32FF9C82, 0x679DD514_582F9FCF, + 0x0F6D2B69_7BD44DA8, 0x77E36F73_04C48942, + 0x3F9D85A8_6A1D36C8, 0x1112E6AD_91D692A1, + ]; + } + else static if (blockSize == 1024 && digestSize == 256) /* SHA-512/256 */ + { + ulong[8] state = [ + 0x22312194_FC2BF72C, 0x9F555FA3_C84C64C2, + 0x2393B86B_6F53B151, 0x96387719_5940EABD, + 0x96283EE2_A88EFFE3, 0xBE5E1E25_53863992, + 0x2B0199FC_2C85B8AA, 0x0EB72DDC_81C52CA2, + ]; + } + else static if (blockSize == 1024 && digestSize == 384) /* SHA-384 */ + { + ulong[8] state = [ + 0xcbbb9d5d_c1059ed8, 0x629a292a_367cd507, + 0x9159015a_3070dd17, 0x152fecd8_f70e5939, + 0x67332667_ffc00b31, 0x8eb44a87_68581511, + 0xdb0c2e0d_64f98fa7, 0x47b5481d_befa4fa4, + ]; + } + else static if (blockSize == 1024 && digestSize == 512) /* SHA-512 */ + { + ulong[8] state = [ + 0x6a09e667_f3bcc908, 0xbb67ae85_84caa73b, + 0x3c6ef372_fe94f82b, 0xa54ff53a_5f1d36f1, + 0x510e527f_ade682d1, 0x9b05688c_2b3e6c1f, + 0x1f83d9ab_fb41bd6b, 0x5be0cd19_137e2179, + ]; + } + else + static assert(0); + + /* constants */ + static if (blockSize == 512) + { + static immutable uint[64] constants = [ + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, + 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, + 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, + 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, + 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, + 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, + 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, + 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, + ]; + } + else static if (blockSize == 1024) + { + static immutable ulong[80] constants = [ + 0x428a2f98_d728ae22, 0x71374491_23ef65cd, 0xb5c0fbcf_ec4d3b2f, 0xe9b5dba5_8189dbbc, + 0x3956c25b_f348b538, 0x59f111f1_b605d019, 0x923f82a4_af194f9b, 0xab1c5ed5_da6d8118, + 0xd807aa98_a3030242, 0x12835b01_45706fbe, 0x243185be_4ee4b28c, 0x550c7dc3_d5ffb4e2, + 0x72be5d74_f27b896f, 0x80deb1fe_3b1696b1, 0x9bdc06a7_25c71235, 0xc19bf174_cf692694, + 0xe49b69c1_9ef14ad2, 0xefbe4786_384f25e3, 0x0fc19dc6_8b8cd5b5, 0x240ca1cc_77ac9c65, + 0x2de92c6f_592b0275, 0x4a7484aa_6ea6e483, 0x5cb0a9dc_bd41fbd4, 0x76f988da_831153b5, + 0x983e5152_ee66dfab, 0xa831c66d_2db43210, 0xb00327c8_98fb213f, 0xbf597fc7_beef0ee4, + 0xc6e00bf3_3da88fc2, 0xd5a79147_930aa725, 0x06ca6351_e003826f, 0x14292967_0a0e6e70, + 0x27b70a85_46d22ffc, 0x2e1b2138_5c26c926, 0x4d2c6dfc_5ac42aed, 0x53380d13_9d95b3df, + 0x650a7354_8baf63de, 0x766a0abb_3c77b2a8, 0x81c2c92e_47edaee6, 0x92722c85_1482353b, + 0xa2bfe8a1_4cf10364, 0xa81a664b_bc423001, 0xc24b8b70_d0f89791, 0xc76c51a3_0654be30, + 0xd192e819_d6ef5218, 0xd6990624_5565a910, 0xf40e3585_5771202a, 0x106aa070_32bbd1b8, + 0x19a4c116_b8d2d0c8, 0x1e376c08_5141ab53, 0x2748774c_df8eeb99, 0x34b0bcb5_e19b48a8, + 0x391c0cb3_c5c95a63, 0x4ed8aa4a_e3418acb, 0x5b9cca4f_7763e373, 0x682e6ff3_d6b2b8a3, + 0x748f82ee_5defb2fc, 0x78a5636f_43172f60, 0x84c87814_a1f0ab72, 0x8cc70208_1a6439ec, + 0x90befffa_23631e28, 0xa4506ceb_de82bde9, 0xbef9a3f7_b2c67915, 0xc67178f2_e372532b, + 0xca273ece_ea26619c, 0xd186b8c7_21c0c207, 0xeada7dd6_cde0eb1e, 0xf57d4f7f_ee6ed178, + 0x06f067aa_72176fba, 0x0a637dc5_a2c898a6, 0x113f9804_bef90dae, 0x1b710b35_131c471b, + 0x28db77f5_23047d84, 0x32caab7b_40c72493, 0x3c9ebe0a_15c9bebc, 0x431d67c4_9c100d4c, + 0x4cc5d4be_cb3e42b6, 0x597f299c_fc657e2a, 0x5fcb6fab_3ad6faec, 0x6c44198c_4a475817, + ]; + } + else + static assert(0); + + /* + * number of bits, modulo 2^64 (ulong[1]) or 2^128 (ulong[2]), + * should just use ucent instead of ulong[2] once it's available + */ + ulong[blockSize/512] count; + ubyte[blockSize/8] buffer; /* input buffer */ + + static immutable ubyte[128] padding = + [ + 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + + 0x00, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 + ]; + + /* + * Basic SHA1/SHA2 functions. + */ + static @safe pure nothrow @nogc + { + /* All SHA1/SHA2 */ + T Ch(T)(T x, T y, T z) { return z ^ (x & (y ^ z)); } + T Maj(T)(T x, T y, T z) { return (x & y) | (z & (x ^ y)); } + + /* SHA-1 */ + uint Parity(uint x, uint y, uint z) { return x ^ y ^ z; } + + /* SHA-224, SHA-256 */ + uint BigSigma0(uint x) { return rotateRight(x, 2) ^ rotateRight(x, 13) ^ rotateRight(x, 22); } + uint BigSigma1(uint x) { return rotateRight(x, 6) ^ rotateRight(x, 11) ^ rotateRight(x, 25); } + uint SmSigma0(uint x) { return rotateRight(x, 7) ^ rotateRight(x, 18) ^ x >> 3; } + uint SmSigma1(uint x) { return rotateRight(x, 17) ^ rotateRight(x, 19) ^ x >> 10; } + + /* SHA-384, SHA-512, SHA-512/224, SHA-512/256 */ + ulong BigSigma0(ulong x) { return rotateRight(x, 28) ^ rotateRight(x, 34) ^ rotateRight(x, 39); } + ulong BigSigma1(ulong x) { return rotateRight(x, 14) ^ rotateRight(x, 18) ^ rotateRight(x, 41); } + ulong SmSigma0(ulong x) { return rotateRight(x, 1) ^ rotateRight(x, 8) ^ x >> 7; } + ulong SmSigma1(ulong x) { return rotateRight(x, 19) ^ rotateRight(x, 61) ^ x >> 6; } + } + + /* + * SHA1 basic transformation. Transforms state based on block. + */ + static void T_0_15(int i, const(ubyte[64])* input, ref uint[16] W, uint A, ref uint B, uint C, uint D, + uint E, ref uint T) pure nothrow @nogc + { + uint Wi = W[i] = bigEndianToNative(*cast(ubyte[4]*)&((*input)[i*4])); + T = Ch(B, C, D) + E + rotateLeft(A, 5) + Wi + 0x5a827999; + B = rotateLeft(B, 30); + } + + static void T_16_19(int i, ref uint[16] W, uint A, ref uint B, uint C, uint D, uint E, ref uint T) + pure nothrow @nogc + { + W[i&15] = rotateLeft(W[(i-3)&15] ^ W[(i-8)&15] ^ W[(i-14)&15] ^ W[(i-16)&15], 1); + T = Ch(B, C, D) + E + rotateLeft(A, 5) + W[i&15] + 0x5a827999; + B = rotateLeft(B, 30); + } + + static void T_20_39(int i, ref uint[16] W, uint A, ref uint B, uint C, uint D, uint E, + ref uint T) pure nothrow @nogc + { + W[i&15] = rotateLeft(W[(i-3)&15] ^ W[(i-8)&15] ^ W[(i-14)&15] ^ W[(i-16)&15], 1); + T = Parity(B, C, D) + E + rotateLeft(A, 5) + W[i&15] + 0x6ed9eba1; + B = rotateLeft(B, 30); + } + + static void T_40_59(int i, ref uint[16] W, uint A, ref uint B, uint C, uint D, uint E, + ref uint T) pure nothrow @nogc + { + W[i&15] = rotateLeft(W[(i-3)&15] ^ W[(i-8)&15] ^ W[(i-14)&15] ^ W[(i-16)&15], 1); + T = Maj(B, C, D) + E + rotateLeft(A, 5) + W[i&15] + 0x8f1bbcdc; + B = rotateLeft(B, 30); + } + + static void T_60_79(int i, ref uint[16] W, uint A, ref uint B, uint C, uint D, uint E, + ref uint T) pure nothrow @nogc + { + W[i&15] = rotateLeft(W[(i-3)&15] ^ W[(i-8)&15] ^ W[(i-14)&15] ^ W[(i-16)&15], 1); + T = Parity(B, C, D) + E + rotateLeft(A, 5) + W[i&15] + 0xca62c1d6; + B = rotateLeft(B, 30); + } + + private static void transformX86(uint[5]* state, const(ubyte[64])* block) pure nothrow @nogc + { + uint A, B, C, D, E, T; + uint[16] W = void; + + A = (*state)[0]; + B = (*state)[1]; + C = (*state)[2]; + D = (*state)[3]; + E = (*state)[4]; + + T_0_15 ( 0, block, W, A, B, C, D, E, T); + T_0_15 ( 1, block, W, T, A, B, C, D, E); + T_0_15 ( 2, block, W, E, T, A, B, C, D); + T_0_15 ( 3, block, W, D, E, T, A, B, C); + T_0_15 ( 4, block, W, C, D, E, T, A, B); + T_0_15 ( 5, block, W, B, C, D, E, T, A); + T_0_15 ( 6, block, W, A, B, C, D, E, T); + T_0_15 ( 7, block, W, T, A, B, C, D, E); + T_0_15 ( 8, block, W, E, T, A, B, C, D); + T_0_15 ( 9, block, W, D, E, T, A, B, C); + T_0_15 (10, block, W, C, D, E, T, A, B); + T_0_15 (11, block, W, B, C, D, E, T, A); + T_0_15 (12, block, W, A, B, C, D, E, T); + T_0_15 (13, block, W, T, A, B, C, D, E); + T_0_15 (14, block, W, E, T, A, B, C, D); + T_0_15 (15, block, W, D, E, T, A, B, C); + T_16_19(16, W, C, D, E, T, A, B); + T_16_19(17, W, B, C, D, E, T, A); + T_16_19(18, W, A, B, C, D, E, T); + T_16_19(19, W, T, A, B, C, D, E); + T_20_39(20, W, E, T, A, B, C, D); + T_20_39(21, W, D, E, T, A, B, C); + T_20_39(22, W, C, D, E, T, A, B); + T_20_39(23, W, B, C, D, E, T, A); + T_20_39(24, W, A, B, C, D, E, T); + T_20_39(25, W, T, A, B, C, D, E); + T_20_39(26, W, E, T, A, B, C, D); + T_20_39(27, W, D, E, T, A, B, C); + T_20_39(28, W, C, D, E, T, A, B); + T_20_39(29, W, B, C, D, E, T, A); + T_20_39(30, W, A, B, C, D, E, T); + T_20_39(31, W, T, A, B, C, D, E); + T_20_39(32, W, E, T, A, B, C, D); + T_20_39(33, W, D, E, T, A, B, C); + T_20_39(34, W, C, D, E, T, A, B); + T_20_39(35, W, B, C, D, E, T, A); + T_20_39(36, W, A, B, C, D, E, T); + T_20_39(37, W, T, A, B, C, D, E); + T_20_39(38, W, E, T, A, B, C, D); + T_20_39(39, W, D, E, T, A, B, C); + T_40_59(40, W, C, D, E, T, A, B); + T_40_59(41, W, B, C, D, E, T, A); + T_40_59(42, W, A, B, C, D, E, T); + T_40_59(43, W, T, A, B, C, D, E); + T_40_59(44, W, E, T, A, B, C, D); + T_40_59(45, W, D, E, T, A, B, C); + T_40_59(46, W, C, D, E, T, A, B); + T_40_59(47, W, B, C, D, E, T, A); + T_40_59(48, W, A, B, C, D, E, T); + T_40_59(49, W, T, A, B, C, D, E); + T_40_59(50, W, E, T, A, B, C, D); + T_40_59(51, W, D, E, T, A, B, C); + T_40_59(52, W, C, D, E, T, A, B); + T_40_59(53, W, B, C, D, E, T, A); + T_40_59(54, W, A, B, C, D, E, T); + T_40_59(55, W, T, A, B, C, D, E); + T_40_59(56, W, E, T, A, B, C, D); + T_40_59(57, W, D, E, T, A, B, C); + T_40_59(58, W, C, D, E, T, A, B); + T_40_59(59, W, B, C, D, E, T, A); + T_60_79(60, W, A, B, C, D, E, T); + T_60_79(61, W, T, A, B, C, D, E); + T_60_79(62, W, E, T, A, B, C, D); + T_60_79(63, W, D, E, T, A, B, C); + T_60_79(64, W, C, D, E, T, A, B); + T_60_79(65, W, B, C, D, E, T, A); + T_60_79(66, W, A, B, C, D, E, T); + T_60_79(67, W, T, A, B, C, D, E); + T_60_79(68, W, E, T, A, B, C, D); + T_60_79(69, W, D, E, T, A, B, C); + T_60_79(70, W, C, D, E, T, A, B); + T_60_79(71, W, B, C, D, E, T, A); + T_60_79(72, W, A, B, C, D, E, T); + T_60_79(73, W, T, A, B, C, D, E); + T_60_79(74, W, E, T, A, B, C, D); + T_60_79(75, W, D, E, T, A, B, C); + T_60_79(76, W, C, D, E, T, A, B); + T_60_79(77, W, B, C, D, E, T, A); + T_60_79(78, W, A, B, C, D, E, T); + T_60_79(79, W, T, A, B, C, D, E); + + (*state)[0] += E; + (*state)[1] += T; + (*state)[2] += A; + (*state)[3] += B; + (*state)[4] += C; + + /* Zeroize sensitive information. */ + W[] = 0; + } + + /* + * SHA2 basic transformation. Transforms state based on block. + */ + static void T_SHA2_0_15(Word)(int i, const(ubyte[blockSize/8])* input, ref Word[16] W, + Word A, Word B, Word C, ref Word D, Word E, Word F, Word G, ref Word H, Word K) + pure nothrow @nogc + { + Word Wi = W[i] = bigEndianToNative(*cast(ubyte[Word.sizeof]*)&((*input)[i*Word.sizeof])); + Word T1 = H + BigSigma1(E) + Ch(E, F, G) + K + Wi; + Word T2 = BigSigma0(A) + Maj(A, B, C); + D += T1; + H = T1 + T2; + } + + static void T_SHA2_16_79(Word)(int i, ref Word[16] W, + Word A, Word B, Word C, ref Word D, Word E, Word F, Word G, ref Word H, Word K) + pure nothrow @nogc + { + W[i&15] = SmSigma1(W[(i-2)&15]) + W[(i-7)&15] + SmSigma0(W[(i-15)&15]) + W[i&15]; + Word T1 = H + BigSigma1(E) + Ch(E, F, G) + K + W[i&15]; + Word T2 = BigSigma0(A) + Maj(A, B, C); + D += T1; + H = T1 + T2; + } + + private static void transformSHA2(Word)(Word[8]* state, const(ubyte[blockSize/8])* block) + pure nothrow @nogc + { + Word A, B, C, D, E, F, G, H; + Word[16] W = void; + + A = (*state)[0]; + B = (*state)[1]; + C = (*state)[2]; + D = (*state)[3]; + E = (*state)[4]; + F = (*state)[5]; + G = (*state)[6]; + H = (*state)[7]; + + T_SHA2_0_15!Word ( 0, block, W, A, B, C, D, E, F, G, H, constants[ 0]); + T_SHA2_0_15!Word ( 1, block, W, H, A, B, C, D, E, F, G, constants[ 1]); + T_SHA2_0_15!Word ( 2, block, W, G, H, A, B, C, D, E, F, constants[ 2]); + T_SHA2_0_15!Word ( 3, block, W, F, G, H, A, B, C, D, E, constants[ 3]); + T_SHA2_0_15!Word ( 4, block, W, E, F, G, H, A, B, C, D, constants[ 4]); + T_SHA2_0_15!Word ( 5, block, W, D, E, F, G, H, A, B, C, constants[ 5]); + T_SHA2_0_15!Word ( 6, block, W, C, D, E, F, G, H, A, B, constants[ 6]); + T_SHA2_0_15!Word ( 7, block, W, B, C, D, E, F, G, H, A, constants[ 7]); + T_SHA2_0_15!Word ( 8, block, W, A, B, C, D, E, F, G, H, constants[ 8]); + T_SHA2_0_15!Word ( 9, block, W, H, A, B, C, D, E, F, G, constants[ 9]); + T_SHA2_0_15!Word (10, block, W, G, H, A, B, C, D, E, F, constants[10]); + T_SHA2_0_15!Word (11, block, W, F, G, H, A, B, C, D, E, constants[11]); + T_SHA2_0_15!Word (12, block, W, E, F, G, H, A, B, C, D, constants[12]); + T_SHA2_0_15!Word (13, block, W, D, E, F, G, H, A, B, C, constants[13]); + T_SHA2_0_15!Word (14, block, W, C, D, E, F, G, H, A, B, constants[14]); + T_SHA2_0_15!Word (15, block, W, B, C, D, E, F, G, H, A, constants[15]); + T_SHA2_16_79!Word(16, W, A, B, C, D, E, F, G, H, constants[16]); + T_SHA2_16_79!Word(17, W, H, A, B, C, D, E, F, G, constants[17]); + T_SHA2_16_79!Word(18, W, G, H, A, B, C, D, E, F, constants[18]); + T_SHA2_16_79!Word(19, W, F, G, H, A, B, C, D, E, constants[19]); + T_SHA2_16_79!Word(20, W, E, F, G, H, A, B, C, D, constants[20]); + T_SHA2_16_79!Word(21, W, D, E, F, G, H, A, B, C, constants[21]); + T_SHA2_16_79!Word(22, W, C, D, E, F, G, H, A, B, constants[22]); + T_SHA2_16_79!Word(23, W, B, C, D, E, F, G, H, A, constants[23]); + T_SHA2_16_79!Word(24, W, A, B, C, D, E, F, G, H, constants[24]); + T_SHA2_16_79!Word(25, W, H, A, B, C, D, E, F, G, constants[25]); + T_SHA2_16_79!Word(26, W, G, H, A, B, C, D, E, F, constants[26]); + T_SHA2_16_79!Word(27, W, F, G, H, A, B, C, D, E, constants[27]); + T_SHA2_16_79!Word(28, W, E, F, G, H, A, B, C, D, constants[28]); + T_SHA2_16_79!Word(29, W, D, E, F, G, H, A, B, C, constants[29]); + T_SHA2_16_79!Word(30, W, C, D, E, F, G, H, A, B, constants[30]); + T_SHA2_16_79!Word(31, W, B, C, D, E, F, G, H, A, constants[31]); + T_SHA2_16_79!Word(32, W, A, B, C, D, E, F, G, H, constants[32]); + T_SHA2_16_79!Word(33, W, H, A, B, C, D, E, F, G, constants[33]); + T_SHA2_16_79!Word(34, W, G, H, A, B, C, D, E, F, constants[34]); + T_SHA2_16_79!Word(35, W, F, G, H, A, B, C, D, E, constants[35]); + T_SHA2_16_79!Word(36, W, E, F, G, H, A, B, C, D, constants[36]); + T_SHA2_16_79!Word(37, W, D, E, F, G, H, A, B, C, constants[37]); + T_SHA2_16_79!Word(38, W, C, D, E, F, G, H, A, B, constants[38]); + T_SHA2_16_79!Word(39, W, B, C, D, E, F, G, H, A, constants[39]); + T_SHA2_16_79!Word(40, W, A, B, C, D, E, F, G, H, constants[40]); + T_SHA2_16_79!Word(41, W, H, A, B, C, D, E, F, G, constants[41]); + T_SHA2_16_79!Word(42, W, G, H, A, B, C, D, E, F, constants[42]); + T_SHA2_16_79!Word(43, W, F, G, H, A, B, C, D, E, constants[43]); + T_SHA2_16_79!Word(44, W, E, F, G, H, A, B, C, D, constants[44]); + T_SHA2_16_79!Word(45, W, D, E, F, G, H, A, B, C, constants[45]); + T_SHA2_16_79!Word(46, W, C, D, E, F, G, H, A, B, constants[46]); + T_SHA2_16_79!Word(47, W, B, C, D, E, F, G, H, A, constants[47]); + T_SHA2_16_79!Word(48, W, A, B, C, D, E, F, G, H, constants[48]); + T_SHA2_16_79!Word(49, W, H, A, B, C, D, E, F, G, constants[49]); + T_SHA2_16_79!Word(50, W, G, H, A, B, C, D, E, F, constants[50]); + T_SHA2_16_79!Word(51, W, F, G, H, A, B, C, D, E, constants[51]); + T_SHA2_16_79!Word(52, W, E, F, G, H, A, B, C, D, constants[52]); + T_SHA2_16_79!Word(53, W, D, E, F, G, H, A, B, C, constants[53]); + T_SHA2_16_79!Word(54, W, C, D, E, F, G, H, A, B, constants[54]); + T_SHA2_16_79!Word(55, W, B, C, D, E, F, G, H, A, constants[55]); + T_SHA2_16_79!Word(56, W, A, B, C, D, E, F, G, H, constants[56]); + T_SHA2_16_79!Word(57, W, H, A, B, C, D, E, F, G, constants[57]); + T_SHA2_16_79!Word(58, W, G, H, A, B, C, D, E, F, constants[58]); + T_SHA2_16_79!Word(59, W, F, G, H, A, B, C, D, E, constants[59]); + T_SHA2_16_79!Word(60, W, E, F, G, H, A, B, C, D, constants[60]); + T_SHA2_16_79!Word(61, W, D, E, F, G, H, A, B, C, constants[61]); + T_SHA2_16_79!Word(62, W, C, D, E, F, G, H, A, B, constants[62]); + T_SHA2_16_79!Word(63, W, B, C, D, E, F, G, H, A, constants[63]); + + static if (is(Word == ulong)) + { + T_SHA2_16_79!Word(64, W, A, B, C, D, E, F, G, H, constants[64]); + T_SHA2_16_79!Word(65, W, H, A, B, C, D, E, F, G, constants[65]); + T_SHA2_16_79!Word(66, W, G, H, A, B, C, D, E, F, constants[66]); + T_SHA2_16_79!Word(67, W, F, G, H, A, B, C, D, E, constants[67]); + T_SHA2_16_79!Word(68, W, E, F, G, H, A, B, C, D, constants[68]); + T_SHA2_16_79!Word(69, W, D, E, F, G, H, A, B, C, constants[69]); + T_SHA2_16_79!Word(70, W, C, D, E, F, G, H, A, B, constants[70]); + T_SHA2_16_79!Word(71, W, B, C, D, E, F, G, H, A, constants[71]); + T_SHA2_16_79!Word(72, W, A, B, C, D, E, F, G, H, constants[72]); + T_SHA2_16_79!Word(73, W, H, A, B, C, D, E, F, G, constants[73]); + T_SHA2_16_79!Word(74, W, G, H, A, B, C, D, E, F, constants[74]); + T_SHA2_16_79!Word(75, W, F, G, H, A, B, C, D, E, constants[75]); + T_SHA2_16_79!Word(76, W, E, F, G, H, A, B, C, D, constants[76]); + T_SHA2_16_79!Word(77, W, D, E, F, G, H, A, B, C, constants[77]); + T_SHA2_16_79!Word(78, W, C, D, E, F, G, H, A, B, constants[78]); + T_SHA2_16_79!Word(79, W, B, C, D, E, F, G, H, A, constants[79]); + } + + (*state)[0] += A; + (*state)[1] += B; + (*state)[2] += C; + (*state)[3] += D; + (*state)[4] += E; + (*state)[5] += F; + (*state)[6] += G; + (*state)[7] += H; + + /* Zeroize sensitive information. */ + W[] = 0; + } + + public: + /** + * SHA initialization. Begins an SHA1/SHA2 operation. + * + * Note: + * For this SHA Digest implementation calling start after default construction + * is not necessary. Calling start is only necessary to reset the Digest. + * + * Generic code which deals with different Digest types should always call start though. + * + * Example: + * -------- + * SHA1 digest; + * //digest.start(); //Not necessary + * digest.put(0); + * -------- + */ + void start() @safe pure nothrow @nogc + { + this = typeof(this).init; + } + + /** + * Use this to feed the digest with data. + * Also implements the $(REF isOutputRange, std,range,primitives) + * interface for $(D ubyte) and $(D const(ubyte)[]). + */ + void put(scope const(ubyte)[] input...) @trusted pure nothrow @nogc + { + enum blockSizeInBytes = blockSize/8; + uint i, index, partLen; + auto inputLen = input.length; + + /* Compute number of bytes mod block size (64 or 128 bytes) */ + index = (cast(uint) count[0] >> 3) & (blockSizeInBytes - 1); + + /* Update number of bits */ + static if (blockSize == 512) + count[0] += inputLen * 8; + else static if (blockSize == 1024) + { + /* ugly hack to work around lack of ucent */ + auto oldCount0 = count[0]; + count[0] += inputLen * 8; + if (count[0] < oldCount0) + count[1]++; + } + else + static assert(0); + + partLen = blockSizeInBytes - index; + + /* Transform as many times as possible. */ + if (inputLen >= partLen) + { + (&buffer[index])[0 .. partLen] = input.ptr[0 .. partLen]; + transform (&state, &buffer); + + for (i = partLen; i + blockSizeInBytes-1 < inputLen; i += blockSizeInBytes) + transform(&state, cast(ubyte[blockSizeInBytes]*)(input.ptr + i)); + + index = 0; + } + else + i = 0; + + /* Buffer remaining input */ + if (inputLen - i) + (&buffer[index])[0 .. inputLen-i] = (&input[i])[0 .. inputLen-i]; + } + + @safe unittest + { + typeof(this) dig; + dig.put(cast(ubyte) 0); //single ubyte + dig.put(cast(ubyte) 0, cast(ubyte) 0); //variadic + ubyte[10] buf; + dig.put(buf); //buffer + } + + + /** + * Returns the finished SHA hash. This also calls $(LREF start) to + * reset the internal state. + */ + ubyte[digestSize/8] finish() @trusted pure nothrow @nogc + { + static if (blockSize == 512) + { + ubyte[32] data = void; + uint index, padLen; + + /* Save number of bits */ + ubyte[8] bits = nativeToBigEndian(count[0]); + + /* Pad out to 56 mod 64. */ + index = (cast(uint) count[0] >> 3) & (64 - 1); + padLen = (index < 56) ? (56 - index) : (120 - index); + put(padding[0 .. padLen]); + + /* Append length (before padding) */ + put(bits); + + /* Store state in digest */ + for (auto i = 0; i < ((digestSize == 160)? 5 : 8); i++) + data[i*4..(i+1)*4] = nativeToBigEndian(state[i])[]; + + /* Zeroize sensitive information. */ + start(); + return data[0 .. digestSize/8]; + } + else static if (blockSize == 1024) + { + ubyte[64] data = void; + uint index, padLen; + + /* Save number of bits */ + ubyte[16] bits; + bits[ 0 .. 8] = nativeToBigEndian(count[1]); + bits[8 .. 16] = nativeToBigEndian(count[0]); + + /* Pad out to 112 mod 128. */ + index = (cast(uint) count[0] >> 3) & (128 - 1); + padLen = (index < 112) ? (112 - index) : (240 - index); + put(padding[0 .. padLen]); + + /* Append length (before padding) */ + put(bits); + + /* Store state in digest */ + for (auto i = 0; i < 8; i++) + data[i*8..(i+1)*8] = nativeToBigEndian(state[i])[]; + + /* Zeroize sensitive information. */ + start(); + return data[0 .. digestSize/8]; + } + else + static assert(0); + } + /// + @safe unittest + { + //Simple example + SHA1 hash; + hash.start(); + hash.put(cast(ubyte) 0); + ubyte[20] result = hash.finish(); + } +} + +alias SHA1 = SHA!(512, 160); /// SHA alias for SHA-1, hash is ubyte[20] +alias SHA224 = SHA!(512, 224); /// SHA alias for SHA-224, hash is ubyte[28] +alias SHA256 = SHA!(512, 256); /// SHA alias for SHA-256, hash is ubyte[32] +alias SHA384 = SHA!(1024, 384); /// SHA alias for SHA-384, hash is ubyte[48] +alias SHA512 = SHA!(1024, 512); /// SHA alias for SHA-512, hash is ubyte[64] +alias SHA512_224 = SHA!(1024, 224); /// SHA alias for SHA-512/224, hash is ubyte[28] +alias SHA512_256 = SHA!(1024, 256); /// SHA alias for SHA-512/256, hash is ubyte[32] + +/// +@safe unittest +{ + //Simple example, hashing a string using sha1Of helper function + ubyte[20] hash = sha1Of("abc"); + //Let's get a hash string + assert(toHexString(hash) == "A9993E364706816ABA3E25717850C26C9CD0D89D"); + + //The same, but using SHA-224 + ubyte[28] hash224 = sha224Of("abc"); + assert(toHexString(hash224) == "23097D223405D8228642A477BDA255B32AADBCE4BDA0B3F7E36C9DA7"); +} + +/// +@safe unittest +{ + //Using the basic API + SHA1 hash; + hash.start(); + ubyte[1024] data; + //Initialize data here... + hash.put(data); + ubyte[20] result = hash.finish(); +} + +/// +@safe unittest +{ + //Let's use the template features: + //Note: When passing a SHA1 to a function, it must be passed by reference! + void doSomething(T)(ref T hash) + if (isDigest!T) + { + hash.put(cast(ubyte) 0); + } + SHA1 sha; + sha.start(); + doSomething(sha); + assert(toHexString(sha.finish()) == "5BA93C9DB0CFF93F52B521D7420E43F6EDA2784F"); +} + +@safe unittest +{ + assert(isDigest!SHA1); + assert(isDigest!SHA224); + assert(isDigest!SHA256); + assert(isDigest!SHA384); + assert(isDigest!SHA512); + assert(isDigest!SHA512_224); + assert(isDigest!SHA512_256); +} + +@system unittest +{ + import std.conv : hexString; + import std.range; + + ubyte[20] digest; + ubyte[28] digest224; + ubyte[32] digest256; + ubyte[48] digest384; + ubyte[64] digest512; + ubyte[28] digest512_224; + ubyte[32] digest512_256; + + SHA1 sha; + sha.put(cast(ubyte[])"abcdef"); + sha.start(); + sha.put(cast(ubyte[])""); + assert(sha.finish() == cast(ubyte[]) x"da39a3ee5e6b4b0d3255bfef95601890afd80709"); + + SHA224 sha224; + sha224.put(cast(ubyte[])"abcdef"); + sha224.start(); + sha224.put(cast(ubyte[])""); + assert(sha224.finish() == cast(ubyte[]) x"d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f"); + + SHA256 sha256; + sha256.put(cast(ubyte[])"abcdef"); + sha256.start(); + sha256.put(cast(ubyte[])""); + assert(sha256.finish() == cast(ubyte[]) x"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"); + + SHA384 sha384; + sha384.put(cast(ubyte[])"abcdef"); + sha384.start(); + sha384.put(cast(ubyte[])""); + assert(sha384.finish() == cast(ubyte[]) hexString!("38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c" + ~"0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b")); + + SHA512 sha512; + sha512.put(cast(ubyte[])"abcdef"); + sha512.start(); + sha512.put(cast(ubyte[])""); + assert(sha512.finish() == cast(ubyte[]) hexString!("cf83e1357eefb8bdf1542850d66d8007d620e4050b571" + ~"5dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e")); + + SHA512_224 sha512_224; + sha512_224.put(cast(ubyte[])"abcdef"); + sha512_224.start(); + sha512_224.put(cast(ubyte[])""); + assert(sha512_224.finish() == cast(ubyte[]) x"6ed0dd02806fa89e25de060c19d3ac86cabb87d6a0ddd05c333b84f4"); + + SHA512_256 sha512_256; + sha512_256.put(cast(ubyte[])"abcdef"); + sha512_256.start(); + sha512_256.put(cast(ubyte[])""); + assert(sha512_256.finish() == cast(ubyte[]) x"c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a"); + + digest = sha1Of (""); + digest224 = sha224Of (""); + digest256 = sha256Of (""); + digest384 = sha384Of (""); + digest512 = sha512Of (""); + digest512_224 = sha512_224Of(""); + digest512_256 = sha512_256Of(""); + assert(digest == cast(ubyte[]) x"da39a3ee5e6b4b0d3255bfef95601890afd80709"); + assert(digest224 == cast(ubyte[]) x"d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f"); + assert(digest256 == cast(ubyte[]) x"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"); + assert(digest384 == cast(ubyte[]) hexString!("38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c" + ~"0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b")); + assert(digest512 == cast(ubyte[]) hexString!("cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83" + ~"f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e")); + assert(digest512_224 == cast(ubyte[]) x"6ed0dd02806fa89e25de060c19d3ac86cabb87d6a0ddd05c333b84f4"); + assert(digest512_256 == cast(ubyte[]) x"c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a"); + + digest = sha1Of ("a"); + digest224 = sha224Of ("a"); + digest256 = sha256Of ("a"); + digest384 = sha384Of ("a"); + digest512 = sha512Of ("a"); + digest512_224 = sha512_224Of("a"); + digest512_256 = sha512_256Of("a"); + assert(digest == cast(ubyte[]) x"86f7e437faa5a7fce15d1ddcb9eaeaea377667b8"); + assert(digest224 == cast(ubyte[]) x"abd37534c7d9a2efb9465de931cd7055ffdb8879563ae98078d6d6d5"); + assert(digest256 == cast(ubyte[]) x"ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb"); + assert(digest384 == cast(ubyte[]) hexString!("54a59b9f22b0b80880d8427e548b7c23abd873486e1f035dce9" + ~"cd697e85175033caa88e6d57bc35efae0b5afd3145f31")); + assert(digest512 == cast(ubyte[]) hexString!("1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05ab" + ~"c54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75")); + assert(digest512_224 == cast(ubyte[]) x"d5cdb9ccc769a5121d4175f2bfdd13d6310e0d3d361ea75d82108327"); + assert(digest512_256 == cast(ubyte[]) x"455e518824bc0601f9fb858ff5c37d417d67c2f8e0df2babe4808858aea830f8"); + + digest = sha1Of ("abc"); + digest224 = sha224Of ("abc"); + digest256 = sha256Of ("abc"); + digest384 = sha384Of ("abc"); + digest512 = sha512Of ("abc"); + digest512_224 = sha512_224Of("abc"); + digest512_256 = sha512_256Of("abc"); + assert(digest == cast(ubyte[]) x"a9993e364706816aba3e25717850c26c9cd0d89d"); + assert(digest224 == cast(ubyte[]) x"23097d223405d8228642a477bda255b32aadbce4bda0b3f7e36c9da7"); + assert(digest256 == cast(ubyte[]) x"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"); + assert(digest384 == cast(ubyte[]) hexString!("cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a" + ~"8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7")); + assert(digest512 == cast(ubyte[]) hexString!("ddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9" + ~"eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f")); + assert(digest512_224 == cast(ubyte[]) x"4634270f707b6a54daae7530460842e20e37ed265ceee9a43e8924aa"); + assert(digest512_256 == cast(ubyte[]) x"53048e2681941ef99b2e29b76b4c7dabe4c2d0c634fc6d46e0e2f13107e7af23"); + + digest = sha1Of ("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + digest224 = sha224Of ("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + digest256 = sha256Of ("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + digest384 = sha384Of ("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + digest512 = sha512Of ("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + digest512_224 = sha512_224Of("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + digest512_256 = sha512_256Of("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"); + assert(digest == cast(ubyte[]) x"84983e441c3bd26ebaae4aa1f95129e5e54670f1"); + assert(digest224 == cast(ubyte[]) x"75388b16512776cc5dba5da1fd890150b0c6455cb4f58b1952522525"); + assert(digest256 == cast(ubyte[]) x"248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1"); + assert(digest384 == cast(ubyte[]) hexString!("3391fdddfc8dc7393707a65b1b4709397cf8b1d162af05abfe" + ~"8f450de5f36bc6b0455a8520bc4e6f5fe95b1fe3c8452b")); + assert(digest512 == cast(ubyte[]) hexString!("204a8fc6dda82f0a0ced7beb8e08a41657c16ef468b228a827" + ~"9be331a703c33596fd15c13b1b07f9aa1d3bea57789ca031ad85c7a71dd70354ec631238ca3445")); + assert(digest512_224 == cast(ubyte[]) x"e5302d6d54bb242275d1e7622d68df6eb02dedd13f564c13dbda2174"); + assert(digest512_256 == cast(ubyte[]) x"bde8e1f9f19bb9fd3406c90ec6bc47bd36d8ada9f11880dbc8a22a7078b6a461"); + + digest = sha1Of ("message digest"); + digest224 = sha224Of ("message digest"); + digest256 = sha256Of ("message digest"); + digest384 = sha384Of ("message digest"); + digest512 = sha512Of ("message digest"); + digest512_224 = sha512_224Of("message digest"); + digest512_256 = sha512_256Of("message digest"); + assert(digest == cast(ubyte[]) x"c12252ceda8be8994d5fa0290a47231c1d16aae3"); + assert(digest224 == cast(ubyte[]) x"2cb21c83ae2f004de7e81c3c7019cbcb65b71ab656b22d6d0c39b8eb"); + assert(digest256 == cast(ubyte[]) x"f7846f55cf23e14eebeab5b4e1550cad5b509e3348fbc4efa3a1413d393cb650"); + assert(digest384 == cast(ubyte[]) hexString!("473ed35167ec1f5d8e550368a3db39be54639f828868e9454c" + ~"239fc8b52e3c61dbd0d8b4de1390c256dcbb5d5fd99cd5")); + assert(digest512 == cast(ubyte[]) hexString!("107dbf389d9e9f71a3a95f6c055b9251bc5268c2be16d6c134" + ~"92ea45b0199f3309e16455ab1e96118e8a905d5597b72038ddb372a89826046de66687bb420e7c")); + assert(digest512_224 == cast(ubyte[]) x"ad1a4db188fe57064f4f24609d2a83cd0afb9b398eb2fcaeaae2c564"); + assert(digest512_256 == cast(ubyte[]) x"0cf471fd17ed69d990daf3433c89b16d63dec1bb9cb42a6094604ee5d7b4e9fb"); + + digest = sha1Of ("abcdefghijklmnopqrstuvwxyz"); + digest224 = sha224Of ("abcdefghijklmnopqrstuvwxyz"); + digest256 = sha256Of ("abcdefghijklmnopqrstuvwxyz"); + digest384 = sha384Of ("abcdefghijklmnopqrstuvwxyz"); + digest512 = sha512Of ("abcdefghijklmnopqrstuvwxyz"); + digest512_224 = sha512_224Of("abcdefghijklmnopqrstuvwxyz"); + digest512_256 = sha512_256Of("abcdefghijklmnopqrstuvwxyz"); + assert(digest == cast(ubyte[]) x"32d10c7b8cf96570ca04ce37f2a19d84240d3a89"); + assert(digest224 == cast(ubyte[]) x"45a5f72c39c5cff2522eb3429799e49e5f44b356ef926bcf390dccc2"); + assert(digest256 == cast(ubyte[]) x"71c480df93d6ae2f1efad1447c66c9525e316218cf51fc8d9ed832f2daf18b73"); + assert(digest384 == cast(ubyte[]) hexString!("feb67349df3db6f5924815d6c3dc133f091809213731fe5c7b5" + ~"f4999e463479ff2877f5f2936fa63bb43784b12f3ebb4")); + assert(digest512 == cast(ubyte[]) hexString!("4dbff86cc2ca1bae1e16468a05cb9881c97f1753bce3619034" + ~"898faa1aabe429955a1bf8ec483d7421fe3c1646613a59ed5441fb0f321389f77f48a879c7b1f1")); + assert(digest512_224 == cast(ubyte[]) x"ff83148aa07ec30655c1b40aff86141c0215fe2a54f767d3f38743d8"); + assert(digest512_256 == cast(ubyte[]) x"fc3189443f9c268f626aea08a756abe7b726b05f701cb08222312ccfd6710a26"); + + digest = sha1Of ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + digest224 = sha224Of ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + digest256 = sha256Of ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + digest384 = sha384Of ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + digest512 = sha512Of ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + digest512_224 = sha512_224Of("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + digest512_256 = sha512_256Of("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"); + assert(digest == cast(ubyte[]) x"761c457bf73b14d27e9e9265c46f4b4dda11f940"); + assert(digest224 == cast(ubyte[]) x"bff72b4fcb7d75e5632900ac5f90d219e05e97a7bde72e740db393d9"); + assert(digest256 == cast(ubyte[]) x"db4bfcbd4da0cd85a60c3c37d3fbd8805c77f15fc6b1fdfe614ee0a7c8fdb4c0"); + assert(digest384 == cast(ubyte[]) hexString!("1761336e3f7cbfe51deb137f026f89e01a448e3b1fafa64039" + ~"c1464ee8732f11a5341a6f41e0c202294736ed64db1a84")); + assert(digest512 == cast(ubyte[]) hexString!("1e07be23c26a86ea37ea810c8ec7809352515a970e9253c26f" + ~"536cfc7a9996c45c8370583e0a78fa4a90041d71a4ceab7423f19c71b9d5a3e01249f0bebd5894")); + assert(digest512_224 == cast(ubyte[]) x"a8b4b9174b99ffc67d6f49be9981587b96441051e16e6dd036b140d3"); + assert(digest512_256 == cast(ubyte[]) x"cdf1cc0effe26ecc0c13758f7b4a48e000615df241284185c39eb05d355bb9c8"); + + digest = sha1Of ("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + digest224 = sha224Of ("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + digest256 = sha256Of ("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + digest384 = sha384Of ("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + digest512 = sha512Of ("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + digest512_224 = sha512_224Of("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + digest512_256 = sha512_256Of("1234567890123456789012345678901234567890"~ + "1234567890123456789012345678901234567890"); + assert(digest == cast(ubyte[]) x"50abf5706a150990a08b2c5ea40fa0e585554732"); + assert(digest224 == cast(ubyte[]) x"b50aecbe4e9bb0b57bc5f3ae760a8e01db24f203fb3cdcd13148046e"); + assert(digest256 == cast(ubyte[]) x"f371bc4a311f2b009eef952dd83ca80e2b60026c8e935592d0f9c308453c813e"); + assert(digest384 == cast(ubyte[]) hexString!("b12932b0627d1c060942f5447764155655bd4da0c9afa6dd9b" + ~"9ef53129af1b8fb0195996d2de9ca0df9d821ffee67026")); + assert(digest512 == cast(ubyte[]) hexString!("72ec1ef1124a45b047e8b7c75a932195135bb61de24ec0d191" + ~"4042246e0aec3a2354e093d76f3048b456764346900cb130d2a4fd5dd16abb5e30bcb850dee843")); + assert(digest512_224 == cast(ubyte[]) x"ae988faaa47e401a45f704d1272d99702458fea2ddc6582827556dd2"); + assert(digest512_256 == cast(ubyte[]) x"2c9fdbc0c90bdd87612ee8455474f9044850241dc105b1e8b94b8ddf5fac9148"); + + ubyte[] onemilliona = new ubyte[1000000]; + onemilliona[] = 'a'; + digest = sha1Of(onemilliona); + digest224 = sha224Of(onemilliona); + digest256 = sha256Of(onemilliona); + digest384 = sha384Of(onemilliona); + digest512 = sha512Of(onemilliona); + digest512_224 = sha512_224Of(onemilliona); + digest512_256 = sha512_256Of(onemilliona); + assert(digest == cast(ubyte[]) x"34aa973cd4c4daa4f61eeb2bdbad27316534016f"); + assert(digest224 == cast(ubyte[]) x"20794655980c91d8bbb4c1ea97618a4bf03f42581948b2ee4ee7ad67"); + assert(digest256 == cast(ubyte[]) x"cdc76e5c9914fb9281a1c7e284d73e67f1809a48a497200e046d39ccc7112cd0"); + assert(digest384 == cast(ubyte[]) hexString!("9d0e1809716474cb086e834e310a4a1ced149e9c00f2485279" + ~"72cec5704c2a5b07b8b3dc38ecc4ebae97ddd87f3d8985")); + assert(digest512 == cast(ubyte[]) hexString!("e718483d0ce769644e2e42c7bc15b4638e1f98b13b20442856" + ~"32a803afa973ebde0ff244877ea60a4cb0432ce577c31beb009c5c2c49aa2e4eadb217ad8cc09b")); + assert(digest512_224 == cast(ubyte[]) x"37ab331d76f0d36de422bd0edeb22a28accd487b7a8453ae965dd287"); + assert(digest512_256 == cast(ubyte[]) x"9a59a052930187a97038cae692f30708aa6491923ef5194394dc68d56c74fb21"); + + auto oneMillionRange = repeat!ubyte(cast(ubyte)'a', 1000000); + digest = sha1Of(oneMillionRange); + digest224 = sha224Of(oneMillionRange); + digest256 = sha256Of(oneMillionRange); + digest384 = sha384Of(oneMillionRange); + digest512 = sha512Of(oneMillionRange); + digest512_224 = sha512_224Of(oneMillionRange); + digest512_256 = sha512_256Of(oneMillionRange); + assert(digest == cast(ubyte[]) x"34aa973cd4c4daa4f61eeb2bdbad27316534016f"); + assert(digest224 == cast(ubyte[]) x"20794655980c91d8bbb4c1ea97618a4bf03f42581948b2ee4ee7ad67"); + assert(digest256 == cast(ubyte[]) x"cdc76e5c9914fb9281a1c7e284d73e67f1809a48a497200e046d39ccc7112cd0"); + assert(digest384 == cast(ubyte[]) hexString!("9d0e1809716474cb086e834e310a4a1ced149e9c00f2485279" + ~"72cec5704c2a5b07b8b3dc38ecc4ebae97ddd87f3d8985")); + assert(digest512 == cast(ubyte[]) hexString!("e718483d0ce769644e2e42c7bc15b4638e1f98b13b20442856" + ~"32a803afa973ebde0ff244877ea60a4cb0432ce577c31beb009c5c2c49aa2e4eadb217ad8cc09b")); + assert(digest512_224 == cast(ubyte[]) x"37ab331d76f0d36de422bd0edeb22a28accd487b7a8453ae965dd287"); + assert(digest512_256 == cast(ubyte[]) x"9a59a052930187a97038cae692f30708aa6491923ef5194394dc68d56c74fb21"); + + assert(toHexString(cast(ubyte[20]) x"a9993e364706816aba3e25717850c26c9cd0d89d") + == "A9993E364706816ABA3E25717850C26C9CD0D89D"); +} + +/** + * These are convenience aliases for $(REF digest, std,digest) using the + * SHA implementation. + */ +//simple alias doesn't work here, hope this gets inlined... +auto sha1Of(T...)(T data) +{ + return digest!(SHA1, T)(data); +} +///ditto +auto sha224Of(T...)(T data) +{ + return digest!(SHA224, T)(data); +} +///ditto +auto sha256Of(T...)(T data) +{ + return digest!(SHA256, T)(data); +} +///ditto +auto sha384Of(T...)(T data) +{ + return digest!(SHA384, T)(data); +} +///ditto +auto sha512Of(T...)(T data) +{ + return digest!(SHA512, T)(data); +} +///ditto +auto sha512_224Of(T...)(T data) +{ + return digest!(SHA512_224, T)(data); +} +///ditto +auto sha512_256Of(T...)(T data) +{ + return digest!(SHA512_256, T)(data); +} + +/// +@safe unittest +{ + ubyte[20] hash = sha1Of("abc"); + assert(hash == digest!SHA1("abc")); + + ubyte[28] hash224 = sha224Of("abc"); + assert(hash224 == digest!SHA224("abc")); + + ubyte[32] hash256 = sha256Of("abc"); + assert(hash256 == digest!SHA256("abc")); + + ubyte[48] hash384 = sha384Of("abc"); + assert(hash384 == digest!SHA384("abc")); + + ubyte[64] hash512 = sha512Of("abc"); + assert(hash512 == digest!SHA512("abc")); + + ubyte[28] hash512_224 = sha512_224Of("abc"); + assert(hash512_224 == digest!SHA512_224("abc")); + + ubyte[32] hash512_256 = sha512_256Of("abc"); + assert(hash512_256 == digest!SHA512_256("abc")); +} + +@safe unittest +{ + string a = "Mary has ", b = "a little lamb"; + int[] c = [ 1, 2, 3, 4, 5 ]; + string d = toHexString(sha1Of(a, b, c)); + version (LittleEndian) + assert(d == "CDBB611D00AC2387B642D3D7BDF4C3B342237110", d); + else + assert(d == "A0F1196C7A379C09390476D9CA4AA11B71FD11C8", d); +} + +/** + * OOP API SHA1 and SHA2 implementations. + * See $(D std.digest) for differences between template and OOP API. + * + * This is an alias for $(D $(REF WrapperDigest, std,digest)!SHA1), see + * there for more information. + */ +alias SHA1Digest = WrapperDigest!SHA1; +alias SHA224Digest = WrapperDigest!SHA224; ///ditto +alias SHA256Digest = WrapperDigest!SHA256; ///ditto +alias SHA384Digest = WrapperDigest!SHA384; ///ditto +alias SHA512Digest = WrapperDigest!SHA512; ///ditto +alias SHA512_224Digest = WrapperDigest!SHA512_224; ///ditto +alias SHA512_256Digest = WrapperDigest!SHA512_256; ///ditto + +/// +@safe unittest +{ + //Simple example, hashing a string using Digest.digest helper function + auto sha = new SHA1Digest(); + ubyte[] hash = sha.digest("abc"); + //Let's get a hash string + assert(toHexString(hash) == "A9993E364706816ABA3E25717850C26C9CD0D89D"); + + //The same, but using SHA-224 + auto sha224 = new SHA224Digest(); + ubyte[] hash224 = sha224.digest("abc"); + //Let's get a hash string + assert(toHexString(hash224) == "23097D223405D8228642A477BDA255B32AADBCE4BDA0B3F7E36C9DA7"); +} + +/// +@system unittest +{ + //Let's use the OOP features: + void test(Digest dig) + { + dig.put(cast(ubyte) 0); + } + auto sha = new SHA1Digest(); + test(sha); + + //Let's use a custom buffer: + ubyte[20] buf; + ubyte[] result = sha.finish(buf[]); + assert(toHexString(result) == "5BA93C9DB0CFF93F52B521D7420E43F6EDA2784F"); +} + +@system unittest +{ + auto sha = new SHA1Digest(); + + sha.put(cast(ubyte[])"abcdef"); + sha.reset(); + sha.put(cast(ubyte[])""); + assert(sha.finish() == cast(ubyte[]) x"da39a3ee5e6b4b0d3255bfef95601890afd80709"); + + sha.put(cast(ubyte[])"abcdefghijklmnopqrstuvwxyz"); + ubyte[22] result; + auto result2 = sha.finish(result[]); + assert(result[0 .. 20] == result2 && result2 == cast(ubyte[]) x"32d10c7b8cf96570ca04ce37f2a19d84240d3a89"); + + debug + assertThrown!Error(sha.finish(result[0 .. 15])); + + assert(sha.length == 20); + + assert(sha.digest("") == cast(ubyte[]) x"da39a3ee5e6b4b0d3255bfef95601890afd80709"); + + assert(sha.digest("a") == cast(ubyte[]) x"86f7e437faa5a7fce15d1ddcb9eaeaea377667b8"); + + assert(sha.digest("abc") == cast(ubyte[]) x"a9993e364706816aba3e25717850c26c9cd0d89d"); + + assert(sha.digest("abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq") + == cast(ubyte[]) x"84983e441c3bd26ebaae4aa1f95129e5e54670f1"); + + assert(sha.digest("message digest") == cast(ubyte[]) x"c12252ceda8be8994d5fa0290a47231c1d16aae3"); + + assert(sha.digest("abcdefghijklmnopqrstuvwxyz") + == cast(ubyte[]) x"32d10c7b8cf96570ca04ce37f2a19d84240d3a89"); + + assert(sha.digest("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") + == cast(ubyte[]) x"761c457bf73b14d27e9e9265c46f4b4dda11f940"); + + assert(sha.digest("1234567890123456789012345678901234567890", + "1234567890123456789012345678901234567890") + == cast(ubyte[]) x"50abf5706a150990a08b2c5ea40fa0e585554732"); + + ubyte[] onemilliona = new ubyte[1000000]; + onemilliona[] = 'a'; + assert(sha.digest(onemilliona) == cast(ubyte[]) x"34aa973cd4c4daa4f61eeb2bdbad27316534016f"); +} |