- Aug 17, 2017
-
-
Pan Bian authored
The return value of dma_map_single() should be checked by dma_mapping_error(). However, in function ccp_init_dm_workarea(), its return value is checked against NULL, which could result in failures. Signed-off-by:
Pan Bian <bianpan2016@163.com> Acked-by:
Gary R Hook <gary.hook@amd.com> Signed-off-by:
Herbert Xu <herbert@gondor.apana.org.au>
-
Stefan Agner authored
Use just @ to denote comments which works with gcc and clang. Otherwise clang reports an escape sequence error: error: invalid % escape in inline assembly string Use %0-%3 as operand references, this avoids: error: invalid operand in inline asm: 'umull ${1:r}, ${0:r}, ${2:r}, ${3:r}' Also remove superfluous casts on output operands to avoid warnings such as: warning: invalid use of a cast in an inline asm context requiring an l-value Signed-off-by:Stefan Agner <stefan@agner.ch> Acked-by:
Arnd Bergmann <arnd@arndb.de> Signed-off-by:
-
Mogens Lauridsen authored
This driver previously used a spinlock. The spinlock is not used any more, but the spinlock variable was still there and also being initialized. Signed-off-by:
Mogens Lauridsen <mlauridsen171@gmail.com> Signed-off-by:
-
Mogens Lauridsen authored
The direction used in dma_unmap_sg in aes calc is wrong. This result in the cache not being invalidated correct when aes calculation is done and result has been dma'ed to memory. This is seen as sporadic wrong result from aes calc. Signed-off-by:
-
- Aug 09, 2017
-
-
Stephan Mueller authored
Consolidate following data structures: skcipher_async_req, aead_async_req -> af_alg_async_req skcipher_rsgl, aead_rsql -> af_alg_rsgl skcipher_tsgl, aead_tsql -> af_alg_tsgl skcipher_ctx, aead_ctx -> af_alg_ctx Consolidate following functions: skcipher_sndbuf, aead_sndbuf -> af_alg_sndbuf skcipher_writable, aead_writable -> af_alg_writable skcipher_rcvbuf, aead_rcvbuf -> af_alg_rcvbuf skcipher_readable, aead_readable -> af_alg_readable aead_alloc_tsgl, skcipher_alloc_tsgl -> af_alg_alloc_tsgl aead_count_tsgl, skcipher_count_tsgl -> af_alg_count_tsgl aead_pull_tsgl, skcipher_pull_tsgl -> af_alg_pull_tsgl aead_free_areq_sgls, skcipher_free_areq_sgls -> af_alg_free_areq_sgls aead_wait_for_wmem, skcipher_wait_for_wmem -> af_alg_wait_for_wmem aead_wmem_wakeup, skcipher_wmem_wakeup -> af_alg_wmem_wakeup aead_wait_for_data, skcipher_wait_for_data -> af_alg_wait_for_data aead_data_wakeup, skcipher_data_wakeup -> af_alg_data_wakeup aead_sendmsg, skcipher_sendmsg ...
-
Fabio Estevam authored
Most of the dentry members from structure caam_drv_private are never used at all, so it is safe to remove them. Since debugfs_remove_recursive() is called, we don't need the file entries. Signed-off-by:
Fabio Estevam <festevam@gmail.com> Acked-by:
Horia Geantă <horia.geanta@nxp.com> Signed-off-by:
-
Arnd Bergmann authored
Without the base RSA code, we run into a link error: ERROR: "rsa_parse_pub_key" [drivers/crypto/ccp/ccp-crypto.ko] undefined! ERROR: "rsa_parse_priv_key" [drivers/crypto/ccp/ccp-crypto.ko] undefined! Like the other drivers implementing RSA in hardware, this can be avoided by always enabling the base support when we build CCP. Fixes: ceeec0af ("crypto: ccp - Add support for RSA on the CCP") Signed-off-by:
-
Arnd Bergmann authored
The added support for version 5 CCPs introduced a false-positive warning in the RSA implementation: drivers/crypto/ccp/ccp-ops.c: In function 'ccp_run_rsa_cmd': drivers/crypto/ccp/ccp-ops.c:1856:3: error: 'sb_count' may be used uninitialized in this function [-Werror=maybe-uninitialized] This changes the code in a way that should make it easier for the compiler to track the state of the sb_count variable, and avoid the warning. Fixes: 6ba46c7d ("crypto: ccp - Fix base RSA function for version 5 CCPs") Signed-off-by:
-
Arnd Bergmann authored
When UBSAN is enabled, we get a very large stack frame for __serpent_setkey, when the register allocator ends up using more registers than it has, and has to spill temporary values to the stack. The code was originally optimized for in-order x86-32 CPU implementations using older compilers, but it now runs into a highly suboptimal case on all CPU architectures, as seen by this warning: crypto/serpent_generic.c: In function '__serpent_setkey': crypto/serpent_generic.c:436:1: error: the frame size of 2720 bytes is larger than 2048 bytes [-Werror=frame-larger-than=] Disabling -fsanitize=alignment would avoid that warning, presumably the option turns off a optimization step that is required for getting the register allocation right, but there is no easy way to do that on gcc-7 (gcc-8 introduces a function attribute for this). I tried to figure out a way to modify the source code instead, and noticed that the two stages of the setkey() function (keyiter ...
-
Stephan Mueller authored
Use the NULL cipher to copy the AAD and PT/CT from the TX SGL to the RX SGL. This allows an in-place crypto operation on the RX SGL for encryption, because the TX data is always smaller or equal to the RX data (the RX data will hold the tag). For decryption, a per-request TX SGL is created which will only hold the tag value. As the RX SGL will have no space for the tag value and an in-place operation will not write the tag buffer, the TX SGL with the tag value is chained to the RX SGL. This now allows an in-place crypto operation. For example: * without the patch: kcapi -x 2 -e -c "gcm(aes)" -p 89154d0d4129d322e4487bafaa4f6b46 -k c0ece3e63198af382b5603331cc23fa8 -i 7e489b83622e7228314d878d -a afcd7202d621e06ca53b70c2bdff7fb2 -l 16 -u -s 00000000000000000000000000000000f4a3eacfbdadd3b1a17117b1d67ffc1f1e21efbbc6d83724a8c296e3bb8cda0c * with the patch: kcapi -x 2 -e -c "gcm(aes)" -p 89154d0d4129d322e4487bafaa4f6b46 -k c0ece3e63198af382b5603331cc23fa8 -i 7e489...
-
Stephan Mueller authored
If no data has been processed during recvmsg, return the error code. This covers all errors received during non-AIO operations. If any error occurs during a synchronous operation in addition to -EIOCBQUEUED or -EBADMSG (like -ENOMEM), it should be relayed to the caller. Signed-off-by:
Stephan Mueller <smueller@chronox.de> Signed-off-by:
-
- Aug 04, 2017
-
-
Gary R Hook authored
Signed-off-by:
-
Gary R Hook authored
The CCP supports a limited set of unit-size values. Change the check for this parameter such that acceptable values match the enumeration. Then clarify the conditions under which we must use the fallback implementation. Signed-off-by:
-
Gary R Hook authored
Vet the key using the available standard function Signed-off-by:
-
Gary R Hook authored
Version 5 CCPs have some new requirements for XTS-AES: the type field must be specified, and the key requires 512 bits, with each part occupying 256 bits and padded with zeroes. cc: <stable@vger.kernel.org> # 4.9.x+ Signed-off-by:
Gary R Hook <ghook@amd.com> Signed-off-by:
-
Ard Biesheuvel authored
For the final round, avoid the expanded and padded lookup tables exported by the generic AES driver. Instead, for encryption, we can perform byte loads from the same table we used for the inner rounds, which will still be hot in the caches. For decryption, use the inverse AES Sbox directly, which is 4x smaller than the inverse lookup table exported by the generic driver. This should significantly reduce the Dcache footprint of our code, which makes the code more robust against timing attacks. It does not introduce any additional module dependencies, given that we already rely on the core AES module for the shared key expansion routines. It also frees up register x18, which is not available as a scratch register on all platforms, which and so avoiding it improves shareability of this code. Signed-off-by:
Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by:
-
Ard Biesheuvel authored
For the final round, avoid the expanded and padded lookup tables exported by the generic AES driver. Instead, for encryption, we can perform byte loads from the same table we used for the inner rounds, which will still be hot in the caches. For decryption, use the inverse AES Sbox directly, which is 4x smaller than the inverse lookup table exported by the generic driver. This should significantly reduce the Dcache footprint of our code, which makes the code more robust against timing attacks. It does not introduce any additional module dependencies, given that we already rely on the core AES module for the shared key expansion routines. Signed-off-by:
-
Ard Biesheuvel authored
Implement a NEON fallback for systems that do support NEON but have no support for the optional 64x64->128 polynomial multiplication instruction that is part of the ARMv8 Crypto Extensions. It is based on the paper "Fast Software Polynomial Multiplication on ARM Processors Using the NEON Engine" by Danilo Camara, Conrado Gouvea, Julio Lopez and Ricardo Dahab (https://hal.inria.fr/hal-01506572 ), but has been reworked extensively for the AArch64 ISA. On a low-end core such as the Cortex-A53 found in the Raspberry Pi3, the NEON based implementation is 4x faster than the table based one, and is time invariant as well, making it less vulnerable to timing attacks. When combined with the bit-sliced NEON implementation of AES-CTR, the AES-GCM performance increases by 2x (from 58 to 29 cycles per byte). Signed-off-by:
-
Ard Biesheuvel authored
Implement a NEON fallback for systems that do support NEON but have no support for the optional 64x64->128 polynomial multiplication instruction that is part of the ARMv8 Crypto Extensions. It is based on the paper "Fast Software Polynomial Multiplication on ARM Processors Using the NEON Engine" by Danilo Camara, Conrado Gouvea, Julio Lopez and Ricardo Dahab (https://hal.inria.fr/hal-01506572 ) On a 32-bit guest executing under KVM on a Cortex-A57, the new code is not only 4x faster than the generic table based GHASH driver, it is also time invariant. (Note that the existing vmull.p64 code is 16x faster on this core). Signed-off-by:
-
Ard Biesheuvel authored
Currently, the AES-GCM implementation for arm64 systems that support the ARMv8 Crypto Extensions is based on the generic GCM module, which combines the AES-CTR implementation using AES instructions with the PMULL based GHASH driver. This is suboptimal, given the fact that the input data needs to be loaded twice, once for the encryption and again for the MAC calculation. On Cortex-A57 (r1p2) and other recent cores that implement micro-op fusing for the AES instructions, AES executes at less than 1 cycle per byte, which means that any cycles wasted on loading the data twice hurt even more. So implement a new GCM driver that combines the AES and PMULL instructions at the block level. This improves performance on Cortex-A57 by ~37% (from 3.5 cpb to 2.6 cpb) Signed-off-by:
-
Ard Biesheuvel authored
Of the various chaining modes implemented by the bit sliced AES driver, only CTR is exposed as a synchronous cipher, and requires a fallback in order to remain usable once we update the kernel mode NEON handling logic to disallow nested use. So wire up the existing CTR fallback C code. Signed-off-by:
-
Ard Biesheuvel authored
To accommodate systems that disallow the use of kernel mode NEON in some circumstances, take the return value of may_use_simd into account when deciding whether to invoke the C fallback routine. Signed-off-by:
-
Ard Biesheuvel authored
To accommodate systems that may disallow use of the NEON in kernel mode in some circumstances, introduce a C fallback for synchronous AES in CTR mode, and use it if may_use_simd() returns false. Signed-off-by:
-
Ard Biesheuvel authored
The arm64 kernel will shortly disallow nested kernel mode NEON. So honour this in the ARMv8 Crypto Extensions implementation of CCM-AES, and fall back to a scalar implementation using the generic crypto helpers for AES, XOR and incrementing the CTR counter. Signed-off-by:
-
Ard Biesheuvel authored
The arm64 kernel will shortly disallow nested kernel mode NEON, so add a fallback to scalar code that can be invoked in that case. Signed-off-by:
-
Ard Biesheuvel authored
In order to be able to reuse the generic AES code as a fallback for situations where the NEON may not be used, update the key handling to match the byte order of the generic code: it stores round keys as sequences of 32-bit quantities rather than streams of bytes, and so our code needs to be updated to reflect that. Signed-off-by:
-
Ard Biesheuvel authored
The arm64 kernel will shortly disallow nested kernel mode NEON, so add a fallback to scalar code that can be invoked in that case. Signed-off-by:
-
Ard Biesheuvel authored
The arm64 kernel will shortly disallow nested kernel mode NEON, so add a fallback to scalar C code that can be invoked in that case. Signed-off-by:
-
Ard Biesheuvel authored
The arm64 kernel will shortly disallow nested kernel mode NEON, so add a fallback to scalar C code that can be invoked in that case. Signed-off-by:
-
Ard Biesheuvel authored
The arm64 kernel will shortly disallow nested kernel mode NEON, so add a fallback to scalar C code that can be invoked in that case. Signed-off-by:
-
Ard Biesheuvel authored
The arm64 kernel will shortly disallow nested kernel mode NEON, so add a fallback to scalar C code that can be invoked in that case. Signed-off-by:
-
Ard Biesheuvel authored
There are quite a number of occurrences in the kernel of the pattern if (dst != src) memcpy(dst, src, walk.total % AES_BLOCK_SIZE); crypto_xor(dst, final, walk.total % AES_BLOCK_SIZE); or crypto_xor(keystream, src, nbytes); memcpy(dst, keystream, nbytes); where crypto_xor() is preceded or followed by a memcpy() invocation that is only there because crypto_xor() uses its output parameter as one of the inputs. To avoid having to add new instances of this pattern in the arm64 code, which will be refactored to implement non-SIMD fallbacks, add an alternative implementation called crypto_xor_cpy(), taking separate input and output arguments. This removes the need for the separate memcpy(). Signed-off-by: -
Ard Biesheuvel authored
In preparation of introducing crypto_xor_cpy(), which will use separate operands for input and output, modify the __crypto_xor() implementation, which it will share with the existing crypto_xor(), which provides the actual functionality when not using the inline version. Signed-off-by:
-
- Aug 03, 2017
-
-
Zain Wang authored
Sometime we would unable to dequeue the crypto request, in this case, we should finish crypto and return the err code. Signed-off-by:
zain wang <wzz@rock-chips.com> Signed-off-by:
-
Zain Wang authored
It's illegal to call the completion function from hardirq context, it will cause runtime tests to fail. Let's build a new task (done_task) for moving update operation from hardirq context. Signed-off-by:
-
Martin Kaiser authored
The driver is ported from Freescale's Linux git and can be found in the vendor/freescale/imx_2.6.35_maintain branch. The driver supports both RNG version C that's part of some Freescale i.MX3 SoCs and version B that is available on i.MX2x chipsets. Signed-off-by:
Steffen Trumtrar <s.trumtrar@pengutronix.de> Signed-off-by:
Martin Kaiser <martin@kaiser.cx> Reviewed-by:
PrasannaKumar Muralidharan <prasannatsmkumar@gmail.com> Signed-off-by:
-
Steffen Trumtrar authored
Add binding documentation for the Freescale RNGC found on some i.MX2/3 SoCs. Signed-off-by:
Rob Herring <robh@kernel.org> Signed-off-by:
-
PrasannaKumar Muralidharan authored
Modify Kconfig help text to reflect the fact that random data from hwrng is fed into kernel random number generator's entropy pool. Signed-off-by:
-
Ard Biesheuvel authored
The scompress code allocates 2 x 128 KB of scratch buffers for each CPU, so that clients of the async API can use synchronous implementations even from atomic context. However, on systems such as Cavium Thunderx (which has 96 cores), this adds up to a non-negligible 24 MB. Also, 32-bit systems may prefer to use their precious vmalloc space for other things,especially since there don't appear to be any clients for the async compression API yet. So let's defer allocation of the scratch buffers until the first time we allocate an acompress cipher based on an scompress implementation. Signed-off-by:
-
Ard Biesheuvel authored
When allocating the per-CPU scratch buffers, we allocate the source and destination buffers separately, but bail immediately if the second allocation fails, without freeing the first one. Fix that. Signed-off-by:
-
