Join us on Discord!
You can help CodeWalrus stay online by donating here.

HASHLIB - Cryptography Library for the CE

Started by ACagliano, July 15, 2021, 02:47:58 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

ACagliano

For the past few months, I've been working with beckadamtheinventor and others on HASHLIB, a library that provides d*mn-near industry standard encryption for the TI-84+ CE. It is a Libload Library, meaning it works the same way as the other libraries distributed with the toolchain (GRAPHX, FILEIOC, etc). Simply send HASHLIB.8xv to your calculator, move the .lib and .h file to your $CEDEV/lib/libload and $CEDEV/include directories respectively, then `#include <hashlib.h>` in your C project source. At this point you can use any of the functions in hashlib in your code.

HASHLIB provides the following crypto standards:


(1) SHA-256 Hash
A rather fast implementation of the SHA-256 crypto hash. First written in C, myself and beck endeavored to rewrite it in EZ80 Assembly because the C version was quite slow. I started things off with a code of the Init, Update, and Final functions and a skeleton of the transform function. Beck "corrected" any mistakes I made due to my Asm knowledge being (1) rusty, and (2) from Z80 Asm, as well as optimized all of the functions, and wrote much of the transform math routines. The current SHA-256 implementation sports a speed increase of 9x over the C version, and can fully hash GRAPHX, the largest of the toolchain libraries, in ~1.5 seconds.

(2) An Entropy-Based Secure Random Number Generator
This PRNG, unlike the toolchain's default is not deterministic in nature. It works by polling every bit in the CE's unmapped memory, searching for a floating bit with the least bias, upon which it sets its state internally to read from the address containing that bit. An add entropy function can be called by the user, but is also called automatically every time a random number is generated. This function reads from the entropy source to a reserved pool 192 bytes large. Because the maximum bias accepted by the entropy source selection function is 75/25, this gives us an entropy of .811 bit per byte, which multiplies out to ~150 bits of entropy for the entire pool. Bear in mind that is the worst case entropy possible--the entropy source selected will likely be higher. This entropy goes into generating a 32-bit random number; the pool is hashed using SHA-256 and then xored cyclically into the random number. The PRNG passes all Dieharder tests when run on 16KB of data as well as 1MB of data.

(3) Advanced Encryption Standard, CBC Mode
As the above implies, HASHLIB provides routines for encrypting and decrypting data using the CBC mode AES block cipher. This implementation has support for 128, 192, and 256 bit keys. Additionally, the CBC-MAC protocol is also provided, which you can pass data to and generate a message authentication code one block size in length. The library provides a macro that takes a plaintext, padding scheme, an output buffer, and two AES key schedules (one for encryption, one for MAC), as well as an IV. It encrypts the plaintext using the IV and encryption key schedule (padding appropriately), then runs the CBC-MAC algorithm on the encryption output and the IV, using the MAC key schedule (the MAC variant of the CBC cipher does not take an IV, but rather initializes it to a constant value causing it to act more like a counter than a nonce), appending the output of that MAC function to the ciphertext, providing an authenticated encryption scheme derived from IPsec.

(4) RSA, Encryption Only
A WIP; This library will also provide support for public key encryption over RSA keys of a modulus length up to 2048 bits. It will depend on a BIGINT lib by beckadamtheinventor currently a WIP as well (which is why RSA is still not added). Because generating two random coprime prime numbers of the required size on a CE would likely take a very long time, I decided to implement the encryption part only. Use this implementation in a server-centric setup where a server generates an RSA keypair, sends the public key to your calculator, and then the calculator encrypts a shared secret for AES, sends that to the server, and then the two communicate using AES.

(5) Padding Schemes (and Strip Padding)
HASHLIB provides functions to implement the ISO, PKCS7, and ANSIX928 padding schemes for AES as well as OAEP for RSA. For those who don't know, the ISO scheme pads with the byte 0x80 and then the bytes 0x00 until the end of the block. The PKCS7 scheme pads with the size of the padding. The ANSIX scheme pads with randomness. For OAEP, the scheme is implemented as indicated by the standard (a Feistel network where the message is padded with zeros to the modulus length minus 16, a 16-byte salt is generated, hashed, and xored with the message. The encoded message is then hashed and xored with the salt, and the encoded message and encoded salt are the OAEP-encoded plaintext). However, rather than an expandable hashing oracle, SHA-256 is applied cyclically to the message. This was done predominantly to save space in the library by using an existing hash rather than adding a new one specifically for this purpose.

The "Strip Padding" functions all properly strip the padding, returning the unpadded plaintext into a provided buffer as well as the size of the unpadded data into a return value. The only padding scheme it cannot decipher is ANSIX; if you are using this padding mode, you will have to know the expected plaintext size.


Other Functions Provided:
(a) hashlib_CompareDigest(): buffer comparison function that is resistant to timing attacks (written by jacobly in ez80 Assembly)
(b) Base64 Encode/Decode: Added these planning to implement bcrypt, but when Blowfish was replaced with AES, decided to just leave them in.
(c) Macros to return the padded size of an AES/RSA plaintext, useful for allocating a buffer large enough.
(d) Context erase function. When you are done with an SHA, AES, or RSA context, you can pass it to this function to write zeroes to it, erasing all traces of cryptographic state.

Special thanks to:
(1) beckadamtheinventor: Converted many of my C routines into EZ80 Assembly for speed
(2) Zeroko: Provided useful information on extracting entropy from the CE
(3) jacobly: Rewriting hashlib_CompareDigest() in EZ80 Assembly
(4) B-Con crypto algorithms, which was my source for the base of the AES and SHA implementations.

Project Page: http://cagstech.com/tistuff/hashlib/
Github: https://github.com/acagliano/hashlib
  • Calculators owned: TI-84+ CE

ACagliano

The Time Has Come!! for another progress update.

HASHLIB v6 is out on Github: https://github.com/acagliano/hashlib/releases/latest

Here's What's New:
(1) MGF1 hashing function is added. It uses SHA-256 and hashes a given length of data as well as a 4-byte counter.
(2) OAEP now uses MGF1 (as is standard) instead of cyclic SHA-256.
(3) SPRNG now minimizes entropy loss due to correlation (thanks, Zeroko!) and runs ~1.5x faster for generating large pools of random data. The entropy pool is reduced from 192 bytes to 119 bytes, but each time we update a byte in the pool from the entropy source, it is a composite of 7 distinct reads xor'd together.
(4) SPRNGRandom() rewritten in assembly. Thanks to jacobly for optimization and some small code corrections.
(5) SPRNG now uses FastMem for its SHA memory and entropy pool to accelerate even faster.
(6) AES ECB-mode single block encrypt/decrypt functions exposed to let skilled users construct their own cipher modes
Still waiting on vint powmod to be fixed in the bigintce library, and then RSA will be added for v7.
  • Calculators owned: TI-84+ CE

ACagliano

Update HASHLIB moves into RC-1

With all the thanks in the world to jacobly from Cemetech for the modular exponentiation function we needed for RSA, HASHLIB is now formally complete and in release candidate phase (apart from the possibility of adding ECDSA in the distant future) and has been released on github for testing.

As many of you may know and others will learn by reading this, HASHLIB contains the following crytographic implementations:

<> A secure PRNG that produces ~96 bits of entropy per 32-bit integer generated.
<> The SHA-256 cryptographic hash.
<> An implementation of Advanced Encryption Standard (AES), for 128, 192, and 256 bit keys.
<> An implementation of RSA encryption up to 2048 bits in key length.
<> An implementation of the appropriate padding schemes for the above encryptions.
<> An implementation of SSL signature verification using the RSA with SHA-256 signing algorithm.

Feel free to download and test against commonly used cryptography libraries and report back on compatibility or lack thereof.

https://github.com/acagliano/hashlib/releases/tag/v7-RC1
  • Calculators owned: TI-84+ CE

ACagliano

    An eighth release of HASHLIB is now available for download. It adds two new implementations as well as some side-channel resistance enhancements. The new library release offers the following:
    • A hardware-based secure RNG
    • SHA-256 hash
    • SHA-256 HMAC
    • SHA-256 MGF1
    • PBKDF2 via SHA-256 HMAC
    • Advanced Encryption Standard (AES), for 128, 192, and 256 bit keys
    • automatic application and stripping of preferred AES padding schemes (ISO-9797 M2 or PKCS#7), where applicable
    • RSA public key encryption for public modulus between 1024 and 2048 bits
    • Automatic application and stripping of RSAEP-OAEP via PKCS#1 v2.2
    • Authentication of SSL certificates signed with RSA with SHA-256
    • Digest comparison function resistant to timing analysis
    • Digest to hex-string function

    In addition, the following measures were taken for side channel resistance:
    • All user-facing encryptor functions zero all memory from the start of the current stack frame to the bottom of the stack before handing back control, to prevent state leak.
    • All user-facing encryptor functions disable interrupts while running, which prevents the OS from responding to USB activity. This serves to stop attempts to map the contents of the device memory while sensitive functions are running.

    In addition, the documentation is split into a "Quick Reference" for general assistance, and a Cryptanalysis that provides more technical details (for peer-review).

    HASHLIB is still under analysis and peer-review to continue to harden it. Feel free to contribute suggestions for this.

https://github.com/acagliano/hashlib/releases/latest
  • Calculators owned: TI-84+ CE

Dream of Omnimaga

I'm glad to see this worked on still :), I wonder how much of this will be of use for your other project TI-Trek? (eg server credentials)
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

ACagliano

#5
Quote from: DJ Omnimaga on February 05, 2022, 06:27:57 PMI'm glad to see this worked on still :), I wonder how much of this will be of use for your other project TI-Trek? (eg server credentials)
TI-Trek does actually use HASHLIB for login, and TI-Trek is actually my method of testing it. TI-Trek's login process is like so:
  • Users register on the web deck (website). Doing so generates them a login token, which is a PBKDF2 key exported into an appvar. They place that appvar on their calculator.
  • Running the game with that appvar present imports the key to user settings and deletes the keyfile.
  • Login to the Trek server causes the server to generate an RSA keypair. The length of the key is configurable between 1024 and 2048. The public key is sent to the calculator. The public exponent is the standard 65537.
  • The calculator uses the SPRNG to generate an AES key that is 256-bits long. That key is encrypted with RSA and sent to the server.
  • The server decrypts that key and sets up an AES session.
  • The calculator AES-encrypts the login token, which the server decrypts to authenticate the user. An account with a matching token is logged into.
  • The AES session is then destroyed.
Because the RSA implementation only encrypts, you have to let the remote host, usually a more-powerful server, actually set up the RSA session, then the AES key goes from calculator to server. It's a bit of a caveat, since doing the necessary legwork to set up an RSA session on a calculator would be computationally intensive and take a long time. But, it is a fully secure construction.

But even apart from TI-Trek, I'm hoping other people will find use of this too. It certainly is of significant use to anyone doing networking who needs secure channels.
  • Calculators owned: TI-84+ CE

ACagliano

#6
Update and API revision

HASHLIB's API got a bit of a facelift recently with the function set changing nomenclature depending on what function is being served by the group of functions. No longer are hashlib functions prefixed with hashlib_, they are prefixed with a class-esque namespace indicating what they do. Like so:

Invocations of the Secure RNG within HASHLIB look like this:
csrand_init();     initializes the entropy source internally
csrand_get();      returns a random 32-bit unsigned int     
csrand_fill();     fills a buffer to a size with random bytes
Invocations of Hashing look like this:
hash_ctx hash;
uint8_t out[32];
hash_init(&hash, ALG_NAME)     initializes a context of type ALG_NAME
hash_update(&hash, data, len)
hash_final(&hash, out)
hash_mgf1(..., ALG_NAME)

/* Due to the structure of the hash context, the following are
viable alternatives to hash_update and hash_final:
uint8_t out[32];
hash_ctx hash;
hash_init(&hash, SHA256);
hash.update(&hash.Hash, data, len);
hash.final(&hash.Hash, out);
Invocations of HMAC are set up and have the same alternatives as hashing:
hmac_init()
hmac_update()
hmac_final()

hmac_pbkdf2(..., ALG_NAME)
The AES cipher is invoked like so:
aes_loadkey()
aes_encrypt()
aes_decrypt()
The RSA cipher is invoked like so:
rsa_encrypt()
// The RSA encryptor takes a hash ID specifier so that, if other hashes
// are added at some point, they can be used with OAEP internal to RSA.
Digest operations look like this:
digest_tostring()
digest_compare()

The latest update is available here.
The header has some slight issues in the documentation... stay tuned for a 9.1 update for a fix, but everything works as of v9.
https://github.com/acagliano/hashlib/releases/latest

I am also opening the floor if anyone wants to contribute hashing algorithms. Fast hashes like sha512, sha1 welcome, as well as slower hashes like argon.
  • Calculators owned: TI-84+ CE

ACagliano

#7
Update v9.3
https://github.com/acagliano/hashlib/releases/tag/v9.3

Major Changes to AES implementation
  • aes_loadkey() is now aes_init()
  • Most of the cipher initialization now happens via the aes_init() function, including the nonce, the cipher mode, and the padding mode, and other functionalities.
  • Cipher configuration parameters passed as flags bitwise-OR'd together.
  • Padding in CBC mode is now added internally within the last block of encryption. Callable functions for padding are no longer used and have been removed.
  • Cipher state "contexts" loaded with init() are stateful and one-directional.Once a context is used for encryption or decryption, attempting to use it for the opposite operation will return an error. Two-way communication now needs two contexts initialized with the same key and parameters, but the correct nonce.
  • Encrypt and decrypt are now chainable operations. This means that `aes_encrypt(msg1+msg2)` is functionally-identical to `aes_encrypt(msg1) + aes_encrypt(msg2)`, once padding is cleared (if applicable). Same is true for decrypt.

New API for AES
aes_init(aes_ctx* ctx, const void* key, size_t keylen, const void* iv, uint24_t flags);
aes_encrypt(aes_ctx* ctx, void* plaintext, size_t len, void* ciphertext);
aes_decrypt(aes_ctx* ctx, void* ciphertext, size_t len, void* plaintext);

Arguments for Flags
// cipher mode (2-bit flag)
AES_MODE_CBC     // default
AES_MODE_CTR

// padding mode (2-bit flag)
PAD_PKCS7        // default
PAD_ISO2

// CTR mode nonce length (4-bit flag)
AES_CTR_NONCELEN(len)     // default = 8 bytes

// CTR mode counter length (4-bit flag)
AES_CTR_COUNTERLEN(len)     // default = 8 bytes


// Ex1: Set CTR mode, with 8 byte nonce length and 4 byte counter length
aes_init(&ctx, key, sizeof key, iv, AES_MODE_CTR | AES_CTR_NONCELEN(8) | AES_CTR_COUNTERLEN(4));

// Ex2: Set CBC mode, with padding mode ISO-9797 M2
aes_init(&ctx, key, sizeof key, iv, AES_MODE_CBC | PAD_ISO2);

Test away!
  • Calculators owned: TI-84+ CE

ACagliano

Update 10.0

HASHLIB has been renamed to CryptX and the entire project has been split up along functional lines into 3 libraries: HASHLIB, ENCRYPT, and ENCODEX. HASHLIB provides hashing, hmac, and related functionality (like hash-based mask generation, hmac-pbkdf, etc). ENCRYPT provides the secure RNG, AES, and RSA. ENCODEX provides an ASN.1 parser, Base 64 encoding/decoding, and BPP encoding/decoding.

Documentation is no longer in the git repository or in the artifacts on github and is now available as a docusite on github: https://acagliano.github.io/cryptx/index.html

Download: https://github.com/acagliano/cryptx/releases/latest
  • Calculators owned: TI-84+ CE

Powered by EzPortal