CodeWalrus

Development => Calculators => Calc Projects, Programming & Tutorials => Topic started by: ACagliano on July 15, 2021, 02:47:58 PM

Title: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on July 15, 2021, 02:47:58 PM
For the past few months, I've been working with beckadamtheinventor and others on HASHLIB, a library that provides d*mn-near industry standard encryption for the TI-84+ CE. It is a Libload Library, meaning it works the same way as the other libraries distributed with the toolchain (GRAPHX, FILEIOC, etc). Simply send HASHLIB.8xv to your calculator, move the .lib and .h file to your $CEDEV/lib/libload and $CEDEV/include directories respectively, then `#include <hashlib.h>` in your C project source. At this point you can use any of the functions in hashlib in your code.

HASHLIB provides the following crypto standards:


(1) SHA-256 Hash
A rather fast implementation of the SHA-256 crypto hash. First written in C, myself and beck endeavored to rewrite it in EZ80 Assembly because the C version was quite slow. I started things off with a code of the Init, Update, and Final functions and a skeleton of the transform function. Beck "corrected" any mistakes I made due to my Asm knowledge being (1) rusty, and (2) from Z80 Asm, as well as optimized all of the functions, and wrote much of the transform math routines. The current SHA-256 implementation sports a speed increase of 9x over the C version, and can fully hash GRAPHX, the largest of the toolchain libraries, in ~1.5 seconds.

(2) An Entropy-Based Secure Random Number Generator
This PRNG, unlike the toolchain's default is not deterministic in nature. It works by polling every bit in the CE's unmapped memory, searching for a floating bit with the least bias, upon which it sets its state internally to read from the address containing that bit. An add entropy function can be called by the user, but is also called automatically every time a random number is generated. This function reads from the entropy source to a reserved pool 192 bytes large. Because the maximum bias accepted by the entropy source selection function is 75/25, this gives us an entropy of .811 bit per byte, which multiplies out to ~150 bits of entropy for the entire pool. Bear in mind that is the worst case entropy possible--the entropy source selected will likely be higher. This entropy goes into generating a 32-bit random number; the pool is hashed using SHA-256 and then xored cyclically into the random number. The PRNG passes all Dieharder tests when run on 16KB of data as well as 1MB of data.

(3) Advanced Encryption Standard, CBC Mode
As the above implies, HASHLIB provides routines for encrypting and decrypting data using the CBC mode AES block cipher. This implementation has support for 128, 192, and 256 bit keys. Additionally, the CBC-MAC protocol is also provided, which you can pass data to and generate a message authentication code one block size in length. The library provides a macro that takes a plaintext, padding scheme, an output buffer, and two AES key schedules (one for encryption, one for MAC), as well as an IV. It encrypts the plaintext using the IV and encryption key schedule (padding appropriately), then runs the CBC-MAC algorithm on the encryption output and the IV, using the MAC key schedule (the MAC variant of the CBC cipher does not take an IV, but rather initializes it to a constant value causing it to act more like a counter than a nonce), appending the output of that MAC function to the ciphertext, providing an authenticated encryption scheme derived from IPsec.

(4) RSA, Encryption Only
A WIP; This library will also provide support for public key encryption over RSA keys of a modulus length up to 2048 bits. It will depend on a BIGINT lib by beckadamtheinventor currently a WIP as well (which is why RSA is still not added). Because generating two random coprime prime numbers of the required size on a CE would likely take a very long time, I decided to implement the encryption part only. Use this implementation in a server-centric setup where a server generates an RSA keypair, sends the public key to your calculator, and then the calculator encrypts a shared secret for AES, sends that to the server, and then the two communicate using AES.

(5) Padding Schemes (and Strip Padding)
HASHLIB provides functions to implement the ISO, PKCS7, and ANSIX928 padding schemes for AES as well as OAEP for RSA. For those who don't know, the ISO scheme pads with the byte 0x80 and then the bytes 0x00 until the end of the block. The PKCS7 scheme pads with the size of the padding. The ANSIX scheme pads with randomness. For OAEP, the scheme is implemented as indicated by the standard (a Feistel network where the message is padded with zeros to the modulus length minus 16, a 16-byte salt is generated, hashed, and xored with the message. The encoded message is then hashed and xored with the salt, and the encoded message and encoded salt are the OAEP-encoded plaintext). However, rather than an expandable hashing oracle, SHA-256 is applied cyclically to the message. This was done predominantly to save space in the library by using an existing hash rather than adding a new one specifically for this purpose.

The "Strip Padding" functions all properly strip the padding, returning the unpadded plaintext into a provided buffer as well as the size of the unpadded data into a return value. The only padding scheme it cannot decipher is ANSIX; if you are using this padding mode, you will have to know the expected plaintext size.


Other Functions Provided:
(a) hashlib_CompareDigest(): buffer comparison function that is resistant to timing attacks (written by jacobly in ez80 Assembly)
(b) Base64 Encode/Decode: Added these planning to implement bcrypt, but when Blowfish was replaced with AES, decided to just leave them in.
(c) Macros to return the padded size of an AES/RSA plaintext, useful for allocating a buffer large enough.
(d) Context erase function. When you are done with an SHA, AES, or RSA context, you can pass it to this function to write zeroes to it, erasing all traces of cryptographic state.

Special thanks to:
(1) beckadamtheinventor: Converted many of my C routines into EZ80 Assembly for speed
(2) Zeroko: Provided useful information on extracting entropy from the CE
(3) jacobly: Rewriting hashlib_CompareDigest() in EZ80 Assembly
(4) B-Con crypto algorithms, which was my source for the base of the AES and SHA implementations.

Project Page: http://cagstech.com/tistuff/hashlib/
Github: https://github.com/acagliano/hashlib
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on July 29, 2021, 04:01:24 PM
The Time Has Come!! for another progress update.

HASHLIB v6 is out on Github: https://github.com/acagliano/hashlib/releases/latest

Here's What's New:
(1) MGF1 hashing function is added. It uses SHA-256 and hashes a given length of data as well as a 4-byte counter.
(2) OAEP now uses MGF1 (as is standard) instead of cyclic SHA-256.
(3) SPRNG now minimizes entropy loss due to correlation (thanks, Zeroko!) and runs ~1.5x faster for generating large pools of random data. The entropy pool is reduced from 192 bytes to 119 bytes, but each time we update a byte in the pool from the entropy source, it is a composite of 7 distinct reads xor'd together.
(4) SPRNGRandom() rewritten in assembly. Thanks to jacobly for optimization and some small code corrections.
(5) SPRNG now uses FastMem for its SHA memory and entropy pool to accelerate even faster.
(6) AES ECB-mode single block encrypt/decrypt functions exposed to let skilled users construct their own cipher modes
Still waiting on vint powmod to be fixed in the bigintce library, and then RSA will be added for v7.
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on October 22, 2021, 03:07:20 AM
Update HASHLIB moves into RC-1

With all the thanks in the world to jacobly from Cemetech for the modular exponentiation function we needed for RSA, HASHLIB is now formally complete and in release candidate phase (apart from the possibility of adding ECDSA in the distant future) and has been released on github for testing.

As many of you may know and others will learn by reading this, HASHLIB contains the following crytographic implementations:

<> A secure PRNG that produces ~96 bits of entropy per 32-bit integer generated.
<> The SHA-256 cryptographic hash.
<> An implementation of Advanced Encryption Standard (AES), for 128, 192, and 256 bit keys.
<> An implementation of RSA encryption up to 2048 bits in key length.
<> An implementation of the appropriate padding schemes for the above encryptions.
<> An implementation of SSL signature verification using the RSA with SHA-256 signing algorithm.

Feel free to download and test against commonly used cryptography libraries and report back on compatibility or lack thereof.

https://github.com/acagliano/hashlib/releases/tag/v7-RC1
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on February 05, 2022, 04:50:00 PM

https://github.com/acagliano/hashlib/releases/latest
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: Dream of Omnimaga on February 05, 2022, 06:27:57 PM
I'm glad to see this worked on still :), I wonder how much of this will be of use for your other project TI-Trek? (eg server credentials)
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on February 06, 2022, 03:34:52 AM
Quote from: DJ Omnimaga on February 05, 2022, 06:27:57 PMI'm glad to see this worked on still :), I wonder how much of this will be of use for your other project TI-Trek? (eg server credentials)
TI-Trek does actually use HASHLIB for login, and TI-Trek is actually my method of testing it. TI-Trek's login process is like so:
Because the RSA implementation only encrypts, you have to let the remote host, usually a more-powerful server, actually set up the RSA session, then the AES key goes from calculator to server. It's a bit of a caveat, since doing the necessary legwork to set up an RSA session on a calculator would be computationally intensive and take a long time. But, it is a fully secure construction.

But even apart from TI-Trek, I'm hoping other people will find use of this too. It certainly is of significant use to anyone doing networking who needs secure channels.
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on April 15, 2022, 03:20:10 AM
Update and API revision

HASHLIB's API got a bit of a facelift recently with the function set changing nomenclature depending on what function is being served by the group of functions. No longer are hashlib functions prefixed with hashlib_, they are prefixed with a class-esque namespace indicating what they do. Like so:

Invocations of the Secure RNG within HASHLIB look like this:
csrand_init();     initializes the entropy source internally
csrand_get();      returns a random 32-bit unsigned int     
csrand_fill();     fills a buffer to a size with random bytes
Invocations of Hashing look like this:
hash_ctx hash;
uint8_t out[32];
hash_init(&hash, ALG_NAME)     initializes a context of type ALG_NAME
hash_update(&hash, data, len)
hash_final(&hash, out)
hash_mgf1(..., ALG_NAME)

/* Due to the structure of the hash context, the following are
viable alternatives to hash_update and hash_final:
uint8_t out[32];
hash_ctx hash;
hash_init(&hash, SHA256);
hash.update(&hash.Hash, data, len);
hash.final(&hash.Hash, out);
Invocations of HMAC are set up and have the same alternatives as hashing:
hmac_init()
hmac_update()
hmac_final()

hmac_pbkdf2(..., ALG_NAME)
The AES cipher is invoked like so:
aes_loadkey()
aes_encrypt()
aes_decrypt()
The RSA cipher is invoked like so:
rsa_encrypt()
// The RSA encryptor takes a hash ID specifier so that, if other hashes
// are added at some point, they can be used with OAEP internal to RSA.
Digest operations look like this:
digest_tostring()
digest_compare()

The latest update is available here.
The header has some slight issues in the documentation... stay tuned for a 9.1 update for a fix, but everything works as of v9.
https://github.com/acagliano/hashlib/releases/latest

I am also opening the floor if anyone wants to contribute hashing algorithms. Fast hashes like sha512, sha1 welcome, as well as slower hashes like argon.
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on August 05, 2022, 02:49:06 AM
Update v9.3
https://github.com/acagliano/hashlib/releases/tag/v9.3

Major Changes to AES implementation

New API for AES
aes_init(aes_ctx* ctx, const void* key, size_t keylen, const void* iv, uint24_t flags);
aes_encrypt(aes_ctx* ctx, void* plaintext, size_t len, void* ciphertext);
aes_decrypt(aes_ctx* ctx, void* ciphertext, size_t len, void* plaintext);

Arguments for Flags
// cipher mode (2-bit flag)
AES_MODE_CBC     // default
AES_MODE_CTR

// padding mode (2-bit flag)
PAD_PKCS7        // default
PAD_ISO2

// CTR mode nonce length (4-bit flag)
AES_CTR_NONCELEN(len)     // default = 8 bytes

// CTR mode counter length (4-bit flag)
AES_CTR_COUNTERLEN(len)     // default = 8 bytes


// Ex1: Set CTR mode, with 8 byte nonce length and 4 byte counter length
aes_init(&ctx, key, sizeof key, iv, AES_MODE_CTR | AES_CTR_NONCELEN(8) | AES_CTR_COUNTERLEN(4));

// Ex2: Set CBC mode, with padding mode ISO-9797 M2
aes_init(&ctx, key, sizeof key, iv, AES_MODE_CBC | PAD_ISO2);

Test away!
Title: Re: HASHLIB - Cryptography Library for the CE
Post by: ACagliano on November 02, 2022, 01:52:06 AM
Update 10.0

HASHLIB has been renamed to CryptX and the entire project has been split up along functional lines into 3 libraries: HASHLIB, ENCRYPT, and ENCODEX. HASHLIB provides hashing, hmac, and related functionality (like hash-based mask generation, hmac-pbkdf, etc). ENCRYPT provides the secure RNG, AES, and RSA. ENCODEX provides an ASN.1 parser, Base 64 encoding/decoding, and BPP encoding/decoding.

Documentation is no longer in the git repository or in the artifacts on github and is now available as a docusite on github: https://acagliano.github.io/cryptx/index.html

Download: https://github.com/acagliano/cryptx/releases/latest