Repository navigation
[FEATURE REQUEST] Fernet: allow getting/using bytes in encrypt/decrypt (gain 80-100% performance?) #4953
Description
Fernet input currently need to be url_base64 encoded, and output is also base64 encoded
When sending such encrypted data over the network, as a mean to save bandwidth (around 40% !) , one need to use raw bytes instead of base64 encoded data:
enc=Fernet.encrypt(data)
AA =base64_decode(enc)
send AA over the network
enc=base64_encode(a)
data=Fernet.decrypt(enc)
A useful feature would be to permit getting raw bytes out of encrypt and providing raw bytes to decrypt to avoid this base64 encode/decode phase (which wastes CPU cycles)
Quick benchmark (If I got it right) that shows that 40-50% the encrypt/decrypt time is spent base64 decoding/encoding:
import time
import base64
from cryptography.fernet import Fernet
import os
data = b'\x00'*15*1024*1024
data = os.urandom(15*1024*1024)
f = Fernet(Fernet.generate_key())
a1=time.time()
g=base64.urlsafe_b64encode(data)
a2 = time.time()
print("base64 encode : {}".format(a2-a1))
a1=time.time()
enc=f.encrypt(data)
a2 = time.time()
print("encrypt : {}".format(a2-a1))
a1=time.time()
g=base64.urlsafe_b64decode(enc)
a2 = time.time()
print("base64 decode : {}".format(a2-a1))
a1=time.time()
g=f.decrypt(enc)
a2 = time.time()
print("decrypt : {}".format(a2-a1))
results with 15 MB of random bytes
base64 encode : 0.04491138458251953
encrypt : 0.10439181327819824
base64 decode : 0.05823516845703125
decrypt : 0.10047411918640137
results with 15 MB of null bytes
base64 encode : 0.038869380950927734
encrypt : 0.11022663116455078
base64 decode : 0.05708146095275879
decrypt : 0.10116434097290039
I jury-rigged a quick test with a patched Fernet without base64 encode/decode and I got an almost 100-300% (was around 80-100% for 50 MB of data) performance increase in encrypting/decrypting :
encrypt : 0.10713529586791992
encrypt without b64: 0.05477595329284668
decrypt : 0.10087037086486816
decrypt without b64: 0.036371469497680664
Activity
alex commentedon Jul 28, 2019
Fernet implements a standard, which does not have a binary encoding. We do not implement any extensions to the spec.
You're welcome to lobby the spec authors for binary support (https://github.com/fernet/spec), but be warned the spec is basically frozen at this point.