Serialization in Python

For JSON, orjson is a fast Rust-backed library that natively serializes dataclasses, datetimes and NumPy arrays; it returns bytes rather than str. msgspec handles JSON and MessagePack and validates against typed Struct classes as it decodes, which makes it a fast combined parser and validator. The msgpack package implements MessagePack, a compact binary format that is a good fit for caches and internal messages. marshmallow uses explicit schemas to convert objects to and from primitive types with validation, and is common in Flask projects.

Pickle covers Python objects that JSON cannot represent. cloudpickle can serialize functions, lambdas and interactively defined classes by value, which is why distributed frameworks such as Dask, Ray and PySpark use it to ship code to workers. dill extends pickle to even more object types. Never unpickle data from an untrusted source, because loading a pickle can execute arbitrary code.