Classifying Serialization Formats for Inter-Service Communication in Distributed Systems

Abstract

This study focuses on classifying serialization formats used in inter-service communication (ISC) within distributed systems and exploring their historical development. We have examined key features of human-readable formats such as XML, JSON, and YAML, binary formats like Protocol Buffers and Apache Avro, and columnar formats such as Apache Parquet and ORC, among others. Our results have indicated a significant shift toward binary formats optimized for speed and compactness in recent years. The industry demand score for Apache Avro and Google Protocol Buffers has been shown to be much higher than for Thrift. JSON remains on top, showing the best score for general technology adoption and industry demand score; Zero-copy formats like Can’n proto and Flatbuffers show lower industry demand scores in comparison to AVRO and Protocol Buffers but are useful in specific scenarios.