Cloud-Native Streaming Data Platform for AI

By leveraging cloud-native architecture and tiered storage, Kafka accommodates vast amounts of real-time data. However, the challenge lies in flexible scaling and cost optimization. AutoMQ, with its new architecture based on EBS shared storage and object storage, provides significant elasticity improvements for scaling. Its feature of separating storage and computing aligns well with current operational requirements based on Kubernetes. When combined with RedNote's current messaging engine architecture, it can lead to greater cost savings and efficiency gains.

JD initially used Kafka, but the double triple-replication strategy led to nine times data redundancy, resulting in six copies being stored. By adopting AutoMQ, which directly relies on the underlying cloud storage CubeFS, the need for upper-layer replication is eliminated.

The architecture is expected to save two-thirds of storage costs when fully implemented. AutoMQ's stateless computing layer perfectly meets the requirements for containerization transformation, significantly enhancing system flexibility.

Geely Auto Group originally used Apache Kafka, the constraints of its architecture made it difficult to scale the cluster storage. Therefore, the only way to avoid expanding the cluster storage was to reduce the data retention time, which had a certain impact on the business.

AutoMQ completely solved the pain point of Kafka's elastic scaling, allowing the cluster to scale up and down quickly, safely, and automatically, greatly reducing the complexity of Kafka's scaling operations.

Poizon used to rely on Kafka to build the observability platform, requiring a team to spend several days each quarter on scaling operations. Since adopting AutoMQ, data storage has been moved to object storage, making the compute layer stateless and fully compatible with Kafka.

This has enabled automatic elastic scaling without manual intervention, significantly reducing cloud resource costs by up to 85%.

Asia's Quora, Zhihu (NYSE: ZH), has significantly reduced operational complexity and saved a lot of costs by using AutoMQ instead of Kafka. Through this strategic implementation, Zhihu has not only streamlined its data operations but also embraced technical innovations that AutoMQ offers over traditional Apache Kafka, ensuring enhanced performance, scalability, and efficiency. Check out this customer case to see how AutoMQ leverages technical innovations on Kafka to help Zhihu build a modern data infrastructure.