Bagaimana Grab Memproses Jutaan Order Per Hari | PZN Reaction

Overview

The video explains how Grab’s backend order platform is designed as a distributed system to handle very high order volume (millions/day). It focuses on:

• Database architecture
• Query types and access patterns
• Consistency strategy (strong vs eventual)
• Data synchronization between operational and analytical systems

After a passenger places a GrabFood/GrabMart order, the workflow is:

1. Merchant prepares the order
2. Driver collects the order
3. Driver delivers the order

The backend must support real-time operational queries while also enabling historical/statistical reporting.

---

1) System goal: handle order workflow at massive scale

Operational workflow needs

• Once an order is created, the system must support ongoing reads/updates needed by operations (merchant/driver coordination).

Scale challenge

• The system must sustain millions of daily orders and related queries in real time.
• It also needs to support longer-term analytics for reports and statistics.

---

2) Database design goals (stability, scalability, cost, consistency)

A key design decision is splitting workloads into:

Transactional workload (OLTP)

Used for the order lifecycle, requiring correctness for active orders.

• Writes
  • Create order
  • Update order
• Reads
  • Get order by Order ID
  • Get ongoing order(s) by Passenger ID
  • (Other lifecycle reads as needed)

Analytical workload (OLAP)

Used for historical and statistical queries.

• Read historical orders
• Compute statistics (e.g., breakdowns, counts)

---

3) Consistency strategy: strong vs eventual

Strong consistency (transactional)

• Applied to transactional queries
• Ensures real-time correctness for active orders

Eventual consistency (analytical)

• Applied to analytical queries
• Analytics can lag because instant correctness isn’t required the same way

---

4) Core strategy: split storage by workload (OLTP vs OLAP)

Grab uses two different databases, aligned with workload type:

OLTP database: DynamoDB (short retention)

• Technology: DynamoDB
• Purpose: transactional/order operations
• Retention: only keeps data for a short window (mentioned: ~3 months)

OLAP database: MySQL on RDS (long retention)

• Technology: MySQL on RDS
• Purpose: historical/statistics queries
• Retention: retains data longer to serve analytics and history

---

5) Data synchronization across OLTP and OLAP: ingestion/engagement pipeline

Because OLTP and OLAP are separate, the system uses asynchronous synchronization:

1. Write order data to DynamoDB (OLTP) first
2. Emit/send events into a streaming/ingestion pipeline
3. Consume/process events and update MySQL (OLAP)
   • This makes analytics eventually consistent

Streaming/messaging components (as described)

• Kafka (via a pipeline concept) for stream processing of order events
• Amazon SQS as a fallback/retry buffer when sending fails
• A dead-letter queue/topic for messages that still fail after retries
• Backoff retry
• Ability to rewind stream consumption for consistency/error recovery

---

6) DynamoDB OLTP design details (query model)

DynamoDB is described as partition-key/primary-key centric.

Access pattern implications

• Get by Order ID
  • Uses the partition key
• Direct “search by non-key attributes” (e.g., ad-hoc filters like a price range)
  • Not feasible in the described query model

Scalability and availability mechanisms mentioned

• Partitioning with 3-way replication
• Strong consistency reads by primary key
• Handling hot keys via an internal adaptive capacity concept
• Provisioning via read/write capacity units / query-per-second targets (pay-target based)

---

7) Supporting queries by Passenger ID: Global Secondary Index (GSI)

To support “ongoing order by Passenger ID”, the system uses a DynamoDB Global Secondary Index (GSI):

• Passenger ID becomes the partition key in the GSI
• Order ID is used as the sort key (per the described structure)

This provides an additional access pattern without changing the OLTP primary partition-key strategy.

The subtitles/description compare this “index” idea to concepts like a clone/view/materialized view.

---

8) Data retention and TTL considerations in DynamoDB

DynamoDB’s TTL exists, but using TTL broadly can cause heavy background scanning on large tables.

Their approach

• Apply TTL only to new items
• For old data that reaches expiry:
  • Perform manual deletion of expired records

Retention strategy

• Keep OLTP data for ~3 months
• Move historical data into OLAP (MySQL) for longer retention

---

9) Why MySQL on RDS for analytics (instead of Aurora)

For OLAP, Grab chooses MySQL on RDS because it is:

• Mature and sufficient for their needs
• Operationally easier/more established
• A practical cost/performance trade-off

(Aurora is mentioned as a possible alternative, but RDS is considered enough.)

---

10) Additional limitations and future directions

Current state

• The system is already in production and described as stable and scalable.

Limitations highlighted

• More complex queries (e.g., free-text search) are difficult with the current relational/OLAP approach.

Future direction

• Explore search-oriented NoSQL options such as Elasticsearch for those use cases.

---

Main speakers / sources (as identifiable from subtitles)

• PZN Reaction (implied video/channel identity from the title)
• The technical content appears to be based on a Grab engineering write-up/article (repeatedly referenced), but the original Grab authors are not clearly named in the subtitles.