“I’m 50th in line, but someone 200th got their ticket first — how?” This question has puzzled countless ticket buyers. The answer lies in the underlying design of the 12306 backup system: seat reuse, per-interval independent queuing, and event-driven matching. This article breaks down this world-class high-concurrency system across five dimensions — not just what it does, but why it works this way.
1. Why is 12306 So Hard to Build?
Before diving into the design, let’s understand the scale of the problem.
1.1 Business Complexity
Railway ticketing differs fundamentally from typical e-commerce flash sales:
Dimension
E-Commerce Flash Sale
12306 Ticketing
Product granularity
SKU level (one phone)
Interval level (Beijing→Jinan, same seat, different intervals)
Inventory management
Just deduct stock
Overlapping intervals, seat reuse
Concurrency contention
Multiple buyers for one SKU
Multiple overlapping intervals for the same seat
Order association
Independent orders
Linked orders (connecting trips, round trips)
The core difference: A single high-speed train from Beijing to Shanghai passes through 10 stations. The same seat can be split into 9 interval segments sold separately. This creates exponential complexity.
1.2 Scale Numbers
Take the 2024 Spring Festival travel rush as an example:
Daily page views: 10 billion+
Peak QPS: 1 million+ per second
Backup orders: daily peak of 20 million+
Train data: 5,000+ trains, each with 500–2,000 seats
At this scale, any design flaw gets amplified instantly into a system-wide failure.
2. Core Foundation: Seat Reuse
2.1 What is Seat Reuse?
Seat reuse means the same seat can be sold to different passengers in segments, as long as their travel intervals don’t overlap.
defcheck_interval_available(redis, train_id, date, seat_no, from_station, to_station): """ Check if a specific interval is available from_station: starting station index (e.g., Beijing=0) to_station: ending station index (e.g., Jinan=2) """ key = f"train:{train_id}:{date}:{seat_no}" # Check all bits from from_station to to_station-1 # i.e., interval [from_station, to_station) for i in range(from_station, to_station): bit = redis.getbit(key, i) if bit == 1: # This interval is already occupied returnFalse returnTrue
defallocate_interval(redis, train_id, date, seat_no, from_station, to_station): """ Allocate interval: set corresponding bits to 1 """ key = f"train:{train_id}:{date}:{seat_no}" for i in range(from_station, to_station): redis.setbit(key, i, 1) returnTrue
A more efficient approach using bitwise operations:
defcheck_and_allocate(redis, train_id, date, seat_no, from_station, to_station): """ Check and allocate interval in one bitwise operation """ key = f"train:{train_id}:{date}:{seat_no}" # 1. Get current bitmap value (represented as integer) current = redis.get(key) if current isNone: current = 0 # 2. Build interval mask # e.g., from=0, to=2, mask is 0b0011 (lower bits = earlier intervals) mask = (1 << to_station) - (1 << from_station) # 0b0011 # 3. Check if interval is already occupied if current & mask != 0: returnFalse# Interval is occupied # 4. Allocate interval (atomic operation, requires Lua script) # Lua script ensures atomicity of check + set lua_script = """ local current = tonumber(redis.call('GET', KEYS[1])) or 0 local mask = tonumber(ARGV[1]) if (current & mask) == 0 then redis.call('SET', KEYS[1], current | mask) return 1 else return 0 end """ result = redis.eval(lua_script, 1, key, mask) return result == 1
3. Distributed Queuing Mechanism
3.1 Core Design: Per-Interval Independent Queuing
Many people assume the backup queue is one big line for the entire train. It’s not.
12306’s design: each travel interval has its own independent queue.
1 2 3 4 5 6 7 8 9 10
Backup queues for Train G1 (Beijing → Qingdao):
Queue 1: Beijing → Tianjin (85 people in line) Queue 2: Beijing → Jinan (320 people in line) Queue 3: Tianjin → Jinan (56 people in line) Queue 4: Jinan → Qingdao (198 people in line) Queue 5: Beijing → Qingdao (1,275 people in line) ...
Each interval has its own Sorted Set
Why queue by interval?
Fairness: Beijing→Tianjin and Beijing→Qingdao aren’t the same tickets at all — mixing them in one queue makes no sense
Parallel processing: Different interval queues can be processed concurrently, increasing throughput
Precise matching: When a seat is released, the system locates the exact queue directly — no iteration needed
This answers a common question:
Why am I 50th in line but someone 200th got their ticket first?
Because you’re not in the same queue. They queued for Tianjin→Jinan; you queued for Beijing→Jinan.
3.2 Implementation: Redis Sorted Set
Data structure design:
1 2 3 4 5 6 7 8 9 10
Key: backup:{train_id}:{date}:{from_station}:{to_station} Value: Sorted set storing backup order IDs Score: Timestamp when user submitted the backup request (lower = earlier)
defget_queue_position(redis, train_id, date, from_station, to_station, order_id): """ Query queue position (ZRANK) Returns current position (0-based, +1 for actual rank) """ key = f"backup:{train_id}:{date}:{from_station}:{to_station}" rank = redis.zrank(key, order_id) if rank isNone: return-1# Not in queue return rank + 1# Actual rank (1-based)
defget_top_n_from_queue(redis, train_id, date, from_station, to_station, n=10): """ Get top N from queue (ZRANGE) """ key = f"backup:{train_id}:{date}:{from_station}:{to_station}" return redis.zrange(key, 0, n - 1)
defremove_from_queue(redis, train_id, date, from_station, to_station, order_id): """ Remove from queue on success or cancellation (ZREM) """ key = f"backup:{train_id}:{date}:{from_station}:{to_station}" redis.zrem(key, order_id) returnTrue
Performance analysis:
Operation
Complexity
Time for million-level queue
ZADD (join queue)
O(log N)
~21 comparisons, microsecond-level
ZRANK (query rank)
O(log N)
Microsecond-level
ZRANGE (get top N)
O(log N + M)
M = number retrieved, millisecond-level
ZREM (leave queue)
O(log N)
Microsecond-level
Sorted Set is a natural fit for queuing — it stays efficient even with millions of entries.
3.3 Data Persistence: MySQL Sharding
Redis is an in-memory database; a crash could lose data. Backup orders must be persisted to MySQL.
Sharding strategy:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Database shard key: user_id % 16 (16 databases by user) Table shard key: order_id % 128 (128 tables per database)
Table structure: CREATE TABLE backup_order_00 ( order_id BIGINT PRIMARY KEY, user_id BIGINT NOT NULL, train_id VARCHAR(20) NOT NULL, travel_date DATE NOT NULL, from_station VARCHAR(20) NOT NULL, to_station VARCHAR(20) NOT NULL, seat_type TINYINT NOT NULL, -- Seat type: business/first/second status TINYINT DEFAULT 0, -- Status: queuing/fulfilled/cancelled queue_position INT, -- Redundant storage of queue position create_time DATETIME, update_time DATETIME, INDEX idx_train_date (train_id, travel_date, from_station, to_station) );
How do Redis and MySQL stay consistent?
This is a classic distributed consistency problem, covered in detail later.
4. Event-Driven: Where Do Backup Tickets Come From?
Backup tickets don’t appear from nowhere — they all come from system events.
Step 3: Query backup queues for these intervals - Queue A: Beijing→Tianjin (85 people) - Queue B: Tianjin→Jinan (56 people) - Queue C: Beijing→Jinan (320 people) ← Exact match, process first
Step 4: Try to match Queue C's first person - Check if seat 12A meets the user's needs (seat type, availability) - Meets → execute fulfillment - Doesn't meet → try Queue C's second person
Step 5: If Queue C has no match, try other queues - Does Queue A + Queue B have the same seat? - This involves cross-interval matching, detailed later
4.4 Complete Fulfillment Flow
After a seat is released, the system must execute an atomic process to complete fulfillment:
The core design was covered above. What keeps the system rock-solid, though, is the meticulous handling of details.
6.1 Challenge 1: Distributed Consistency
Problem: How do you keep the Redis queue and MySQL orders in sync?
1 2 3 4 5 6 7 8 9 10 11
Scenario: User cancels backup
Approach 1: Update MySQL first, then delete Redis 1. MySQL DELETE succeeds 2. Redis ZREM fails (network blip) → Result: No order in MySQL, still queuing in Redis — "ghost queue"
Approach 2: Delete Redis first, then update MySQL 1. Redis ZREM succeeds 2. MySQL DELETE fails → Result: Redis deleted, MySQL still has it — user can't recover their spot
Solution: Local message table + eventual consistency
defcancel_backup_order(order_id): """Cancel backup order (ensures eventual consistency)""" # Step 1: Local transaction — write operation log + update order status with db.transaction(): # Mark order as "cancelling" db.execute("UPDATE backup_order SET status = 'CANCELING' WHERE order_id = ?", order_id) # Write operation log log_id = db.execute(""" INSERT INTO backup_operation_log (order_id, operation_type, redis_status, mysql_status) VALUES (?, 'CANCEL', 0, 0) """, order_id) # Step 2: Async process Redis operation mq.send('backup_operation', {'log_id': log_id, 'order_id': order_id, 'type': 'CANCEL'}) returnTrue
defprocess_cancel_message(message): """Process cancellation message""" log_id = message['log_id'] order_id = message['order_id'] try: # Step 3: Remove from Redis queue redis.zrem(f"backup:{train}:{date}:{from}:{to}", order_id) # Step 4: Update MySQL order status db.execute("UPDATE backup_order SET status = 'CANCELED' WHERE order_id = ?", order_id) # Step 5: Update operation log db.execute("UPDATE backup_operation_log SET redis_status=1, mysql_status=1 WHERE log_id=?", log_id) except Exception as e: # Retry on failure, alert after 3 attempts retry_count = db.query("SELECT retry_count FROM backup_operation_log WHERE log_id=?", log_id) if retry_count < 3: db.execute("UPDATE backup_operation_log SET retry_count = retry_count + 1 WHERE log_id=?", log_id) mq.send('backup_operation', message, delay=60) # Retry after 60 seconds else: alert(f"Backup cancellation failed, log_id={log_id}, error={e}")
Scheduled reconciliation task:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
defreconcile_backup_data(): """Reconcile every hour to ensure Redis and MySQL are consistent""" # Step 1: Scan all backup queues all_queues = redis.keys("backup:*") for queue_key in all_queues: order_ids = redis.zrange(queue_key, 0, -1) for order_id in order_ids: # Step 2: Check if order exists in MySQL order = db.query("SELECT * FROM backup_order WHERE order_id = ?", order_id) ifnot order or order.status == 'CANCELED': # Step 3: Exists in Redis but not in MySQL (or cancelled) — clean up Redis redis.zrem(queue_key, order_id) log.info(f"Cleaned ghost queue entry: {order_id}")
6.2 Challenge 2: Concurrent Contention and Overselling
Problem: When the same seat is released, how do multiple interval queues compete for it?
Without concurrency control: 1. Queue A assigns 12A to user X (Beijing→Tianjin) 2. Queue C assigns 12A to user Y (Beijing→Jinan) → X and Y's intervals overlap — oversold!
defallocate_seat_to_backup(train_id, date, seat_no, from_station, to_station, user_id): """ Allocate seat to backup user Core idea: lock all involved atomic intervals to prevent concurrent overselling """ # Step 1: Get all atomic intervals for this range (adjacent station pairs) atomic_intervals = get_atomic_intervals(train_id, from_station, to_station) # e.g., Beijing→Jinan = [(Beijing, Tianjin), (Tianjin, Jinan)] # Step 2: Generate lock keys (sorted lexicographically to prevent deadlock) lock_keys = sorted([ f"lock:{train_id}:{date}:{seat_no}:{f}:{t}" for f, t in atomic_intervals ]) # Step 3: Try to acquire all locks (SET NX PX with timeout) lock_acquired = [] for key in lock_keys: success = redis.set(key, user_id, nx=True, px=5000) # 5 second timeout ifnot success: # Failed to acquire — release already acquired locks for acquired_key in lock_acquired: redis.delete(acquired_key) returnFalse, "Seat is occupied" lock_acquired.append(key) try: # Step 4: Double-check seat availability ifnot check_seat_available(train_id, date, seat_no, from_station, to_station): returnFalse, "Seat unavailable" # Step 5: Execute fulfillment # 5.1 Mark seat as occupied mark_seat_occupied(train_id, date, seat_no, from_station, to_station, user_id) # 5.2 Create order (MySQL local transaction) order_id = create_ticket_order(user_id, train_id, date, seat_no, from_station, to_station) # 5.3 Freeze prepayment freeze_payment(user_id, order_id) # 5.4 Remove from backup queue redis.zrem(f"backup:{train_id}:{date}:{from_station}:{to_station}", user_id) # 5.5 Send notification send_notification(user_id, f"Backup successful! Order: {order_id}") returnTrue, order_id finally: # Step 6: Release locks for key in lock_acquired: # Use Lua script to ensure only releasing locks you own redis.eval(""" if redis.call("GET", KEYS[1]) == ARGV[1] then return redis.call("DEL", KEYS[1]) else return 0 end """, 1, key, user_id)
Key points:
Lock granularity: Lock at the “atomic interval” level (between adjacent stations), not the entire seat
Lock ordering: Acquire locks in lexicographic order to prevent deadlock (A waits for B, B waits for A)
Double-check: Re-check availability after acquiring the lock to prevent concurrent bypass
Atomic release: Use Lua scripts to ensure you only release locks you own
6.3 Challenge 3: Cross-Interval Matching
Problem: User backs up a “long interval,” but only “short intervals” are released — how to match?
1 2 3 4 5 6 7 8 9 10 11
Scenario: User backup: Beijing → Qingdao (full route)
Current releases: - Zhang cancels: Beijing → Tianjin (first segment only) - Li cancels: Tianjin → Jinan (second segment only) - Wang cancels: Jinan → Qingdao (last two segments)
Can these match? - If all three release simultaneously, they could be combined for the user - In reality, cancellations are discrete events — hard to assemble all intervals
Approach 1: No cross-interval matching (simple, 12306’s current approach)
1 2 3 4 5 6 7 8 9 10 11
Rules: - User backing up Beijing→Qingdao needs a complete Beijing→Qingdao interval to be free - Cannot be assembled from Beijing→Tianjin + Tianjin→Qingdao
defsmart_match_backup(train_id, date, from_station, to_station, user_id): """ Smart matching: try to combine multiple short intervals to satisfy a long interval Core logic: iterate all seats, find one that's free across all intervals """ # Step 1: Get all atomic intervals atomic_intervals = get_atomic_intervals(train_id, from_station, to_station) # e.g., Beijing→Qingdao = [Beijing→Tianjin, Tianjin→Jinan, Jinan→Zibo, Zibo→Qingdao] # Step 2: Get all seats for this train all_seats = get_all_seats(train_id, date, seat_type) # e.g., ['12A', '12B', '12C', '13A', '13B', ...] # Step 3: Check each seat for availability across all intervals for seat_no in all_seats: bitmap_key = f"train:{train_id}:{date}:{seat_no}" # Check if this seat is free in all atomic intervals all_available = True for i, (f, t) in enumerate(atomic_intervals): station_idx = get_station_index(f) bit = redis.getbit(bitmap_key, station_idx) if bit == 1: # This interval is occupied all_available = False break if all_available: # Found a seat free across all intervals return seat_no # Step 4: No complete match found # Optional: record missing intervals, wait for future releases returnNone
defon_seat_released(train_id, date, seat_no, from_station, to_station): """ Triggered by seat release event """ # Query all potentially matching backup queues # Includes: exact match + users waiting for cross-interval match related_queues = find_related_backup_queues(train_id, date, from_station, to_station) for queue_key in related_queues: # Get top 10 from queue, try to match candidates = redis.zrange(queue_key, 0, 9) for user_id in candidates: # Get this user's backup request request = get_backup_request(user_id) # Try to match a seat for this user seat = smart_match_backup( train_id, date, request.from_station, request.to_station, user_id ) if seat: allocate_seat(seat, request.from_station, request.to_station, user_id) break# Seat assigned, exit this queue
Challenges of cross-interval matching:
Challenge
Details
Complexity spike
Need to maintain a “missing interval index,” query efficiency drops
More complex concurrency
Multiple partially-matching users compete for the same interval
Unpredictable UX
With partial matching, users don’t know which queue they’re in
Practical trade-off: 12306 currently uses the “no cross-interval matching” strategy, sacrificing some user experience for system simplicity.
6.4 Challenge 4: Hotspot Skew
Problem: During Spring Festival rush, a single interval queue can hit millions.
1 2 3 4 5 6 7 8 9 10 11 12
Scenario: Week before Spring Festival, Beijing→Harbin, backup queue has 2 million people
Sorted Set performance: - ZADD: O(log N) = log(2,000,000) ≈ 21 operations - ZRANK: O(log N) ≈ 21 operations - Single operations are microsecond-level — not the problem
The issue: 1. Frequent ZRANK queries for queue position — hot key 2. Frequent ZRANGE to get queue head — hot key 3. Redis is single-threaded — single key becomes the bottleneck
# Approach 1: Queue sharding defget_shard_key(train_id, date, from_station, to_station, user_id): """ Split one large queue into multiple smaller queues Shard by user_id, ensuring same user always lands in same shard """ shard_count = 32 shard_id = hash(user_id) % shard_count returnf"backup:{train_id}:{date}:{from_station}:{to_station}:shard:{shard_id}"
defadd_backup_order_with_fallback(train_id, date, from_station, to_station, user_id): """Join backup queue (with fallback)""" try: # Try writing to Redis redis.zadd(f"backup:{train_id}:{date}:{from_station}:{to_station}", {user_id: time.time()}) returnTrue, "Queued successfully" except RedisError as e: # Redis failure — fall back to MQ log.error(f"Redis failure, falling back to MQ: {e}") # Write to MQ mq.send('backup_queue_pending', { 'train_id': train_id, 'date': date, 'from': from_station, 'to': to_station, 'user_id': user_id, 'timestamp': time.time() }) # Write to MySQL (status=PENDING, awaiting recovery) db.execute(""" INSERT INTO backup_order (user_id, train_id, travel_date, from_station, to_station, status) VALUES (?, ?, ?, ?, ?, 'PENDING') """, user_id, train_id, date, from_station, to_station) returnTrue, "Queued successfully. System busy — position available later"
defrecover_redis_from_mq(): """ Background task: restore queue data from MQ after Redis recovers Key: use idempotent operations to prevent duplicate additions """ messages = mq.consume('backup_queue_pending', batch_size=100) for msg in messages: try: queue_key = f"backup:{msg['train_id']}:{msg['date']}:{msg['from']}:{msg['to']}" # Idempotency check: verify user isn't already in queue existing_score = redis.zscore(queue_key, msg['user_id']) if existing_score isnotNone: # User already in queue — skip (preserve original Score, don't update) log.info(f"User {msg['user_id']} already in queue, skipping recovery") else: # User not in queue — add them redis.zadd(queue_key, {msg['user_id']: msg['timestamp']}) # Update MySQL status to QUEUED db.execute(""" UPDATE backup_order SET status = 'QUEUED' WHERE user_id = ? AND train_id = ? AND travel_date = ? """, msg['user_id'], msg['train_id'], msg['date']) except RedisError as e: log.error(f"Redis still not recovered, will retry: {e}") break# Redis still unavailable — stop and wait for next recovery
Key points in the degradation flow:
Key point
Details
Idempotent recovery
Check if user is already in queue before adding — avoids duplicates or Score overwrites
State tracking
MySQL tracks PENDING→QUEUED status transitions for monitoring
Batch consumption
Consume 100 at a time — one failure doesn’t block the entire batch
Graceful exit
Break when Redis is still down — unprocessed messages stay in MQ for next recovery
7. Summary and Reflections
7.1 Design Essentials of the 12306 Backup System
Design point
Core idea
Implementation
Seat reuse
Sell same seat in segments to maximize utilization
Redis Bitmap
Distributed queuing
Per-interval independent queues — fair and precise
Redis Sorted Set
Event-driven
Four ticket sources trigger automatic fulfillment
Kafka + consumer pattern
High-concurrency architecture
Four-layer defense + rate limiting + degradation
Gateway + Caffeine + Redis + MySQL
Consistency guarantee
Local message table + eventual consistency
MySQL transaction + MQ
Concurrency control
Distributed lock + double-check
Redis SET NX + Lua
Disaster recovery
MQ buffering + idempotent recovery
Kafka + scheduled reconciliation
7.2 Why Official Backup Beats Third-Party Tools
Dimension
Official backup
Third-party ticket scalpers
Data source
Direct access to seat data, no delay
Polls 12306 API, has latency
Queue fairness
Timestamp-sorted, first-come-first-served
Cannot access real queue data
Ticket coverage
All four sources including dynamic additions
Can only monitor some sources
System stability
HA architecture + degradation
Gets rate-limited and blocked by 12306
7.3 Architecture Principles Worth Borrowing
Data structure choice sets the system ceiling: Bitmap and Sorted Set keep million-level data efficient
Divide and conquer solves scale problems: Independent per-interval queuing breaks the global problem into local ones
Event-driven decouples complex logic: Refunds, timeouts, reschedules, additions — all unified as events
Four-layer defense absorbs high concurrency: Gateway rate limiting → local cache → Redis → MySQL, pressure drops at each level
Eventual consistency beats strong consistency: In distributed systems, eventual consistency is easier to implement and provides better user experience
Idempotent design prevents duplicate operations: Check before adding during degraded recovery to avoid data corruption