Mahjong AI (HK Taiwan Style) - Comprehensive Research & Plan

1. Objective

To develop a web application capable of identifying Mahjong tiles (HK Style Taiwan 16-tile variant) from user-uploaded photos and accurately calculating the "Fan" score according to local rules.

2. Vision System: Technology Comparison

Criteria	YOLO (v10/v11)	Multimodal LLM (GPT-4o)
Accuracy	Very High (Deterministic)	High, but prone to counting errors
Latency	<100ms (Local)	2-5s (API)
Cost	Zero (Client-side)	Recurring (Per-call)
Effort	High (Data Tagging required)	Low (Immediate Start)

🛡️ Hybrid Roadmap Strategy:

Phase 1 (Prototype): Launch with GPT-4o-vision to enable immediate photo upload functionality.
Phase 2 (Data Collection): Use prototype images to train a custom YOLO model.
Phase 3 (Production): Deploy client-side YOLO for 100% accuracy and zero operational costs.

3. Scoring Logic Engine

Constraint: LLMs (GPT/Claude) are probabilistic and often fail at the recursive math required for 16-tile Mahjong hands.

Solution: Implement a deterministic Python Rule Engine using a shanten algorithm modified for HKTW rules. The LLM will serve only as the "Translator" that explains the resulting Fan breakdown to the user.

4. Test & Quality Assurance Plan

A. Recognition Validation

Synthetic Augmentation: Use 3D renders of tiles to simulate different lighting conditions.
Counting Test: Validate that the system identifies exactly 16 or 17 tiles (including the winning tile) in every test hand.

B. Calculation Validation

The "Gold Standard" Suite: A library of 200+ edge-case hands (e.g., 嚦咕嚦咕, 十六不搭).
Cross-Verification: Automated comparison against existing mobile Fan calculators.

C. Data Sourcing

Public Datasets: Roboflow Universe (Mahjong sets).
Authentic Rules: Cross-referencing HK01 and Wikipedia's 港式台灣麻雀 guides.

5. Next Steps

Initialize basic Web App shell (React/Vue).
Implement GPT-4o-vision API wrapper for initial testing.
Begin drafting the shanten-based scoring logic in Python.

Mahjong AI: Research & Strategy