Deployment Server Resource Specification Reference
6/11/26About 2 minInstallation and DeploymentServerMinimum ConfigurationRecommended Configuration
To ensure stable operation and good performance of the qData Data Platform system, the following lists the minimum configuration and recommended configuration server resources required for deployment. Choose the appropriate plan based on the actual project scale and budget.
Minimum Configuration Requirements
Suitable for small and medium-sized business scenarios or test environments. Servers can be consolidated based on the actual situation.
| No. | Server Name | Purpose Description | Operating System | CPU Specification | Memory Specification | Disk Configuration | Quantity |
|---|---|---|---|---|---|---|---|
| 1 | Data platform application server | Deploy qData frontend and backend services, including portal, task management, data resource management, and other modules. | Ubuntu / Kylin | 4 | 8 | 256G | 1 |
| 2 | DS scheduler | Deploy DolphinScheduler frontend and backend services (pseudo-distributed), while also hosting Redis and RabbitMQ middleware. | Ubuntu / Kylin | 8 | 16 | 512G | 1 |
| 3 | Spark execution engine | Used to run Spark computing tasks and support data development, ETL scheduling, and offline computing. | Ubuntu / Kylin | 16 | 64 | 256G | 1 |
Important Reminder
- The data platform application server and DS scheduler can be consolidated on one 8-core, 32G server. A common low-cost server is sufficient.
Recommended Configuration Requirements
Suitable for production environments and large-scale business needs. A distributed high-availability architecture is recommended to improve system stability and scalability.
| No. | Server Name | Purpose Description | Operating System | CPU Specification | Memory Specification | Disk Configuration | Quantity |
|---|---|---|---|---|---|---|---|
| 1 | Data platform application server | Deploy qData frontend and backend services, portal, task management, data resource management, and other modules. | Ubuntu / Kylin | 8 | 16G | 256G SSD | 1 |
| 2 | DS scheduler Master node | Deploy the DolphinScheduler Master node, responsible for coordinating scheduled tasks.[Recommended: use 2 nodes in HA mode (managed by ZK); currently single-node] | Ubuntu / Kylin | 8 | 16G | 512G SSD | 1 |
| 3 | DS scheduler Worker node | Deploy DolphinScheduler Worker nodes to execute specific tasks.Elastically scalable. | Ubuntu / Kylin | 16 | 32G | 256G SSD | 2 |
| 4 | Middleware server | Deploy Redis, RabbitMQ, and other middleware. | Ubuntu / Kylin | 4 | 16G | 256G SSD | 1 |
| 5 | Spark Master node | Spark control node.[Recommended: use 2 nodes in HA mode (managed by ZK); currently single-node] | Ubuntu / Kylin | 8 | 16G | 512G SSD | 1 |
| 6 | Spark Worker node | Spark worker node, used for actual computing task execution.Elastically scalable. | Ubuntu / Kylin | 16 | 64G | 256G SSD | 2 |
| 7 | Data service gateway / API gateway | Provides unified data service access, permission control, rate limiting, and other functions.Elastically scalable. | Ubuntu / Kylin | 4 | 8G | 128G SSD | 2 |
Recommendations
- SSD disks are recommended for all nodes to improve read/write performance;
- Appropriately expand the number of Spark and DS Worker nodes based on actual project concurrency and data processing scale;
- Ubuntu and Kylin systems are both well supported. Stable versions are recommended;
- All servers are recommended to be deployed in an intranet environment with unified permission and security control.