Docker Compose Deployment

Windows users

This is the Linux / macOS version of the Docker Compose deployment guide.
If you use Windows, see 👉 Docker Compose Deployment (Windows).

Before You Start

Who should read this guide?

• People trying qData for the first time who want to experience the features quickly
• People who do not want to wrestle with complex environment configuration or frontend packaging
• Users who want to experience the complete platform with a simple "one-click startup" flow

Before you continue

• This guide is intended for users with basic Docker and Docker Compose knowledge. You should be able to understand common commands, container status, and basic error messages.
• Deployment requires an internet connection to download Docker images. Make sure the deployment machine can access the image registries over a stable network.
• In mainland China, configure Docker with an available domestic registry mirror in advance. Use an acceleration service provided by your cloud provider or organization, and verify that Docker can pull images before deployment.

Docker images support two architectures

The Docker images used by this deployment package support both amd64 (x86_64) and arm64. When pulling images, Docker automatically detects the host CPU architecture and selects the matching image. You do not need to switch image tags manually.

We have prepared a complete ready-to-use deployment package for you, including:

• ✅ Frontend static assets (the packaged dist folder)
• ✅ DolphinScheduler scheduler (no extra installation required)
• ✅ Complete Hadoop stack (HDFS + YARN)
• ✅ Spark 3.5.6 (already extracted)
• ✅ Preset .env environment variable file
• ✅ Complete directory structure and Docker Compose configuration

There is no need to manually install a pile of dependencies, build the frontend, or handle complex configuration. After extracting the deployment package, run the Docker Compose commands. The required images will be downloaded over the network before the big data platform and qData start.

---

Step 1: Install Docker and Docker Compose

qData runs in Docker containers, so Docker and Docker Compose must be installed first.

• Recommended versions:
  • Docker: ≥ v19.03
  • Docker Compose: ≥ v2.20.2

docker compose and docker-compose

• docker compose is the recommended Docker Compose V2 command.
• docker-compose is the legacy standalone Docker Compose V1 command.
• This guide uses docker compose consistently. If your environment only provides docker-compose, you can substitute it for docker compose, but upgrading to Compose V2 is recommended.

🖥️ Windows / macOS

Download and install Docker Desktop directly.
After installation, Docker Compose is included automatically and no extra installation is required.

🐧 Linux (Ubuntu 18.04+)

The deployment package already includes offline installation packages and an installation script, so you can run them directly:

# Enter the Docker installation package directory
cd ~/qData/docker-install

# Install Docker components
sudo dpkg -i containerd.io_1.6.9-1_amd64.deb docker-ce_24.0.7-1~ubuntu.20.04~focal_amd64.deb docker-ce-cli_24.0.7-1~ubuntu.20.04~focal_amd64.deb docker-compose-plugin_2.6.0~ubuntu-focal_amd64.deb

# Add the current user to the docker group (sudo-free usage is optional)
sudo usermod -aG docker $USER
newgrp docker

# Verify the installation
sudo docker -v
# Example output: Docker version 24.0.7, build afdd5

Install Docker Compose:

# Grant execute permission and install it as a Docker CLI plugin
sudo chmod +x docker-compose
sudo mkdir -p /usr/local/lib/docker/cli-plugins
sudo mv docker-compose /usr/local/lib/docker/cli-plugins/docker-compose

# Verify the installation
sudo docker compose version
# Example output: Docker Compose version v2.20.2

Step 2: Deployment Package Download Address

🔗 Baidu Netdisk: https://pan.baidu.com/s/1sKgeBnMZmRnIr42_pVHmiw

🔑 Extraction code: Join the QQ discussion group and check the group files.👉 Click here to join the QQ discussion group

After downloading, extract the deployment package

---

Step 3: Start qData

Extracting the deployment package produces a docker folder directly.
Enter the docker folder, which contains multiple .yml files, before running the following commands.

3.1 Use MySQL as the Primary Database

⚠️ Tip: If you use Dameng Database (DM8), skip this step.

---

Step 1: Enter the installation directory

cd ~/(deployment-package-extraction-path)

Step 2: Edit the .env file

sudo vi .env

Find the following configuration and change dm8 to mysql:

# Database type. Available values: dm8 or mysql
DB_TYPE=mysql

3.2 Initialize the Database (required on first run)

sudo docker compose --profile schema up -d

💡 If you use **MySQL** as the primary database, run:
sudo docker compose -f docker-compose-mysql.yml --profile schema up -d

⚠️ Note: entrypoint.sh file permission issue
If the first run of sudo docker compose --profile schema up -d reports permission or Windows line-ending issues, run the following commands in order:

# Grant script permissions and fix line endings
cd ~/qData(deployment-package-extraction-path)
sudo chmod -R 755 docker
sudo chown -R $USER:$USER ./docker

sed -i 's/\r$//' ./database/dm8/entrypoint.sh
sed -i 's/\r$//' ./database/dm8/entrypoint-arm64.sh

# Verify permissions
ls -l ./database/dm8/entrypoint.sh
# Expected: -rwxr-xr-x

# Run again
sudo docker compose --profile schema up -d

💡 If you use **MySQL** as the primary database, run:
sudo docker compose -f docker-compose-mysql.yml --profile schema up -d

3.3 Start qData

Choose lightweight mode or full mode according to your verification needs.

Lightweight Mode (light)

For quick deployment and feature verification, you can start qData without DolphinScheduler and Spark:

sudo docker compose --profile light up -d

💡 If you use **MySQL** as the primary database, run:
sudo docker compose -f docker-compose-mysql.yml --profile light up -d

Full Mode (all)

To use all dependency services, continue to use the all mode:

sudo docker compose --profile all up -d

💡 If you use **MySQL** as the primary database, run:
sudo docker compose -f docker-compose-mysql.yml --profile all up -d

3.4 Start Demo Data (Optional)

To verify qData with the built-in data source, start the demo mode separately. The demo mode is independent of all:

sudo docker compose --profile demo up -d

💡 If you use **MySQL** as the primary database, run:
sudo docker compose -f docker-compose-mysql.yml --profile demo up -d

3.5 Start Local Source-code Dependencies

# Steps 4.1 and 4.2 above do not need to be run
docker compose --profile local up -d

💡 If you use **MySQL** as the primary database, run:
sudo docker compose -f docker-compose-mysql.yml --profile all up -d (to be adapted)

⚠️ Note: port occupation issue
If sudo docker compose --profile all up -d reports that a port is occupied:

• Follow the error message and stop the process occupying the corresponding port on the server;
• Or edit the *.yaml configuration files in the deployment package and comment out or delete port mappings other than port 80 (this does not affect container-internal operation).

5.5 Other Common Commands

⚠️ Tip:
The following commands use all as the example. If you started lightweight mode, replace all with light. Manage the demo mode separately.

If you use a custom docker-compose-xxx.yml file, every startup command must append
-f docker-compose-xxx.yml after docker compose.

Example: sudo docker compose -f docker-compose-mysql.yml --profile all up -d

View the running status of all services:

sudo docker compose --profile all ps

Stop all qData services temporarily (data is retained):

sudo docker compose --profile all stop

Restart all services:

sudo docker compose --profile all restart

Completely stop services and delete data (restore the default environment):

sudo docker compose --profile all down
sudo docker compose --profile schema down

Middleware required for local source-code startup

docker compose --profile local up -d

⚠️ down clears data. If you want to keep the data, use stop only.

---

Deployment Complete! 🎉

After deployment is complete, access the following addresses:

• 🌐 qData Data Platform
  http://<server-IP>:80
  Account: admin
  Password: qData123

• 📅 Scheduler DolphinScheduler
  http://<server-IP>:12345/dolphinscheduler/ui/home
  Account: admin
  Password: dolphinscheduler123

⚠️ DolphinScheduler starts only in all mode.

🔑 Replace <server-IP> with your server's public or private IP.

💡 Important Reminder

• After the first startup, we recommend waiting 1-2 minutes so all containers can finish initialization.
• You can run docker ps to view container status.
• If the service is inaccessible, check whether the server firewall has opened port 80.