Start Locally from Source Code

Tutorial Video

Complete tutorial for building the qData Data Platform open-source local development environment, IDEA + Docker Compose

Complete tutorial for building the qData Data Platform open-source local development environment, IDEA + Docker Compose

Important

If you encounter environment configuration or task execution exceptions during deployment, we recommend watching the official qData Data Platform troubleshooting video collection.
This video series explains common troubleshooting methods and covers typical scenarios such as task failure, unable to add or execute tasks, and no task instances, helping you quickly locate and resolve issues.

• Watch the troubleshooting video collection

This document explains how to start qData locally using the source code method. Follow the steps below to prepare the environment, configure services, and start the application.

---

1. Prerequisites

1.1. Install Docker and Docker Compose

Before installing qData, make sure your device meets the following minimum requirements. We recommend installing Docker-related components on a Linux server or virtual machine:

• CPU: >= 4 cores
• Memory (RAM): >= 8 GiB

⚠️ Note: Make sure Docker and Docker Compose are correctly installed and can be executed normally in the terminal.

2. Version Requirements

Component	Version Requirement
JDK	1.8
Maven	3.6+
yarn	v1.22.22+
Node.js	18.0+
Docker	19.03 or later
Docker Compose	1.28 or later

3. Clone the qData Repository

git clone https://gitee.com/qiantongtech/qData.git

4. Start Middleware with Docker Compose

The qData backend depends on the following middleware:

• Databases: DM8 / MySQL, PostgreSQL
• Cache and message queue: Redis, RabbitMQ
• Files and storage: MongoDB, Hadoop, kkFileView
• Scheduling: DolphinScheduler

Start middleware through Docker Compose:

# Local source-code mode in the Docker Compose deployment document
docker-compose --profile local up -d

👉 See Docker Compose deployment documentation

5. Start the Backend

5.1. Environment Preparation

• JDK = 1.8
• Maven >= 3.6

5.2. Start the Project in IDEA

Open the project with IntelliJ IDEA.

5.3. Build the Project

mvn clean -DskipTests=true install

5.4. Required Configuration

All of the following configuration items must be modified in the application-dev.yml file.

5.4.1. qdata-server Configuration

• Modify the qData service Redis connection information by editing qdata-server/src/main/resources/application-dev.yml.

spring:
  redis:
    host: IP
    port: PORT
    database: 0
    password: PASSWORD

• Modify RabbitMQ connection information by editing qdata-server/src/main/resources/application-dev.yml.

spring:
  rabbitmq:
    host: IP
    port: PORT
    username: ACCOUNT
    password: PASSWORD

• Modify database connection information by editing qdata-server/src/main/resources/application-dev.yml.

# Primary data source selection
datasource:
  type: mysql # mysql or dm8. Fill in the selected type and modify the corresponding settings below.

# MySQL configuration
mysql:
  # JDBC driver class name
  driver-class-name: com.mysql.cj.jdbc.Driver
  # Primary database JDBC connection URL
  url: jdbc:mysql://IP:PORT/qdata?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8
  # Primary database username
  username: ACCOUNT
  # Primary database password
  password: PASSWORD

# DM configuration
dm8:
  # JDBC driver class name
  driver-class-name: dm.jdbc.driver.DmDriver
  # Primary database JDBC connection URL
  url: jdbc:dm://IP:PORT/QDATA?STU&zeroDateTimeBehavior=convertToNull&useUnicode=true&characterEncoding=utf-8&schema=QDATA&serverTimezone=Asia/Shanghai
  # Primary database username
  username: ACCOUNT
  # Primary database password
  password: PASSWORD

• Modify scheduler-related configuration by editing qdata-server/src/main/resources/application-dev.yml.

# Scheduler-related configuration
ds:
  # Scheduler API prefix
  base_url: http://IP:12345/dolphinscheduler
  # Data quality service component HTTP callback address
  quality_url: http://127.0.0.1:8083/quality/qualityTaskExecutor/runExecuteTask
  # DolphinScheduler project code for data quality tasks
  http_quality_projectCode: 134799536571008
  # Scheduler token, which can be changed as needed
  token: b07f0e57c4818043a57ba05a28da291a
  # Spark-related configuration
  spark:
    # Spark master address
    master_url: spark://spark:7077
    # JAR path in DolphinScheduler
    main_jar: file:/dolphinscheduler/default/resources/spark-jar/qdata-etl-3.8.8.jar
    # JAR main entry class
    main_class: tech.qiantong.qdata.spark.etl.EtlApplication
  # HDFS-related configuration
  hdfs:
    # NameNode RPC address, the entry address for client connections to HDFS
    url: hdfs://namenode:8020
  # Scheduler Redis configuration, used to obtain the latest data source information and ensure task execution
  redis:
    host: redis
    port: 6379
    database: 0
    password: PASSWORD

• Modify data quality service configuration by editing qdata-server/src/main/resources/application-dev.yml.

path:
  # Data quality service prefix
  quality_url: http://127.0.0.1:8083/quality/qualityTaskExecutor

5.4.2. qdata-quality Configuration

• Modify RabbitMQ connection information by editing qdata-quality/src/main/resources/application-dev.yml.

spring:
  rabbitmq:
    host: IP
    port: PORT
    username: ACCOUNT
    password: PASSWORD

• Modify MongoDB connection information by editing qdata-quality/src/main/resources/application-dev.yml.

spring:
  data:
    mongodb:
      # Connection address
      uri: mongodb://ACCOUNT:PASSWORD@IP:PORT/data?authSource=admin
      field-naming-strategy: org.springframework.data.mapping.model.SnakeCaseFieldNamingStrategy
      print: true
      slowQuery: true
      slowTime: 1000

• Modify database connection information by editing qdata-quality/src/main/resources/application-dev.yml.

# Primary data source selection
datasource:
  type: mysql # mysql or dm8. Fill in the selected type and modify the corresponding settings below.

# MySQL configuration
mysql:
  # JDBC driver class name
  driver-class-name: com.mysql.cj.jdbc.Driver
  # Primary database JDBC connection URL
  url: jdbc:mysql://IP:PORT/qdata?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8
  # Primary database username
  username: ACCOUNT
  # Primary database password
  password: PASSWORD

# DM configuration
dm8:
  # JDBC driver class name
  driver-class-name: dm.jdbc.driver.DmDriver
  # Primary database JDBC connection URL
  url: jdbc:dm://IP:PORT/QDATA?STU&zeroDateTimeBehavior=convertToNull&useUnicode=true&characterEncoding=utf-8&schema=QDATA&serverTimezone=Asia/Shanghai
  # Primary database username
  username: ACCOUNT
  # Primary database password
  password: PASSWORD

• Modify RabbitMQ connection information by editing qdata-quality/src/main/resources/application-dev.yml.

spring:
  rabbitmq:
    host: IP
    port: PORT
    username: ACCOUNT
    password: PASSWORD

5.4.3. Start the qdata-server Service

Run qdata-server/src/main/java/tech/qiantong/qdata/server/QDataApplication.java

5.4.4. Start the qdata-quality Service

Run qdata-quality/src/main/java/tech/qiantong/qdata/quality/QualityApplication.java

6. Start the Frontend

6.1. Environment Preparation

• Node.js >= 18.0
• yarn >= v1.22.22

6.2. Start the Frontend Service

# Enter the frontend directory
cd qdata-ui
# Install dependencies
yarn install
# Start
yarn run dev

6.3. Reset the admin Password

# Execute the following SQL in the database
UPDATE SYSTEM_USER
SET PASSWORD='$2a$10$3dzj0nGSmlNjPw4iuZNHse5.lxrNzumTCQ8aEaV7nNEtn2pXmmimW'
WHERE USER_ID=1;

6.4. Access qData

Open http://localhost:81 in your browser.
Default account/password: admin/qData123

If the login page displays correctly, login succeeds, and menus and pages display normally, the environment has been set up successfully.