Scheduler Deployment

1. Prerequisites

(1) Make sure ZooKeeper has been deployed

(2) Make sure PostgreSQL has been deployed

(3) Make sure RabbitMQ has been deployed

(4) Create the directory required by the resource center

- 🔗 [Baidu Netdisk](https://pan.baidu.com/s/5A7-TUZ_EujpsWO93RektIg)

# Create directory 
sudo mkdir -p /dolphinscheduler

Note

When installing Apache DolphinScheduler, if you do not want to use the default resource center base path (dolphinscheduler), manually edit the configuration files:

Extract the installation package:

tar -xzvf apache-dolphinscheduler-3.2.2-bin.tar.gz

Edit the parameters in the following two files:

• api-server/conf/common.properties
• worker-server/conf/common.properties

Change the configuration item resource.storage.upload.base.path to a custom path for it to take effect.

2. Configure Password-free User Access and Permissions

Create a deployment user and make sure password-free sudo is configured. The following example creates the dolphinscheduler user.

# Create the user while logged in as root
sudo useradd dolphinscheduler

# Add a password. The "dolphinscheduler" text below is the password. On Kylin, change it to a complex password that meets Kylin password requirements.
echo 'dolphinscheduler:Str0ng@Pass!' | sudo chpasswd

# Configure password-free sudo
sudo sed -i '$adolphinscheduler  ALL=(ALL)  NOPASSWD: NOPASSWD: ALL' /etc/sudoers
sudo sed -i 's/Defaults    requiretty/#Defaults    requiretty/g' /etc/sudoers

# Modify directory permissions so the deployment user can operate the extracted apache-dolphinscheduler-*-bin directory
sudo chown -R dolphinscheduler:dolphinscheduler apache-dolphinscheduler-*-bin
sudo chmod -R 755 apache-dolphinscheduler-*-bin
sudo chown -R dolphinscheduler:dolphinscheduler /dolphinscheduler
sudo chown -R dolphinscheduler:dolphinscheduler /tmp/dolphinscheduler

3. Edit the dolphinscheduler_env.sh File

The ./bin/env/dolphinscheduler_env.sh file describes the following configuration:
DolphinScheduler database configuration. For detailed configuration information, see [Database and Middleware Deployment]
External dependency paths or library files for some task types, such as JAVA_HOME and SPARK_HOME, are defined here.

# JAVA_HOME
export JAVA_HOME=/opt/java/openjdk

# Database configuration
export DATABASE=postgresql
export SPRING_PROFILES_ACTIVE=postgresql
export SPRING_DATASOURCE_URL="jdbc:postgresql://127.0.0.1:5432/dolphinscheduler"
export SPRING_DATASOURCE_USERNAME=root
export SPRING_DATASOURCE_PASSWORD=Idgf~V4pco&PF


# Registry center
export REGISTRY_TYPE=zookeeper
export REGISTRY_ZOOKEEPER_CONNECT_STRING=localhost:2181

# Component paths
export SPARK_HOME=/opt/soft/spark

export PATH=$SPARK_HOME/bin:$JAVA_HOME/bin:$PATH

    Note: If the dolphinscheduler_env.sh file was edited by a user other than dolphinscheduler, run the following commands again.

sudo chown -R dolphinscheduler:dolphinscheduler apache-dolphinscheduler-*-bin
sudo chmod -R 755 apache-dolphinscheduler-*-bin

4. Start

# Run the following commands after changing to the extracted binary directory
# Switch to the dolphinscheduler user created in step 2
su dolphinscheduler
# Start and stop Master
bash ./bin/dolphinscheduler-daemon.sh start master-server
bash ./bin/dolphinscheduler-daemon.sh stop master-server

# Start and stop Worker. To deploy multiple servers, repeat from step 1 on each server; in this startup step, only start Worker.
bash ./bin/dolphinscheduler-daemon.sh start worker-server
bash ./bin/dolphinscheduler-daemon.sh stop worker-server

# Start and stop Api
bash ./bin/dolphinscheduler-daemon.sh start api-server
bash ./bin/dolphinscheduler-daemon.sh stop api-server

# Start and stop Alert
bash ./bin/dolphinscheduler-daemon.sh start alert-server
bash ./bin/dolphinscheduler-daemon.sh stop alert-server

5. Access

Access address: http://server-IP:12345/dolphinscheduler/ui 
Account: admin
Password: dolphinscheduler123

qdata-etl/target/qdata-etl-3.8.8.jar

6. Add the jar Package to the Resource Center

Upload qdata-etl/target/qdata-etl-3.8.8.jar from Application Build to the spark-jar directory in the resource center.