Source Code Guide

Qiantong Tech8/13/26About 8 minSource StructureCore AdvantagesSource CodeStructureSecondary Development

qData Source Code Structure

Who should read this guide?

Developers who are new to the qData source code and want to understand the project structure quickly
Developers preparing to build extensions or custom features on qData
Developers who need to locate code quickly from a page, API, or database table

Treat the current branch's pom.xml and actual source code as the authority for module lists, versions, and dependencies.

1. Top-level directories

qData/
├── pom.xml                    # Maven aggregator and unified version management
├── qdata-server/              # Main service startup module
├── qdata-framework/           # Shared security, data access, cache, file, and scheduling capabilities
├── qdata-module-system/       # Users, roles, menus, dictionaries, and logs
├── qdata-module-att/          # Projects, categories, tags, topics, and rules
├── qdata-module-da/           # Data sources, assets, discovery, and asset operations
├── qdata-module-dm/           # Data domains, business categories, warehouse layers, and subject areas
├── qdata-module-dp/           # Data elements, code mappings, logical models, and standards
├── qdata-module-dpp/          # Data integration, development, scheduling, and quality orchestration
├── qdata-module-ds/           # API data services, authentication, and call logs
├── qdata-module-mc/           # Metadata collection tasks, instances, and versions
├── qdata-module-dg/           # Data classification, sensitive-data detection, and desensitization governance
├── qdata-module-ai/           # AI model, conversation, and message domain data and management APIs
├── qdata-api-ds/              # DolphinScheduler contracts and adapter implementation
├── qdata-executor-etl/        # Spark / DataX data processing programs
├── qdata-service-quality/     # Independent data-quality execution service
├── qdata-service-ai/          # Independent intelligent data-query Maven project
├── qdata-ui/                  # Vue 3 frontend project
├── sql/                       # MySQL, DM, and DolphinScheduler SQL scripts
├── docker/                    # Compose, Nginx, middleware, and service deployment configuration
├── docs/                      # Multilingual documentation maintained with the repository
└── images/                    # README and documentation images

Architectural advantages

Separated frontend and backend: qdata-ui is built independently and accesses backend services through APIs, so page and service development remain decoupled.
Business modularity: System, assets, standards, development, data services, metadata, and governance are split by business domain for easy navigation and extension.
Decoupled management and execution: The main service manages tasks, while ETL, quality, and AI run in independent units with separately configurable resources and logs.
Reusable common capabilities: Security, data access, caching, files, and scheduling are centralized in qdata-framework.
Clear deployment boundaries: Source code, database scripts, and Docker configuration are clearly separated for local and containerized deployment.

2. Runtime entry points

Runtime unit	Entry file	Default port	Main responsibility	Build location
Main service	`QDataApplication.java`	8080	Login/authentication, system management, and business APIs	Root Maven project
Quality service	`QualityApplication.java`	8083	Generate and execute quality-check SQL	Root Maven project
ETL program	`EtlApplication.java`	None	Read, transform, and write data	Root Maven project; invoked by the scheduler
Intelligent query service	`QDataAiApplication.java`	8087	Model calls, conversations, and Text2SQL	Standalone `qdata-service-ai` project
Frontend application	`qdata-ui/src/main.js`	81 by default in development	Initialize Vue, routing, Pinia, and i18n	`qdata-ui` project

3. Frontend project structure

3.1 Frontend root

qdata-ui/
├── .env.development          # Development environment variables
├── .env.production           # Production environment variables
├── .env.staging              # Staging environment variables
├── .eslintignore             # ESLint ignore rules
├── eslint.config.js          # ESLint rules
├── index.html                # Vite entry page
├── package.json              # Dependencies, versions, and npm scripts
├── yarn.lock                 # Yarn dependency lock file
├── vite.config.js            # Vite, proxy, and build configuration
├── run-dev.sh                # Development startup script
├── bin/                      # Windows build and startup scripts
│   ├── build.bat
│   ├── package.bat
│   └── run-web.bat
├── html/
│   └── ie.html               # Browser compatibility notice
├── public/                   # Public static assets excluded from compilation
│   ├── excel/                # Excel import templates
│   ├── favicon.ico
│   └── qData-simlogo.png
├── vite/
│   └── plugins/              # Vite plugin configuration
│       ├── auto-import.js
│       ├── compression.js
│       ├── setup-extend.js
│       ├── svg-icon.js
│       └── index.js
└── src/                      # Frontend business source code

Frontend root advantages

Centralized configuration: Environment variables, dependencies, and build rules are maintained together.
Convenient workflows: Startup and build scripts support multiple platforms.
Clear boundaries: Static assets, build configuration, and business source code remain separate.

3.2 `src` source directory

qdata-ui/src/
├── App.vue                   # Vue root component
├── main.js                   # Application initialization entry
├── permission.js             # Login checks and global route guards
├── settings.js               # Page title, theme, and layout settings
├── api/                      # HTTP request functions
├── views/                    # Pages and page-level components
├── router/                   # Route definitions
├── store/                    # Pinia state management
├── layout/                   # Main page layout
├── components/               # Cross-page shared components
├── composables/              # Vue composables
├── directive/                # Custom directives
├── plugins/                  # Global plugins
├── utils/                    # Shared utilities
├── assets/                   # Build-time static assets
├── locales/                  # Page and menu translations
└── i18n/                     # Internationalization compatibility resources

src structure advantages

Fast navigation: views and api align by business domain, making requests easy to trace.
Clear responsibilities: Pages, routes, state, and shared capabilities use separate directories.
Easy extension: Shared components and localization resources are centrally reused.

3.3 Routing

Frontend routes have two sources:

Fixed or hidden routes (login, details, editing, and so on) are maintained in src/router.
Menu routes are returned by the backend at /getRouters. src/store/system/permission.js uses import.meta.glob to map component paths to Vue components under src/views and registers them dynamically.

3.4 How to Find the Backend Module for a Frontend Feature

The frontend organizes pages and requests by business directory, while the backend uses the same business boundaries for its modules. Common mappings include:

Business function	Frontend module	Backend module
Data assets	`src/views/da`, `src/api/da`	`qdata-module-da`
Metadata collection	`src/views/mc`, `src/api/mc`	`qdata-module-mc`
Data integration and development	`src/views/dpp`, `src/api/dpp`	`qdata-module-dpp`
Data services	`src/views/ds`, `src/api/ds`	`qdata-module-ds`
System management	`src/views/system`, `src/api/system`	`qdata-module-system`

The frontend and backend share business abbreviations and module boundaries—for example, da means data assets and mc means metadata collection.

Frontend–backend module mapping advantages

Intuitive structure: A frontend directory immediately indicates the corresponding backend business module.
Aligned responsibilities: Pages, requests, and backend implementation for one business domain remain together, reducing cross-module coupling.
Efficient collaboration: Frontend and backend developers can develop and communicate around the same business module.
Easy maintenance: Changes and troubleshooting can be limited quickly to the relevant code area.
Consistent extension: New business features can follow the same directory and module conventions.

3.5 How to Locate Frontend Page Source Code

For secondary development, use the following methods to locate the .vue file for a page shown in the browser:

Record page details: Open the target page and note its browser URL, menu name, and distinctive visible text.
Find a fixed page: Search src/router for the URL path. The route's component points to the corresponding file under src/views.
Find it through Menu Management: Go to System Management → Menu Management, locate the target menu by name, check its component path, and then locate the corresponding page file under src/views.
Search by business directory: Use the business abbreviation in the URL to find the matching directory under src/views; for example, da/asset usually maps to src/views/da/asset.
Search by visible text: If the path does not reveal the file, search src/views for the page title, button label, or message text.

After adding a .vue file, it must be referenced by a route or another component before it can be accessed.

Backend code consists primarily of business modules, the shared framework, and standalone execution programs. The main service assembles each business biz module through Maven; ETL, quality, and AI run independently according to their runtime characteristics.

4.1 `qdata-framework`

Shared-framework aggregator containing the following submodules:

Submodule	Capabilities
`qdata-common`	Unified responses, exceptions, base objects, enums, annotations, utilities, Excel support, encryption and masking, and database adapters
`qdata-config`	Spring, thread pools, i18n, filters, captcha, RabbitMQ, OpenAPI, and server-monitoring configuration
`qdata-mybatis`	MyBatis-Plus, pagination, base Mappers, dynamic data sources, data-scope queries, and type handling
`qdata-security`	Login authentication, token validation, API permissions, data permissions, rate limiting, operation logs, and global exception handling
`qdata-auth`	Sa-Token, OAuth2, and authorization-mode configuration
`qdata-redis`	Redis serialization, cache configuration, and shared cache services
`qdata-websocket`	WebSocket configuration, online-session management, and message push
`qdata-quartz`	Quartz task management, execution strategies, run logs, and unified scheduling adapters
`qdata-file`	File upload, storage configuration, and file-processing utilities
`qdata-generator`	Database-table parsing and frontend, backend, SQL, and localization template generation
`qdata-neo4j`	Neo4j nodes, relationships, repositories, and data-lineage access
`qdata-pay`	Alipay, WeChat Pay, refunds, callback notifications, and signature verification

Module advantage

Centralized reuse: foundational capabilities such as security, database access, caching, files, scheduling, and graph-data access are maintained centrally, allowing business modules to focus on domain logic while keeping technical implementations consistent.

4.2 Splitting `api` and `biz`

Most business modules follow this layout:

qdata-module-xxx/
├── pom.xml
├── qdata-module-xxx-api/src/main/java/.../api/
│   ├── dto/                     # Cross-module data objects
│   ├── service/                 # Service interfaces exposed to other modules
│   └── enums/                   # Shared enums where applicable
└── qdata-module-xxx-biz/src/main/
    ├── java/.../module/xxx/
    │   ├── controller/          # HTTP APIs and request/response VOs
    │   ├── service/              # Business interfaces and implementations
    │   ├── dal/dataobject/      # Database entities (DO)
    │   ├── dal/mapper/           # MyBatis Mapper interfaces
    │   ├── convert/              # DTO, VO, and DO conversion
    │   └── utils/                # Internal utilities
    └── resources/
        ├── mapper/               # MyBatis XML
        ├── i18n/                 # Module translations
        └── application-*.yml

Module advantage

Clear cross-module boundaries: *-api exposes only interfaces, DTOs, and shared enums; *-biz keeps Controllers, Services, Mappers, and business implementations, preventing other modules from depending directly on internal implementations.

*-api is the contract layer for cross-module calls. Prefer its interfaces and DTOs when reusing capabilities.
*-biz is the complete implementation layer, assembled by qdata-server.

4.3 ETL module

qdata-executor-etl reads, cleans, transforms, and writes data; it does not expose Web APIs.

qdata-executor-etl/src/main/
├── java/tech/qiantong/qdata/
│   ├── spark/etl/
│   │   ├── EtlApplication.java     # Spark ETL entry
│   │   ├── reader/                 # Data readers
│   │   ├── transition/             # Cleaning and transformation
│   │   ├── writer/                 # Data writers
│   │   └── utils/                  # Logging, RabbitMQ, Redis, and database utilities
│   └── datax/
│       ├── DataXExecutor.java       # DataX executor
│       ├── DataXJsonBuilder.java    # DataX Job JSON builder
│       ├── DataXProperties.java     # DataX configuration
│       └── DataXResult.java         # DataX execution result
└── resources/
    ├── i18n/                       # ETL translations
    └── json/                       # Task configuration examples

Module advantage

Management and execution are decoupled: DPP configures and schedules tasks, while qdata-executor-etl focuses on Spark / DataX computation. Reader, Transition, and Writer are separated by responsibility, making input, transformation, and output extensions independent.

Component	Responsibility
`ReaderFactory`	Select a database, CSV, or Excel reader by component type
`TransitionFactory`	Select field, constant, deduplication, derivation, sorting, or value-mapping transforms
`WriterFactory`	Select a data-writer implementation by component type
`RabbitmqUtils`	Report process instances, node instances, and logs
`RedisUtils`	Persist execution state such as incremental-read state
`DataXJsonBuilder`	Build DataX Job JSON from input, processing, and output nodes
`DataXExecutor`	Invoke DataX and collect its result

Spark ETL call chain:

qdata-module-dpp
  → qdata-api-ds
  → DolphinScheduler
  → EtlApplication
  → ReaderFactory → TransitionFactory → WriterFactory
  → RabbitMQ
  → qdata-module-dpp/listener updates status and logs

The status listeners are located at:

qdata-module-dpp/qdata-module-dpp-biz/src/main/java/
└── tech/qiantong/qdata/module/dpp/listener/
    ├── ProcessListener.java
    ├── TaskListener.java
    └── TaskLogListener.java

Prepare the qData main service, DolphinScheduler, Spark, RabbitMQ, Redis, and the source and target data sources required by the task. Point the backend to the correct Spark master, ETL JAR, and entry class:

ds:
  spark:
    master_url: spark://spark:7077
    main_jar: file:/dolphinscheduler/default/resources/spark-jar/qdata-executor-etl.jar
    main_class: tech.qiantong.qdata.spark.etl.EtlApplication

Build and upload the JAR to the DolphinScheduler resource directory referenced by ds.spark.main_jar:

mvn clean package -pl qdata-executor-etl -am -DskipTests

Then, on the qData ETL page:

Configure and test source and target data sources.
Create a data-integration task with one reader, zero or more transforms, and one writer.
Save and publish the task; DolphinScheduler submits it to Spark.
Review status and errors in task-instance and node-log pages.

4.4 AI module

The standalone qdata-service-ai project handles model calls, conversation orchestration, and Text2SQL.

Module	Role	Main responsibility
`qdata-ai-core`	Core module of the standalone AI project	Prompts, model adapters, conversation orchestration, and Text2SQL
`qdata-ai-server`	Startup module of the standalone AI project	HTTP APIs, authentication, exception handling, and runtime configuration

Standalone qdata-service-ai structure:

qdata-service-ai/
├── pom.xml
├── qdata-ai-core/
│   └── src/main/java/.../ai/core/
│       ├── enums/                    # Platform, message, and reply types
│       ├── prompt/                   # Prompt construction
│       ├── service/                  # Conversation, message, and model services
│       ├── service/impl/             # Core intelligent-query implementation
│       ├── utils/                    # LLM utilities
│       └── vo/                       # Request and response objects
└── qdata-ai-server/
    └── src/main/
        ├── java/.../ai/
        │   ├── server/               # Startup, security, and exception handling
        │   └── controller/admin/     # Model, conversation, and message APIs
        └── resources/                # AI service configuration

AI request flow:

Intelligent query page
  → requestAi.js → /prod-ai/chat/message
  → qdata-ai-server → qdata-ai-core → LLM service

Module advantage

Independent deployment: qdata-service-ai handles model calls and Text2SQL independently, so it can be deployed, upgraded, and scaled according to AI workload.

qdata-service-ai requires JDK 17, while the main project remains on JDK 8. The AI project is based on Spring Boot 3.5.8, Spring AI 1.1.0, and the Jakarta ecosystem. These dependencies require Java 17 and cannot run directly on the main project's JDK 8.

The AI service is a standalone Maven project and must be built and started separately:

cd qdata-service-ai
mvn clean package -DskipTests
java -jar qdata-ai-server/target/qdata-ai-server.jar --spring.profiles.active=dev

You can also configure JDK 17 for qdata-service-ai in an IDE and run:

qdata-service-ai/qdata-ai-server/src/main/java/
└── tech/qiantong/qdata/ai/server/QDataAiApplication.java

The service defaults to port 8087; the frontend accesses it through /prod-ai. Before startup, verify that the database, Redis, Neo4j, and model API settings used by the AI service are available. After changing shared DTOs, APIs, or dependencies, validate both the Java 8 main project and the Java 17 AI project.

5. Configuration, database, and deployment files

5.1 Project configuration

Configuration	Main location	Description
Main service common	`qdata-server/src/main/resources/application.yml`	Port, profiles, MyBatis, API docs, and common Spring settings
Main service environments	`qdata-server/src/main/resources/application-dev.yml`, `application-prod.yml`	Database, Redis, RabbitMQ, DolphinScheduler, and quality-service endpoints
Quality service	`qdata-service-quality/src/main/resources/application*.yml`	Quality database, MongoDB, files, and messaging
AI service	`qdata-service-ai/qdata-ai-server/src/main/resources/application*.yml`	Data source, Redis, model, and domain settings
File fragment	`qdata-framework/qdata-file/src/main/resources/application-file-*.yml`	File storage
System fragment	`qdata-module-system/qdata-module-system-biz/src/main/resources/application-system-*.yml`	System-module settings
Metadata fragment	`qdata-module-mc/qdata-module-mc-biz/src/main/resources/application-mc-*.yml`	Metadata collection
Frontend environment	`qdata-ui/.env.*`, `qdata-ui/vite.config.js`	API prefixes, authentication mode, dev proxy, and build settings

5.2 Database scripts

sql/
├── mysql/initialization/       # Complete initialization scripts by version
├── mysql/upgrade/              # Upgrade scripts between adjacent versions
├── dm/initialization/          # DM initialization scripts
├── dm/upgrade/                 # DM upgrade scripts
└── dolphinscheduler/upgrade/   # DolphinScheduler upgrade scripts

5.3 Docker

File or directory	Purpose
`docker/docker-compose.yml`	Compose entry point for the complete environment
`docker/docker-compose-base*.yml`	Database, Redis, RabbitMQ, and other foundations
`docker/docker-compose-qdata*.yml`	qData service orchestration
`docker/docker-compose-dolphinscheduler.yml`	DolphinScheduler orchestration
`docker/docker-compose-spark.yml`	Spark orchestration
`docker/docker-compose-hadoop.yml`	Hadoop components
`docker/nginx/sites/qdata.conf`	Frontend static files and `/prod-api` / `/prod-ai` reverse proxies
`docker/qdata-server`	Main-service image build files
`docker/qdata-service-quality`	Quality-service image build files
`docker/qdata-service-ai`	AI-service image build files
`docker/dolphinscheduler`	DolphinScheduler configuration, SQL, resources, and dependencies