Automated Build Via Jenkins

Continuing on from the previous experiment, we will implement a one-click deployment project through Jenkins.

Project Overview

The project is a frontend and backend separation project. The backend is a multimodule project using SpringBoot, and its structure is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
+---backend             # Backend 
|   +---src
|   \---target
+---frontend            # Frontend
|   +---dist
|   \---src
+---others              # Other Backend Modules
|   +---mqtt
|       +---src
|       \---target
+---data                # Database Config
|   +---conf
|   +---init
|   \---sql
+---conf                # Config
|   +---emqx
|   \---nginx
+---dockerfile
+---shell               # Scripts To Help Build Images
\---docker-compose.yml

Problems

For convenience, the automated build process is done in a Jenkins container. With Docker, we don’t have to spend a lot of time configuring the environment. Next, let me introduce the problems encountered.

Commends not found

Through the logs, I observed that the npm command could not be found. Because I manged the Node environment through nvm and didn’t configure the PATH variable, npm command always failed to execute.

Finally I mounted the nvm installation directory and configuration to the Jenkins container. When the container is started, first make the configuration file in the /etc/profile.d directory take effect, and then switch the node environment. The Pipeline I wrote is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
stage('build-frontend') {
    steps {
        dir("./frontend") {
            sh '''
                source /etc/profile.d/nvm.sh
                nvm use 16.15.0
                npm cache clean –force
                npm cache verify
                if [ -f dist ]; then rm dist -rf; fi;
                pnpm config get registry
                pnpm config set registry https://registry.npmmirror.com/
                pnpm install
                pnpm run build:prod
                '''
        }
    }
}

Note: Errors such as commands not found can be solved by mounting its installation directory and making the configuration effective.

How to deploy remotely via ssh

After configuring the private key on the global credentials, it is best to deploy remotely in Pipeline through a plugin (SSH Pipeline Steps). You can refer to my configuration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
steps {
    withCredentials(bindings: [ \
            sshUserPrivateKey(credentialsId: 'private-key',
            keyFileVariable: 'primaryKeyVar',
            passphraseVariable: 'pwdVar',
            usernameVariable: 'userVar')]) {
        script {
            def remote = [:]
            remote.name = '192.168.137.2'
            remote.host = '192.168.137.2'
            remote.user = userVar
            remote.identityFile = primaryKeyVar
            remote.allowAnyHosts = true
            stage('Remote SSH') {
                sshCommand remote: remote, command: "mkdir -p ${REMOTE_WORKDIR}/frontend"
                // Frontend Packaging File
                sshPut remote: remote, from: "./frontend/dist", into: "${REMOTE_WORKDIR}/frontend"
                // Show All Files Transferred
                sshCommand remote: remote, command: "ls -lrt ${REMOTE_WORKDIR}/**"
            }
        }
    }
}

How to control the startup order of services

When writing docker-compose.yml, if you need to control the startup order, the depends_on field alone is not enough. It only controls the startup order, but doesn’t guarantee that the next service will be started only after the service is successfully started.

I referred to many articles and found a solution. By mounting wait-for-it.sh, write a script to replace the container’s entrypoint. If you have any question, you can also refer to [this article] (https://www.cnblogs.com/wang_yb/p/9400291.html).

I downloaded the script called *wait-for-it from [https: //github.com/vishnubob/wait-for-it](https: //github.com/vishnubob/wait-for-it). I renamed it entrypoint.sh and put it in the shell directory. Below is part of the docker-compose configuration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
service:
    backend:
        build:
            context: .
            dockerfile: dockerfile/backend-Dockerfile
        image: backend_app
        container_name: prod-backend-app
        ports:
            - "13245:13245"
        environment:
            TZ: Asia/Shanghai
            PORT: 13245
            PROFILE: prod
        volumes:
            - ./shell/entrypoint.sh:/entrypoint.sh
        depends_on:
            - emq
            - influx

The Dockerfile of the service called backend is renamed backend-Dockerfile and is placed in the dockerfile directory. Since the backend service relies on the mysql and redis services, the content of its Dockerfile is as follows:

1
2
3
4
5
FROM openjdk:8-jdk
ADD backend/target/*.jar /app.jar
CMD bash /entrypoint.sh mysqldb:3306 -t 30 -- echo "mysql started" \
    && bash /entrypoint.sh redisdb:6379 -t 20 -- echo "redisdb started" \
    && java -jar -Dfile.encoding=utf-8 -Dserver.port=${PORT} -Dspring.profiles.active=${PROFILE} /app.jar

How to configure Emqx

First of all, the version of Emqx is 4.4.2.

I referred to the official document and configured the container through environment variables. The emq_web_hook plugins is effective but the emq_http_auth plugin is always ineffective. I tried various methods but failed. At last, solved it by mounting its configuration file.

You can refer to my docker-compose configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
service:
    emq:
        user: root
        restart: always
        container_name: prod-emqx
        image: emqx/emqx:4.4.2
        ports:
            - "18083:18083"
            - "1883:1883"
            - "8083:8083"
            - "8084:8084"
            - "8883:8883"
        volumes:
            - /etc/localtime:/etc/localtime
            - /root/docker/emqx/log:/opt/emqx/log
            - ./conf/emqx/emqx_auth_http.conf:/opt/emqx/etc/plugins/emqx_auth_http.conf
            - ./conf/emqx/emqx_web_hook.conf:/opt/emqx/etc/plugins/emqx_web_hook.conf
        environment:
            EMQX_ALLOW_ANONYMOUS: false # allow_anonymous
            EMQX_LOADED_PLUGINS: "emqx_management,emqx_auth_http,emqx_dashboard,emqx_web_hook"
            EMQX_DASHBOARD__DEFAULT_USER__LOGIN: root
            EMQX_DASHBOARD__DEFAULT_USER__PASSWORD: 12345
        networks:
            app_net:
                ipv4_address: 172.31.0.10
        depends_on:
            - auth
        links:
            - auth

The frontend can’t access the backend container’s interface

Many articles introduce the use of nginx as the base image to build the front-end image.

I referred this article and wrote the nginx configuration file, the example is as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
server {
    listen       13246;
    server_name  localhost;
    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }
    #Location配置
    location /api/ {
        rewrite  /api/(.*)  /$1  break;
        proxy_pass http://backend:13245;
    }
}

Since I had not learned nginx in depth, I encountered various problems. After repeated attempts, I finally achieved my expectations.

Additional Notes

The project is based on Vue3 and SpringBoot, and uses Jenkins container for automated construction and deployment. For convenience and space saving, the best way is to install common tool software on the host machine and then mount it into the container.

Next I will add some installation dependencies

Install Docker

1
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun

Install jdk8

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
rm /tmp/jdk*.tar.gz -f
wget https://repo.huaweicloud.com/java/jdk/8u202-b08/jdk-8u202-linux-x64.tar.gz -P /tmp
ls -lht /tmp/ | grep jdk
mkdir -p /usr/local/java/
tar -zxf /tmp/jdk-8u202-linux-x64.tar.gz -C /usr/local/java/ && ls /usr/local/java/ -l
# Add environment jdk1.8 environment variables
cat > /etc/profile.d/java.sh <<EOF
export JAVA_HOME=/usr/local/java/jdk1.8.0_202
export JRE_HOME=\${JAVA_HOME}/jre
export CLASSPATH=.:\${JAVA_HOME}/lib:\${JRE_HOME}/lib
export PATH=\${JAVA_HOME}/bin:\$PATH
EOF

source /etc/profile.d/java.sh
ln -s ${JAVA_HOME}/bin/java /usr/bin/java
java -version

Install Maven

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
maven_version=3.9.0
rm /tmp/*maven*.tar.gz -rf
#wget https://dlcdn.apache.org/maven/maven-3/3.9.0/binaries/apache-maven-3.9.0-bin.tar.gz -P /tmp
wget https://mirrors.aliyun.com/apache/maven/maven-3/3.9.0/binaries/apache-maven-3.9.0-bin.tar.gz -P /tmp
ls -lht /tmp/ | grep maven
mkdir -p /usr/local/maven/
tar -zxf /tmp/apache-maven-*.tar.gz -C /usr/local/maven/ && ls /usr/local/maven/ -l
# Add Maven environment variables
cat > /etc/profile.d/maven.sh << EOF
export MAVEN_VERSION="${maven_version}"
export M2_HOME=/usr/local/maven/apache-maven-\${MAVEN_VERSION}
export MAVEN_HOME=/usr/local/maven/apache-maven-3.9.0
export PATH=\${MAVEN_HOME}/bin:\${PATH}
EOF
chmod +x /etc/profile.d/maven.sh
source /etc/profile.d/maven.sh
mvn -version

Install nvm

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
nvm_version=0.39.3
rm /tmp/v${nvm_version}.tar.gz -rf
wget https://github.com/nvm-sh/nvm/archive/refs/tags/v${nvm_version}.tar.gz -P /tmp
ls -lht /tmp/ | grep "v${nvm_version}.tar.gz"
mkdir -p /usr/local/nvm/
tar -zxf /tmp/v${nvm_version}.tar.gz -C /usr/local/nvm/ && ls /usr/local/nvm/ -l
# Add nvm environment variables
cat << EOF  > /etc/profile.d/nvm.sh
export NVM_VERSION="${nvm_version}"
export NVM_DIR="/usr/local/nvm/nvm-\${NVM_VERSION}"
[ -s "\$NVM_DIR/nvm.sh" ] && \. "\$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "\$NVM_DIR/bash_completion" ] && \. "\$NVM_DIR/bash_completion"
EOF

chmod +x /etc/profile.d/nvm.sh
source /etc/profile.d/nvm.sh
nvm version

Install docker-compose

It is recommended to download docker-compose from github official website, some mirror website packages are too old.

1
2
3
curl -L https://github.com/docker/compose/releases/download/v2.16.0/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose -v

Summary

By applying the newly learned Jenkins automation technology to the project, time can be greatly saved. Operation and maintenance do not have to repeatedly deploy manually, and testing becomes a part of development. Only test scripts need to be written. Each time the developer submits code, the test scripts will be automatically executed and then deployed to the remote via ssh. From the traditional stage-by-stage development, where a stage cannot be skipped directly, development can now be quickly tested and put into production to get feedback, doubling efficiency. Perhaps this is why the CI/CD concept and tools are popular.

References

2023-02-16 First Update

2023-02-26 Add Dependent Installation configuration

2024-07-06 Support English