微服务CI/cd实践-GitOps完整设计与实现

作者：Lizeyang 2021-09-07 08:23:45

云计算 当我在Gitlab提交了代码，会通过GitLab webhook 触发Jenkins Scheduler 作业， 会将此次提交代码所产生的hook data数据信息以POST的方式传给Jenkins Job。

专注于为中小企业提供成都网站制作、成都网站设计服务,电脑端+手机端+微信端的三站合一,更高效的管理,为中小企业川汇免费做网站提供优质的服务。我们立足成都，凝聚了一批互联网行业人才，有力地推动了上千多家企业的稳健成长，帮助中小企业通过网站建设实现规模扩充和转变。

单应用与环境

多应用与环境

CI持续集成

首先，准备一个代码库：

https://github.com/DevOpsCICDCourse/microservicescicd/blob/main/microservice-demo-service-master.zip

我们来梳理一下CI流水线的步骤：

• 由于此次实现的代码仓库类型为单一存储库，即一个存储库存放多个服务模块代码，每个子目录为一个服务模块。
• 首先，我们的持续集成流水线需要能够正确获取，当前的commit是哪个服务的代码。
• 确定好服务，然后下载该服务的代码，进行编译打包、单元测试、代码扫描和构建镜像等步骤。

如何获取commit的服务信息?这里我们使用GitLab WebHook功能和Jenkins 的job 构建触发器对接来实现。

工作流程是：当我在Gitlab提交了代码，会通过GitLab webhook 触发Jenkins Scheduler 作业， 会将此次提交代码所产生的hook data数据信息以POST的方式传给Jenkins Job。此时Jenkins job可以编写使用Generic Hook插件获取此次POST请求传输过来的请求体Body信息。是一段JSON数据， 该job运行后编写Pipeline 解析JSON中的数据拿到所变更的服务模块信息。最后触发对应服务的CI作业进行构建。

CI-Scheduler 作业

此作业只需要开启webhook， 配置触发token(唯一性)。生成hookurl：http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CI

Jenkinsfile

  
 
 
 
   
  
  
  
pipeline { 
   
  
  
  
 agent any  
   
  
  
  
 
   
  
  
  
 stages{ 
   
  
  
  
 
   
  
  
  
  stage("GetData"){ 
   
  
  
  
   steps{ 
   
  
  
  
    script { 
   
  
  
  
     echo "${webHookData}" 
   
  
  
  
 
   
  
  
  
     data = readJSON  text: "${webHookData}" 
   
  
  
  
 
   
  
  
  
     println(data) 
   
  
  
  
 
   
  
  
  
     env.branchName = data.ref - "refs/heads/" 
   
  
  
  
     env.commitId = data.checkout_sha 
   
  
  
  
     env.projectId = data.project_id 
   
  
  
  
     commits = data["commits"] 
   
  
  
  
 
   
  
  
  
     println("${env.branchName}") 
   
  
  
  
     println("${env.commitID}") 
   
  
  
  
     println("${env.projectId}") 
   
  
  
  
 
   
  
  
  
     //env.moduleName = "service01" 
   
  
  
  
     changeServices = [] 
   
  
  
  
                    for(commit in commits) { 
   
  
  
  
                        println(commit.id) 
   
  
  
  
 
   
  
  
  
                        //added 
   
  
  
  
                        for (add in commit.added) { 
   
  
  
  
                            s = add.split("/") as List 
   
  
  
  
                            if (s.size() > 1){ 
   
  
  
  
                                if (changeServices.indexOf(s[0]) == -1){ 
   
  
  
  
                                    changeServices.add(s[0]) 
   
  
  
  
                                } 
   
  
  
  
                            } 
   
  
  
  
                        } 
   
  
  
  
 
   
  
  
  
                        //modified 
   
  
  
  
                        for (m in commit.modified) { 
   
  
  
  
                            s = m.split("/") as List 
   
  
  
  
                            // println s 
   
  
  
  
                            // println s.size() 
   
  
  
  
                            // println s[0] 
   
  
  
  
                            if (s.size() > 1){ 
   
  
  
  
                                // println changeServices.indexOf(s[0]) 
   
  
  
  
                                if (changeServices.indexOf(s[0]) == -1){ 
   
  
  
  
                                    changeServices.add(s[0]) 
   
  
  
  
                                } 
   
  
  
  
                            } 
   
  
  
  
                        } 
   
  
  
  
 
   
  
  
  
                        //removed 
   
  
  
  
                        for (r in commit.removed) { 
   
  
  
  
                            s = r.split("/") as List 
   
  
  
  
                            println s 
   
  
  
  
                            if (s.size() > 1){ 
   
  
  
  
                                if (changeServices.indexOf(s[0]) == -1){ 
   
  
  
  
                                    changeServices.add(s[0]) 
   
  
  
  
                                } 
   
  
  
  
                            } 
   
  
  
  
                        } 
   
  
  
  
                    } 
   
  
  
  
 
   
  
  
  
                    println(changeServices) 
   
  
  
  
                    //currentBuild.description = " Trigger by  ${eventType} ${changeServices}  
   
  
  
  
    } 
   
  
  
  
   } 
   
  
  
  
  } 
   
  
  
  
 
   
  
  
  
  stage('DefineService') { 
   
  
  
  
            steps { 
   
  
  
  
                script{ 
   
  
  
  
                    println(changeServices) 
   
  
  
  
                    //服务构建顺序控制 
   
  
  
  
                    services = ['service02', 'service01'] 
   
  
  
  
                    for (service in services){ 
   
  
  
  
                        if (changeServices.indexOf(service) != -1){ 
   
  
  
  
                            jobName = 'microservicecicd-'+service+'-service-CI' 
   
  
  
  
                            build job: jobName, wait: false,  parameters: [string(name: 'branchName', value: "${env.branchName}" ), 
   
  
  
  
                                                                           string(name: 'commitId',   value: "${env.commitId}" ),  
   
  
  
  
                                                                           string(name: 'projectId',  value: "${env.projectId}" )] 
   
  
  
  
                        } 
   
  
  
  
                    } 
   
  
  
  
                } 
   
  
  
  
            } 
   
  
  
  
        } 
   
  
  
  
 } 
   
  
  
  
} 
  
 
 

GitLab 配置WebHook

开启webhook，配置hookurl：http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CI

CI流水线-CI作业

每个微服务创建一个CI作业，具有三个字符串参数：分支名称、commitID、项目ID。

Jenkinsfile

  
 
 
 
   
  
  
  
String branchName = "${env.branchName}" 
   
  
  
  
String moduleName = "${JOB_NAME}".split("/")[1].split("-")[1] 
   
  
  
  
String srcUrl = "http://gitlab.idevops.site/microservicecicd/microservicecicd-demo-service.git" 
   
  
  
  
String commitId = "${env.commitId}" 
   
  
  
  
String projectId = "${env.projectId}" 
   
  
  
  
 
   
  
  
  
pipeline { 
   
  
  
  
    agent { node { label "build" } } 
   
  
  
  
 
   
  
  
  
    stages { 
   
  
  
  
        stage('GetCode') { 
   
  
  
  
            steps { 
   
  
  
  
                script { 
   
  
  
  
                    checkout([$class: 'GitSCM',  
   
  
  
  
                            branches: [[name: "${branchName}"]],  
   
  
  
  
                            doGenerateSubmoduleConfigurations: false, 
   
  
  
  
                            extensions: [[$class: 'SparseCheckoutPaths',  
   
  
  
  
                                        sparseCheckoutPaths: [[path: "${moduleName}"],[path: 'Dockerfile']]]],  
   
  
  
  
                            submoduleCfg: [],  
   
  
  
  
                            userRemoteConfigs: [[credentialsId: 'gitlab-admin-user', 
   
  
  
  
                                                url: "${srcUrl}"]]]) 
   
  
  
  
                } 
   
  
  
  
                 
   
  
  
  
            } 
   
  
  
  
        } 
   
  
  
  
 
   
  
  
  
        stage("Build&Test"){ 
   
  
  
  
            steps{ 
   
  
  
  
                script{ 
   
  
  
  
                    echo "Build..........." 
   
  
  
  
 
   
  
  
  
                    sh """ 
   
  
  
  
                    cd ${moduleName}  
   
  
  
  
                    mvn clean package 
   
  
  
  
 
   
  
  
  
                    """ 
   
  
  
  
                } 
   
  
  
  
            } 
   
  
  
  
            post { 
   
  
  
  
                always { 
   
  
  
  
                    junit "${moduleName}/target/surefire-reports/*.xml" 
   
  
  
  
                } 
   
  
  
  
            } 
   
  
  
  
        } 
   
  
  
  
 
   
  
  
  
        stage("SonarScan"){ 
   
  
  
  
            steps{ 
   
  
  
  
                script{ 
   
  
  
  
 
   
  
  
  
                    def sonarDate = sh returnStdout: true, script: 'date  +%Y%m%d%H%M%S' 
   
  
  
  
                    sonarDate = sonarDate - "\n" 
   
  
  
  
 
   
  
  
  
                    withCredentials([string(credentialsId: 'sonar-admin-user', variable: 'sonartoken'), 
   
  
  
  
                                    string(credentialsId: 'gitlab-user-token', variable: 'gitlabtoken')]) { 
   
  
  
  
                        // some block 
   
  
  
  
                        sh """ 
   
  
  
  
                        cd ${moduleName}  
   
  
  
  
                        sonar-scanner \ 
   
  
  
  
                        -Dsonar.projectKey=${JOB_NAME} \ 
   
  
  
  
                        -Dsonar.projectName=${JOB_NAME} \ 
   
  
  
  
                        -Dsonar.projectVersion=${sonarDate} \ 
   
  
  
  
                        -Dsonar.ws.timeout=30 \ 
   
  
  
  
                        -Dsonar.projectDescription="xxxxxxx" \ 
   
  
  
  
                        -Dsonar.links.homepage=http://www.baidu.com \ 
   
  
  
  
                        -Dsonar.sources=src \ 
   
  
  
  
                        -Dsonar.sourceEncoding=UTF-8 \ 
   
  
  
  
                        -Dsonar.java.binaries=target/classes \ 
   
  
  
  
                        -Dsonar.java.test.binaries=target/test-classes \ 
   
  
  
  
                        -Dsonar.java.surefire.report=target/surefire-reports \ 
   
  
  
  
                        -Dsonar.host.url="http://sonar.idevops.site" \ 
   
  
  
  
                        -Dsonar.login=${sonartoken} \ 
   
  
  
  
                        -Dsonar.gitlab.commit_sha=${commitId} \ 
   
  
  
  
                        -Dsonar.gitlab.ref_name=${branchName} \ 
   
  
  
  
                        -Dsonar.gitlab.project_id=${projectId} \ 
   
  
  
  
                        -Dsonar.dynamicAnalysis=reuseReports \ 
   
  
  
  
                        -Dsonar.gitlab.failure_notification_mode=commit-status \ 
   
  
  
  
                        -Dsonar.gitlab.url=http://gitlab.idevops.site \ 
   
  
  
  
                        -Dsonar.gitlab.user_token=${gitlabtoken} \ 
   
  
  
  
                        -Dsonar.gitlab.api_version=v4 
   
  
  
  
 
   
  
  
  
                        """ 
   
  
  
  
 
   
  
  
  
                    } 
   
  
  
  
  
   
  
  
  
                } 
   
  
  
  
            } 
   
  
  
  
        } 
   
  
  
  
 
   
  
  
  
        stage("BuildImage"){ 
   
  
  
  
            steps{ 
   
  
  
  
                script{ 
   
  
  
  
 
   
  
  
  
                     withCredentials([usernamePassword(credentialsId: 'aliyun-registry-admin', passwordVariable: 'password', usernameVariable: 'username')]) { 
   
  
  
  
                 
   
  
  
  
                         env.nowDate = sh  returnStdout: true, script: 'date  +%Y%m%d%H%M%S' 
   
  
  
  
                         env.nowDate = env.nowDate - "\n" 
   
  
  
  
 
   
  
  
  
                         env.releaseVersion = "${env.branchName}" 
   
  
  
  
                         env.imageTag = "${releaseVersion}-${nowDate}-${commitId}" 
   
  
  
  
                         env.dockerImage = "registry.cn-beijing.aliyuncs.com/microservicecicd/microservicecicd-${moduleName}-service:${env.imageTag}" 
   
  
  
  
                         env.jarName = "${moduleName}-${branchName}-${commitId}" 
   
  
  
  
                         sh """ 
   
  
  
  
                             docker login -u ${username} -p ${password}  registry.cn-beijing.aliyuncs.com 
   
  
  
  
                             cd ${moduleName} && docker build -t ${dockerImage} -f ../Dockerfile --build-arg SERVICE_NAME=${jarName} . 
   
  
  
  
                             sleep 1 
   
  
  
  
                             docker push ${dockerImage} 
   
  
  
  
                             sleep 1 
   
  
  
  
                             docker rmi ${dockerImage} 
   
  
  
  
                          """ 
   
  
  
  
                    } 
   
  
  
  
 
   
  
  
  
 
   
  
  
  
                } 
   
  
  
  
            } 
   
  
  
  
        } 
   
  
  
  
 
   
  
  
  
         
   
  
  
  
    } 
   
  
  
  
} 
  
 
 

GitOps-CI扩展部分

在原始CI作业的步骤基础上，增加了一个更新环境的步骤。GitOps实践会将当前的基础环境部署文件存放到一个Git仓库中。我们的CI作业在完成镜像上传后，同时更新环境部署文件中的镜像标签信息。(所以我们需要先获取该环境文件并更新上传)

  
 
 
 
   
  
  
  
stage("PushFile"){ 
   
  
  
  
          // when { 
   
  
  
  
          //   expression { "${env.branchName}".contains("RELEASE-") } 
   
  
  
  
          // } 
   
  
  
  
          steps{ 
   
  
  
  
            script{ 
   
  
  
  
              if ("${env.branchName}".contains("RELEASE-")){ 
   
  
  
  
                println("branchName = branchName") 
   
  
  
  
                env.branchName = "master" 
   
  
  
  
 
   
  
  
  
              } else { 
   
  
  
  
                env.branchName = "feature" 
   
  
  
  
              } 
   
  
  
  
 
   
  
  
  
                for (i = 0; i < 3; i++) { 
   
  
  
  
                    //下载版本库文件  
   
  
  
  
                    response = GetRepoFile(40,"${moduleName}%2fvalues.yaml", "${env.branchName}") 
   
  
  
  
                    //println(response) 
   
  
  
  
                     
   
  
  
  
                    //替换文件中内容 
   
  
  
  
                    yamlData = readYaml text: """${response}""" 
   
  
  
  
 
   
  
  
  
                    println(yamlData.image.version) 
   
  
  
  
                    println(yamlData.image.commit) 
   
  
  
  
                    yamlData.image.version = "${releaseVersion}-${env.nowDate}" 
   
  
  
  
                    yamlData.image.commit  = "${commitId}" 
   
  
  
  
 
   
  
  
  
                    println(yamlData.toString()) 
   
  
  
  
 
   
  
  
  
                    sh "rm -fr test.yaml" 
   
  
  
  
                    writeYaml charset: 'UTF-8', data: yamlData, file: 'test.yaml' 
   
  
  
  
                    newYaml = sh returnStdout: true, script: 'cat test.yaml' 
   
  
  
  
                     
   
  
  
  
                    println(newYaml) 
   
  
  
  
                    //更新gitlab文件内容 
   
  
  
  
                    base64Content = newYaml.bytes.encodeBase64().toString() 
   
  
  
  
 
   
  
  
  
                    // 会有并行问题，同时更新报错 
   
  
  
  
                    try { 
   
  
  
  
                      UpdateRepoFile(40,"${moduleName}%2fvalues.yaml",base64Content, "${env.branchName}") 
   
  
  
  
                      break; 
   
  
  
  
                    } catch(e){ 
   
  
  
  
                      sh "sleep 2" 
   
  
  
  
                      continue; 
   
  
  
  
                    } 
   
  
  
  
                } 
   
  
  
  
            } 
   
  
  
  
          } 
   
  
  
  
        } 
   
  
  
  
         
   
  
  
  
 //封装HTTP请求 
   
  
  
  
def HttpReq(reqType,reqUrl,reqBody){ 
   
  
  
  
    def gitServer = "http://gitlab.idevops.site/api/v4" 
   
  
  
  
    withCredentials([string(credentialsId: 'gitlab-token', variable: 'gitlabToken')]) { 
   
  
  
  
      result = httpRequest customHeaders: [[maskValue: true, name: 'PRIVATE-TOKEN', value: "${gitlabToken}"]],  
   
  
  
  
                httpMode: reqType,  
   
  
  
  
                contentType: "APPLICATION_JSON", 
   
  
  
  
                consoleLogResponseBody: true, 
   
  
  
  
                ignoreSslErrors: true,  
   
  
  
  
                requestBody: reqBody, 
   
  
  
  
                url: "${gitServer}/${reqUrl}" 
   
  
  
  
                //quiet: true 
   
  
  
  
    } 
   
  
  
  
    return result 
   
  
  
  
} 
   
  
  
  
 
   
  
  
  
 
   
  
  
  
//获取文件内容 
   
  
  
  
def GetRepoFile(projectId,filePath,branchName){ 
   
  
  
  
    apiUrl = "projects/${projectId}/repository/files/${filePath}/raw?ref=${branchName}" 
   
  
  
  
    response = HttpReq('GET',apiUrl,'') 
   
  
  
  
    return response.content 
   
  
  
  
} 
   
  
  
  
 
   
  
  
  
//更新文件内容 
   
  
  
  
def UpdateRepoFile(projectId,filePath,fileContent, branchName){ 
   
  
  
  
    apiUrl = "projects/${projectId}/repository/files/${filePath}" 
   
  
  
  
    reqBody = """{"branch": "${branchName}","encoding":"base64", "content": "${fileContent}", "commit_message": "update a new file"}""" 
   
  
  
  
    response = HttpReq('PUT',apiUrl,reqBody) 
   
  
  
  
    println(response) 
   
  
  
  
 
   
  
  
  
} 
  
 
 

images

GitOps-CD部分

CD-Scheduler作业

此作业其实也是接收GitLab的webhook请求， 与CI-scheduler作业类似。不同的是这个CD-scheduler作业是用来接收环境仓库的代码变更。开启webhook， 配置触发token。生成hookurl：http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CD

Jenkinsfile

  
 
 
 
   
  
  
  
pipeline { 
   
  
  
  
    agent any 
   
  
  
  
 
   
  
  
  
    stages { 
   
  
  
  
        stage('GetCommitService') { 
   
  
  
  
            steps { 
   
  
  
  
                script{ 
   
  
  
  
                    echo 'Hello World' 
   
  
  
  
                    echo "${WebHookData}" 
   
  
  
  
                     
   
  
  
  
                    // Git Info 
   
  
  
  
                    webhookdata = readJSON text: """${WebHookData}""" 
   
  
  
  
                    eventType = webhookdata["object_kind"] 
   
  
  
  
                    commits = webhookdata["commits"] 
   
  
  
  
                    branchName = webhookdata["ref"] - "refs/heads/" 
   
  
  
  
                    projectID = webhookdata["project_id"] 
   
  
  
  
                    commitID = webhookdata["checkout_sha"] 
   
  
  
  
 
   
  
  
  
 
   
  
  
  
                    changeServices = [] 
   
  
  
  
                    for(commit in commits) { 
   
  
  
  
                        println(commit.id) 
   
  
  
  
 
   
  
  
  
                        //added 
   
  
  
  
                        for (add in commit.added) { 
   
  
  
  
                            s = add.split("/") as List 
   
  
  
  
                            if (s.size() > 1){ 
   
  
  
  
                                if (changeServices.indexOf(s[0]) == -1){ 
   
  
  
  
                                    changeServices.add(s[0]) 
   
  
  
  
                                } 
   
  
  
  
                            } 
   
  
  
  
                        } 
   
  
  
  
 
   
  
  
  
                        //modified 
   
  
  
  
                        for (m in commit.modified) { 
   
  
  
  
                            s = m.split("/") as List 
   
  
  
  
                            // println s 
   
  
  
  
                            // println s.size() 
   
  
  
  
                            // println s[0] 
   
  
  
  
                            if (s.size() > 1){ 
   
  
  
  
                                // println changeServices.indexOf(s[0]) 
   
  
  
  
                                if (changeServices.indexOf(s[0]) == -1){ 
   
  
  
  
                                    changeServices.add(s[0]) 
   
  
  
  
                                } 
   
  
  
  
                            } 
   
  
  
  
                        } 
   
  
  
  
 
   
  
  
  
                        //removed 
   
  
  
  
                        for (r in commit.removed) { 
   
  
  
  
                            s = r.split("/") as List 
   
  
  
  
                            println s 
   
  
  
  
                            if (s.size() > 1){ 
   
  
  
  
                    &nbs            

                文章标题：微服务CI/CD实践-GitOps完整设计与实现
                

                URL标题：http://www.csdahua.cn/qtweb/news23/405773.html
            
            

                网站建设、网络推广公司-快上网，是专注品牌与效果的网站制作，网络营销seo公司；服务项目有等
                                
            

                广告
                
            
            

                声明：本网站发布的内容（图片、视频和文字）以用户投稿、用户转载内容为主，如果涉及侵权请尽快告知，我们将会在第一时间删除。文章观点不代表本网站立场，如需处理请联系客服。电话：028-86922220；邮箱：631063699@qq.com。内容未经允许不得转载，或转载时需注明来源：
                快上网