摘要：自定義鏡像部分自定義鏡像部分安裝工具安裝工具工具官方下載工具解壓工具到查看版本初始化目錄結構初始化目錄結構筆者是在目錄下開始操作的，提前說下免得讀者操作有誤。修改修改腳本只作啟動功能，為了方便傳調優參數，定制了環境變量供使用。

本文目標

由于 OpenShift 官方提供的鏡像與模板（OpenJDK8）不完全滿足業務需要：

• 不包含飛行記錄功能。只有 OpenJDK11 以上才被 Oracle 開源

• 生成堆 dump 很大很慢（公司項目 JVM 堆最小16G，最大32G），導出-壓縮-傳輸 較長（半小時以上），對業務停頓時間過長

• 鏡像內置的獲取性能指標的命令很少，常規網絡狀態查詢命令如 netstat、ping 等都沒有

• 鏡像內部s2i流程編寫得過于復雜，鏡像構建過程緩存多且未移除，導致最終鏡像普遍體積過大（350M的業務應用，打出鏡像有 1.1G…）

為了方便定位性能問題，筆者仔細閱讀了 OpenShift v3.11 版本的 s2i 構建流程、鏡像自定義等相關文檔，定制出了自己的模板與鏡像，目前已經在生產環境上驗證通過，在這里記錄一二。如果有幫到你，點個關注和贊再走吧~

本文主體思路是使用 s2i工具創建基礎的構建調試環境，修改Dockerfile與構建腳本生成鏡像，再推送鏡像和模板到上篇文章OpenShift 本地開發環境配置(基于 Minishift)的環境中，用一個demo測試下最終效果。

自定義 s2i 鏡像部分

安裝 s2i 工具

s2i 工具官方Release Page：https://github.com/openshift/source-to-image/releases

#下載s2i工具wget https://github.com/openshift/source-to-image/releases/download/v1.3.1/source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz#解壓s2i工具到/usr/binsudo tar zxf source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz -C /usr/bin#查看s2i版本s2i version

初始化 s2i 目錄結構

筆者是在 ~/WorkSpace/openshift 目錄下開始操作的，提前說下免得讀者操作有誤。

#生成s2i腳本目錄結構，create參數為創建，第3參數為鏡像名，第3參數為生成目錄名s2i create oracle-jdk-8 oracle-jdk-8-catelog#進入生成的目錄中cd oracle-jdk-8-catelog#查看基本目錄結構tree -L 3.├── Dockerfile #鏡像Dockerfile├── Makefile #Makefile構建腳本├── README.md├── s2i│   └── bin #s2i腳本目錄│       ├── assemble #裝配腳本│       ├── run #運行腳本│       ├── save-artifacts #可選，保存制品腳本，增量構建時用│       └── usage #使用說明腳本└── test    ├── run #測試運行腳本    └── test-app #構建的源碼（本例中可以是jar包）        └── index.html

這里筆者用不到 s2i/bin/save-artifacts，因為不需要增量構建，刪除之; test/test-app/index.html 也用不到，刪除之。

rm -rf s2i/bin/save-artifactsrm -rf test/test-app/index.html

下載并解壓 OracleJDK8安裝包，適度精簡

由于筆者已經將jdk8安裝包下載好了，下載 OracleJDK 付費版本以前的版本，參考我的博客，這里就直接從 ~/Downloads 解壓到 ~/WorkSpace/openshift/oracle-jdk-8-catelog 下

#解壓jdk安裝包tar zxvf ~/Downloads/jdk-8u181-linux-x64.tar.gz -C ~/WorkSpace/openshift/oracle-jdk-8-catelog#進入jdk解壓目錄cd /home/hellxz/WorkSpace/openshift/oracle-jdk-8-catelog/jdk1.8.0_181#適度精簡，刪除源碼包，能小26Mrm -rf javafx-src.zip src.zip#回到模板初始化的目錄cd ..

修改 Dockerfile

vim Dockerfile，把默認的都刪除掉，添加下邊的內容：

FROM debian:busterENV BUILDER_VERSION 1.0# Set labels used in OpenShift to describe the builder imageLABEL description="Source To Image (S2I) image for Hellxz Providing Oracle JDK 8" /      maintainer="Hellxz Zhang " /      io.k8s.description="Platform for building and running Java applications with Oracle JDK 8" /      io.k8s.display-name="Java Applications" /      io.openshift.expose-services="8080:http" /      io.openshift.tags="builder,java" /      io.openshift.s2i.scripts-url="image:///usr/libexec/s2i" /      io.openshift.s2i.destination="/tmp"# Copy Oracle JDK8COPY ./jdk1.8.0_181 /usr/lib/jvm# Copy scripts to s2i build path, must same as `io.openshift.s2i.scripts-url` label specified.COPY ./s2i/bin/ /usr/libexec/s2i# EnvironmentsENV JAVA_HOME="/usr/lib/jvm" /    PATH="/usr/lib/jvm/bin:${PATH}" /    TZ="Asia/Shanghai"# Create oraclejdk user with home dir /deployments# Grant user exec privilege# Set apt mirror and install some utils then clean # Generate UTF-8 localesRUN mkdir /deployments && /    useradd -M -d /deployments -u 1001  -s /bin/bash oraclejdk8 && /    chown oraclejdk8 /deployments && chmod 777 /deployments && /    bash -c "echo -e /"deb http://mirrors.163.com/debian/ buster main non-free contrib/ndeb http://mirrors.163.com/debian/ buster-updates main non-free contrib/ndeb http://mirrors.163.com/debian/ buster-backports main non-free contrib/ndeb http://mirrors.163.com/debian-security/ buster/updates main non-free contrib/" > /etc/apt/sources.list" && /    apt-get update -y && apt-get install -y net-tools locales procps && /    apt-get clean all -y && rm -rf /var/lib/apt/lists/* && /    sed -i "/en_US.UTF-8/s/^# //g" /etc/locale.gen && locale-gen# To avoid build image error tips set LC env after locale.genENV LC_ALL="en_US.UTF-8"# Switch userUSER 1001# Switch WorkDirWORKDIR /deploymentsEXPOSE 8080 8443# Set the default CMD for the imageCMD ["/usr/libexec/s2i/usage"]

上述的Dockerfile做了什么呢？

• 基于非常穩定的 Debian10
• 為鏡像打Label標簽，s2i 構建時會讀取這些標簽
• 復制OracleJDK目錄及s2i腳本
• 配置JDK環境變量、UTF-8字符集、指定中國時區
• 創建部署目錄、創建運行用戶、更換國內源、安裝必要收集性能指標工具（netstat和ps）
• 指定工作目錄、運行用戶、默認暴露的端口、默認的 CMD入口

完成Dockerfile的修改還沒完，我們還需要修改 s2i 構建與運行腳本，以適應剛剛修改的鏡像配置

修改 s2i 腳本

s2i腳本在 s2i/bin 目錄下，我們需要修改 assemble 腳本來改變組裝流程，修改 run 腳本以更換啟動命令。

修改assemble

vim s2i/bin/assemble

#!/bin/bash -e## S2I assemble script for the "oracle-jdk-8" image.# The "assemble" script builds your application source so that it is ready to run.## For more information refer to the documentation:#	https://github.com/openshift/source-to-image/blob/master/docs/builder_image.md## Prevent some horror bugs. example: rm -rf ${SOME_UNEXISTS_VARIABLE}/* set -e# If the "oracle-jdk-8" assemble script is executed with the "-h" flag, print the usage.if [[ "$1" == "-h" ]]; then	exec /usr/libexec/s2i/usagefiecho "---> Moving Artifact from source..."mv /tmp/src/*.jar /deployments/echo "---> Clearing tmp dir..."rm /tmp/src/.git -rfecho "<--- Build Success..."

以上腳本主要做了兩件事，一個是從 /tmp/src 下將 jar 包復制到部署目錄，在之前的文章中介紹過 s2i 的工作流程，在構建時會從 Git 上拉取制品或源碼到 /tmp/src 下；另一個是刪除 /tmp/src/.git 目錄，原因是 Git 的 .git 目錄會同樣保存壓縮后的制品或源碼文件，刪除以減小鏡像體積。

修改run

vim s2i/bin/run

#!/bin/bash -e## S2I run script for the "oracle-jdk-8" image.# The run script executes the server that runs your application.## For more information see the documentation:#	https://github.com/openshift/source-to-image/blob/master/docs/builder_image.md#exec java ${JVM_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /deployments/*.jar

run 腳本只作啟動功能，為了方便傳 JVM 調優參數，定制了 JVM_OPTS 環境變量供使用。參數中的隨機數配置能提高服務生成隨機數的能力，提高啟動速度。

修改 Makefile

修改Makefile的作用是簡化構建鏡像的命令，筆者這里修改了鏡像名以及添加了推送私有鏡像倉庫的參數，這點上 npm 有點類似于 Makefile。

IMAGE_NAME = 192.168.99.1:5000/oracle-jdk-8.PHONY: testtest:	docker build -t $(IMAGE_NAME)-candidate .	IMAGE_NAME=$(IMAGE_NAME)-candidate test/run.PHONY: pushpush:	docker build -t $(IMAGE_NAME) .	docker push $(IMAGE_NAME)

想了解更多 Makefile 寫法，可以參考 左耳朵耗子 陳皓 寫的《跟我一起寫Makefile》

編寫測試腳本

test/run 這個測試腳本相當于完成了一次 s2i 到部署的過程，先準備測試鏡像，再運行s2i的構建過程增量構建（將測試制品程序與鏡像進行組裝，容器內部調用了 assemble 腳本），然后啟動容器，運行定時測試腳本訪問容器端口地址，通過就算成功了。

由于默認測試腳本中測試服務是否可用是訪問地址判斷返回值是否200狀態碼，我這邊提供個springboot的demo 部署后訪問 /test 端點返回結果 ，首先構建下這個demo:

#克隆測試demogit clone https://github.com/hellxz/cicd-demo.git test/test-app#手動構建cd test/test-appmvn package -DskipTests#復制jar包并刪除本文無關的內容mv target/*.jar .rm -rf *.sh *file *.xml *.md src target .git *.yaml .gitignore .mvn#回到Makefile所在目錄cd ../..

修改 test/run 腳本，調整探測間隔與訪問的接口地址，本demo中需修改116行為5，以及120和121行訪問路徑添加 /test

執行測試

make test

我們能看到 springboot 的 banner 下有行字 Starting CicdDemoApplication v0.0.1 using Java 1.8.0_181 on f770c60456f4 with PID 1 (/deployments/cicd-demo-0.0.1.jar started by oraclejdk8 in /deployments)，這可以證明測試的demo的確是以 oraclejdk8 用戶運行成功的，而且 pid 為 1。再往下看也沒有檢測報錯的信息，說明測試通過，可以使用 Minishift測試了。

推送鏡像到倉庫中

#執行Makefile中定義的構建推送命令make push

hellxz@debian:~/WorkSpace/openshift/oracle-jdk-8-catelog$ make pushdocker build -t 192.168.99.1:5000/oracle-jdk-8 .Sending build context to Docker daemon  389.2MBStep 1/12 : FROM debian:buster ---> 2b6f409b1d24Step 2/12 : ENV BUILDER_VERSION 1.0 ---> Using cache ---> 46de72742080Step 3/12 : LABEL description="Source To Image (S2I) image for Hellxz Providing Oracle JDK 8"       maintainer="Hellxz Zhang "       io.k8s.description="Platform for building and running Java applications with Oracle JDK 8"       io.k8s.display-name="Java Applications"       io.openshift.expose-services="8080:http"       io.openshift.tags="builder,java"       io.openshift.s2i.scripts-url="image:///usr/libexec/s2i"       io.openshift.s2i.destination="/tmp" ---> Using cache ---> fcbadb0d0d39Step 4/12 : COPY ./jdk1.8.0_181 /usr/lib/jvm ---> Using cache ---> ebb551cc9d4cStep 5/12 : COPY ./s2i/bin/ /usr/libexec/s2i ---> Using cache ---> 5f08421b9527Step 6/12 : ENV JAVA_HOME="/usr/lib/jvm"     PATH="/usr/lib/jvm/bin:${PATH}"     TZ="Asia/Shanghai" ---> Using cache ---> 3c663a5370c8Step 7/12 : RUN mkdir /deployments &&     useradd -M -d /deployments -u 1001  -s /bin/bash oraclejdk8 &&     chown oraclejdk8 /deployments && chmod 777 /deployments &&     bash -c "echo -e /"deb http://mirrors.163.com/debian/ buster main non-free contrib/ndeb http://mirrors.163.com/debian/ buster-updates main non-free contrib/ndeb http://mirrors.163.com/debian/ buster-backports main non-free contrib/ndeb http://mirrors.163.com/debian-security/ buster/updates main non-free contrib/" > /etc/apt/sources.list" &&     apt-get update -y && apt-get install -y net-tools locales procps &&     apt-get clean all -y && rm -rf /var/lib/apt/lists/* &&     sed -i "/en_US.UTF-8/s/^# //g" /etc/locale.gen && locale-gen ---> Using cache ---> 5e9c181bb69bStep 8/12 : ENV LC_ALL="en_US.UTF-8" ---> Using cache ---> 60091bb63284Step 9/12 : USER 1001 ---> Using cache ---> f8e6a2e96a8aStep 10/12 : WORKDIR /deployments ---> Using cache ---> 66f6925e3ab6Step 11/12 : EXPOSE 8080 8443 ---> Using cache ---> bda814465820Step 12/12 : CMD ["/usr/libexec/s2i/usage"] ---> Using cache ---> f30388d3181cSuccessfully built f30388d3181cSuccessfully tagged 192.168.99.1:5000/oracle-jdk-8:latestdocker push 192.168.99.1:5000/oracle-jdk-8Using default tag: latestThe push refers to repository [192.168.99.1:5000/oracle-jdk-8]16970671df83: Pushed 97a51654d23f: Pushed 16fd63c1b14c: Pushed d6a325d281f2: Pushed latest: digest: sha256:904e52bcfe7c4ea8a156de4bd9cb073770f4af703ac6acb59a9180c272d41b79 size: 1160

導入鏡像到 OpenShift 內部鏡像倉庫中

這里Minishift導入鏡像到內部倉庫和OpenShift是一致的，說是OpenShift也可以。

#導入需要oc命令行工具，minishift默認已經提供了，但未加入到Path中，可以執行如下命令臨時把oc加入Patheval (minishift oc-env) #如果報錯，則只執行括號內命令，根據提示執行相應命令oc import-image 192.168.99.1:5000/oracle-jdk-8:latest --confirm --insecure

hellxz@debian:~/WorkSpace/openshift$ oc import-image 192.168.99.1:5000/oracle-jdk-8:latest --confirm --insecureimagestream.image.openshift.io/oracle-jdk-8 importedName:			oracle-jdk-8Namespace:		myprojectCreated:		1 second agoLabels:			Annotations:		openshift.io/image.dockerRepositoryCheck=2021-11-21T14:24:57ZDocker Pull Spec:	172.30.1.1:5000/myproject/oracle-jdk-8Image Lookup:		local=falseUnique Images:		1Tags:			1latest  tagged from 192.168.99.1:5000/oracle-jdk-8:latest    will use insecure HTTPS or HTTP connections  * 192.168.99.1:5000/oracle-jdk-8@sha256:904e52bcfe7c4ea8a156de4bd9cb073770f4af703ac6acb59a9180c272d41b79      1 second agoImage Name:	oracle-jdk-8:latestDocker Image:	192.168.99.1:5000/oracle-jdk-8@sha256:904e52bcfe7c4ea8a156de4bd9cb073770f4af703ac6acb59a9180c272d41b79Name:		sha256:904e52bcfe7c4ea8a156de4bd9cb073770f4af703ac6acb59a9180c272d41b79Created:	1 second agoAnnotations:	image.openshift.io/dockerLayersOrder=ascendingImage Size:	221.8MB in 4 layersLayers:		50.44MB	sha256:07471e81507f7cf1100827f10c60c3c0422d1222430e34e527d97ec72b14a193		161.8MB	sha256:98b418031cdbc51e52d88a2632516069e753d5293ec440dae46163a45880492a		899B	sha256:61addb0f8207e85f3a734299eda8b8afae987076de0cf67b60f33c9c9846f6b6		9.598MB	sha256:09daea12eb7b9cd76b4629e22d566a6646eb6e19ea3c92706c7bc41bf0285384Image Created:	30 minutes agoAuthor:		Arch:		amd64Command:	/usr/libexec/s2i/usageWorking Dir:	/deploymentsUser:		1001Exposes Ports:	8080/tcp, 8443/tcpDocker Labels:	description=Source To Image (S2I) image for Hellxz Providing Oracle JDK 8		io.k8s.description=Platform for building and running Java applications with Oracle JDK 8		io.k8s.display-name=Java Applications		io.openshift.expose-services=8080:http		io.openshift.s2i.destination=/tmp		io.openshift.s2i.scripts-url=image:///usr/libexec/s2i		io.openshift.tags=builder,java		maintainer=Hellxz Zhang Environment:	PATH=/usr/lib/jvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin		BUILDER_VERSION=1.0		JAVA_HOME=/usr/lib/jvm		TZ=Asia/Shanghai		LC_ALL=en_US.UTF-8

如果報錯，優先考慮是否登錄了oc，oc login -u developer ，執行后再嘗試導入。

自定義s2i模板部分

定義模板文件

這里直接提供模板內容，基于 openjdk8的模板修改而來的：

oraclejdk-8-basic-s2i.yaml

apiVersion: template.openshift.io/v1kind: Templatemessage: A new java application has been created in your project.metadata:  annotations:    description: An Java application using Oracle JDK 8.    iconClass: icon-rh-openjdk    openshift.io/display-name: OracleJDK 8 by Hellxz    openshift.io/provider-display-name: Hellxz Zhang .    tags: java    template.openshift.io/long-description: This template defines resources needed to develop Oracle JDK 8 Java based application.    template.openshift.io/support-url: hellxz001@foxmail.com    version: 0.0.1  labels:    template: oraclejdk-8-basic-s2i    xpaas: 0.0.1  name: oraclejdk-8-basic-s2iobjects:- apiVersion: v1  kind: Service  metadata:    annotations:      description: The application"s http port.    labels:      application: ${APPLICATION_NAME}    name: ${APPLICATION_NAME}  spec:    ports:    - port: 8080      targetPort: 8080    selector:      deploymentConfig: ${APPLICATION_NAME}- apiVersion: v1  id: ${APPLICATION_NAME}-http  kind: Route  metadata:    annotations:      description: Route for application"s http service.    labels:      application: ${APPLICATION_NAME}    name: ${APPLICATION_NAME}  spec:    host: ${HOSTNAME_HTTP}    to:      name: ${APPLICATION_NAME}- apiVersion: v1  kind: ImageStream  metadata:    labels:      application: ${APPLICATION_NAME}    name: ${APPLICATION_NAME}- apiVersion: v1  kind: BuildConfig  metadata:    labels:      application: ${APPLICATION_NAME}    name: ${APPLICATION_NAME}  spec:    output:      to:        kind: ImageStreamTag        name: ${APPLICATION_NAME}:latest    source:      contextDir: ""      git:        ref: ${SOURCE_REPOSITORY_REF}        uri: ${SOURCE_REPOSITORY_URL}      type: Git    strategy:      sourceStrategy:        forcePull: true        from:          kind: ImageStreamTag          name: oracle-jdk-8:latest          namespace: ${IMAGE_STREAM_NAMESPACE}      type: Source    triggers:    - imageChange: {}      type: ImageChange    - type: ConfigChange- apiVersion: v1  kind: DeploymentConfig  metadata:    labels:      application: ${APPLICATION_NAME}    name: ${APPLICATION_NAME}  spec:    replicas: 1    selector:      deploymentConfig: ${APPLICATION_NAME}    strategy:      type: Recreate    template:      metadata:        labels:          application: ${APPLICATION_NAME}          deploymentConfig: ${APPLICATION_NAME}        name: ${APPLICATION_NAME}      spec:        containers:        - env: []          image: ${APPLICATION_NAME}          imagePullPolicy: Always          name: ${APPLICATION_NAME}          ports:          - containerPort: 8778            name: jolokia            protocol: TCP          - containerPort: 8080            name: http            protocol: TCP          - containerPort: 8443            name: https            protocol: TCP        terminationGracePeriodSeconds: 75    triggers:    - imageChangeParams:        automatic: true        containerNames:        - ${APPLICATION_NAME}        from:          kind: ImageStreamTag          name: ${APPLICATION_NAME}:latest      type: ImageChange    - type: ConfigChangeparameters:- description: The name for the application.  displayName: Application Name  name: APPLICATION_NAME  required: true  value: oraclejdk8-app- description: Git source URI for application  displayName: Git Repository URL  name: SOURCE_REPOSITORY_URL  required: true- description: Git branch/tag reference  displayName: Git Reference  name: SOURCE_REPOSITORY_REF  value: master- description: "Custom hostname for http service route.  Leave blank for default hostname,    e.g.: -."  displayName: Custom http Route Hostname  name: HOSTNAME_HTTP- description: Namespace in which the ImageStreams for Red Hat Middleware images are    installed. These ImageStreams are normally installed in the openshift namespace.    You should only need to modify this if you"ve installed the ImageStreams in a    different namespace/project.  displayName: ImageStream Namespace  name: IMAGE_STREAM_NAMESPACE  required: true  value: myproject

模板的結構簡單說明：

• 定義格式基于k8s，是紅帽擴展的 Template API類型
• 源數據 - 記錄模板的內容，會在打開模板的UI上顯示
• objects - 各種構建到部署過程的所有對象定義
  • Service - k8s原生有的服務對象
  • Route - OpenShift獨有的路由對象
  • ImageStream - OpenShift獨有的鏡像流對象
  • BuildConfig - OpenShift獨有的構建配置對象
  • DeploymentConfig - OpenShift獨有的部署配置對象，生成k8s中的Deployment對象
• parameters - 可修改參數，或者變量，供objects中各種對象引用，從而生成相關聯的一系列對象

導入模板到 OpenShift

oc apply -f oraclejdk-8-basic-s2i.yaml

hellxz@debian:~/WorkSpace/openshift$ oc apply -f=oraclejdk-8-basic-s2i.yaml template.template.openshift.io/oraclejdk-8-basic-s2i created

使用自定義模板和鏡像部署 Java 程序

預備工作

由于OpenShift是由Git倉庫拉取制品或源碼進行構建的，所以需要把測試程序（上文中用到的cicd-demo-0.0.1.jar）上傳到 Git 版本控制中，這里延用上篇文章中 Minishift 開發環境中配置的 Gitea（需要預先創建個賬號和倉庫，這里創建的倉庫名為demo，用戶名為hellxz）。

#進入測試程序jar所在目錄cd ~/WorkSpace/openshift/oracle-jdk-8-catelog/test/test-app#初始化當前目錄為git倉庫git init#添加遠程倉庫地址（按實際操作來，變通一些）git remote add origin git remote add origin http://localhost:3000/hellxz/demo.git#設置用戶名和emailgit config --local user.name hellxzgit config --local user.email hellxz001@foxmail.com#提交git add .git commit -m "init"#推送遠程倉庫git push --set-upstream origin master

如下圖，已經推送完畢。

使用自定義模板部署程序

模板中引用的是自定義鏡像的名稱，命名空間也是myproject（Minishift默認的，生產環境可以改成需要的名稱再上傳，調整可多次導入）

首次部署流程比較麻煩，這里分多個動圖錄制

創建應用

Gitea 配置的對外地址配置得有點問題，不過問題不大……

創建拉取Git的secret

需要告知s2i構建腳本 Git 倉庫的用戶和密碼，這樣才能拉得下來制品，僅首次創建應用時需要創建。

修改Build配置yaml，添加 Git 倉庫 secret 引用

執行構建，查看構建日志

查看測試程序運行狀態

如圖，容器運行已經是 Ready: true 狀態，說明啟動狀態正常。

使用域名訪問服務看看

由于沒有外部的dns，域名可以修改 /etc/hosts 文件代替

測試通過，自定義鏡像和模板都能正常工作。

寫作不易，如果本文對您有所幫助，就點個關注點個贊再走唄~ 我是 Hellxz，我們下次再見。

本文同步發布以下兩個地址，未經許可禁止轉載。

• 博客園 https://www.cnblogs.com/hellxz
• CSDN https://blog.csdn.net/u012586326