github workflow发布多平台go二进制包并自动上传s3

前言

因工作需要,采用goreleaser进行多操作系统、平台架构的二进制包发布并且自动上传s3

安装goreleaser进行本地调试

本地安装配置

直接用github workflow来调试,代价比较高。参考官方文档安装本地的goreleaser

1
2
3
## mac安装方法如下
brew install goreleaser/tap/goreleaser

可以用goreleaser init初始化一个demo配置,我的.goreleaser.yaml配置如下,功能比较简单,生成mac、linux的x86、arm架构的二进制包。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# ref: https://goreleaser.com/customization/archive/
before:
hooks:
- go mod tidy

builds:
### 指定main的路径,从.goreleaser.yaml所在的路径开始
- main: "./pkg/cmd/cli/main.go"
binary: amq-installer
goos:
- darwin
- linux
goarch:
- amd64
- arm64
env:
- CGO_ENABLED=0

checksum:
name_template: "{{ .ProjectName }}_checksums.txt"

archives:
- name_template: "{{ .ProjectName }}_{{ .Os }}_{{ .Arch }}"

snapshot:
name_template: "{{ .Tag }}-snapshot"

本地调试

掌握以下命令就可以完成基本的调试工作了,验证本地生成的二进制包是否有效即可

1
2
3
4
5
6
7
8
9
## 检查.goreleaser.yaml配置的有效性
goreleaser check

### 不正式发release包,执行下流程,看看有没有一些执行错误
goreleaser release --snapshot --clean

### 本地发布包测试
goreleaser build --single-target

调试s3-sync-action

上传s3可以用s3-synac-action。我使用的配置如下,本地调试直接硬编码ak/sk信息即可展开测试:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
name: Upload Website

on:
push:
branches:
- master

jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- uses: jakejarvis/s3-sync-action@master
with:
args: --acl public-read --follow-symlinks --delete
env:
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: 'us-west-1' # optional: defaults to us-east-1
SOURCE_DIR: 'public' # optional: defaults to entire repository

官方给的例子设置了参数 --acl public-read,这个不是使用s3的最佳实践,默认创建的bucket是禁用ACL的,如果不涉及跨账户的访问,建议不开启ACL,去除这边的–acl pubic-read以免产生如下报错:

1
An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs

上传的对象需要公网访问内容,可以增加如下的桶policy(改成自己的bucketName)

1
2
3
4
5
6
7
8
9
10
11
12
 {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck20150319",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": ["arn:aws-cn:s3:::YourBucketName","arn:aws-cn:s3:::YourBucketName/*"]
}
]
}

整合s3-sync-action和goreleaser action以及交叉编译

因为我的项目用到go-sqlite,需要开启CGO,不过goreleaser本身不支持,不过我们可以参考bytebase的实践经验[6],在macos和linux的runner上分别运行自己job,然后一起release即可。下面是我的一个配置,没有触发release(开了snapshot),只是构建出二进制包后自动上传到S3方便分发。

  • workflow的配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
name: release installer and upload s3

on:
create:
tags:
- v*

permissions:
contents: write

jobs:
goreleaser-for-linux:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
name: Checkout
with:
fetch-depth: 0
- uses: actions/setup-go@v4
name: Set up Go
- uses: goreleaser/goreleaser-action@v5
name: Run GoReleaser
with:
# either 'goreleaser' (default) or 'goreleaser-pro'
distribution: goreleaser
version: latest
args: release --clean --snapshot --config=.goreleaser-for-linux.yaml
## 必须设置子目录,否则后续执行action会找不到go.mod等文件导致.goreleaser.yaml没法正常工作
workdir: ./amq-installer
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Your GoReleaser Pro key, if you are using the 'goreleaser-pro' distribution
# GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
- uses: jakejarvis/s3-sync-action@master
name: s3-upload
with:
args: --follow-symlinks --delete
env:
AWS_S3_BUCKET: ${{ secrets.AWS_CN_INSTALLER_RELEASE_BUCKET_NAME }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_CN_INSTALLER_RELEASE_AK }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_CN_INSTALLER_RELEASE_SK }}
AWS_REGION: 'cn-northwest-1' # optional: defaults to us-east-1
SOURCE_DIR: 'amq-installer/dist' # optional: defaults to entire repository

goreleaser-for-mac:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
name: Checkout
with:
fetch-depth: 0
- uses: actions/setup-go@v4
name: Set up Go
- uses: goreleaser/goreleaser-action@v5
name: Run GoReleaser
with:
# either 'goreleaser' (default) or 'goreleaser-pro'
distribution: goreleaser
version: latest
args: release --clean --snapshot --config=.goreleaser-for-darwin.yaml
## 必须设置子目录,否则后续执行action会找不到go.mod等文件导致.goreleaser.yaml没法正常工作
workdir: ./amq-installer
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
## ref: https://github.com/jakejarvis/s3-sync-action/issues/54
## github worker runner自带aws cli
- name: Upload S3
run: aws s3 sync . s3://${{ secrets.AWS_CN_INSTALLER_RELEASE_BUCKET_NAME }}/ --no-progress
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_CN_INSTALLER_RELEASE_AK }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_CN_INSTALLER_RELEASE_SK }}
AWS_REGION: 'cn-northwest-1'
working-directory: amq-installer/dist
  • 针对mac的goreleaser配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# ref: https://goreleaser.com/customization/archive/
before:
hooks:
## 在workflow配置的workdir下执行
- go mod tidy

builds:
### 指定main的路径,从.goreleaser.yaml所在的路径开始
- main: "./pkg/cmd/cli/main.go"
binary: amq-installer
goos:
- darwin
goarch:
- amd64
- arm64
env:
- VERSION="development"
- GO_VERSION="1.21.3"
- CGO_ENABLED=1
- GIT_COMMIT="unknown"
- BUILD_TIME="unknown"
- BUILD_USER="unknown"

checksum:
name_template: "{{ .ProjectName }}_checksums.txt"

archives:
- name_template: "{{ .ProjectName }}_{{ .Os }}_{{ .Arch }}"

snapshot:
name_template: "{{ .Tag }}-snapshot"

  • 针对Linux的goreleaser配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# ref: https://goreleaser.com/customization/archive/
# 参考bytebase交叉编译的实践 https://blog.csdn.net/Bytebase/article/details/126020247
before:
hooks:
## 在workflow配置的workdir下执行
- go mod tidy

builds:
### 指定main的路径,从.goreleaser.yaml所在的路径开始
- main: "./pkg/cmd/cli/main.go"
binary: amq-installer
goos:
- linux
# We currently cannot run mysqlbinlog on arm64.
goarch:
- amd64
overrides:
- goos: linux
goarch: amd64
env:
- CC=gcc
env:
- VERSION="development"
- GO_VERSION="1.21.3"
- CGO_ENABLED=1
- GIT_COMMIT="unknown"
- BUILD_TIME="unknown"
- BUILD_USER="unknown"

checksum:
name_template: "{{ .ProjectName }}_checksums.txt"

archives:
- name_template: "{{ .ProjectName }}_{{ .Os }}_{{ .Arch }}"

snapshot:
name_template: "{{ .Tag }}-snapshot"

参考资料