一. gin_scaffold 企业级脚手架
- 为什么使用脚手架,防止重复造轮子,只关注业务开发即可
- gin_scaffold 优点:
- 提供了能够覆盖mysql/redis/request的链路功能,日志功能
- 支持多语言错误信息提示以及自定义错误提示
- 支持多环境配置
- 封装了log/redis/mysql/http.client常用方法等
- 支持swagger文档
二. gin_scaffold 脚手架安装及使用演示
- gin_scaffold地址: https://github.com/e421083458/gin_scaffold
文件分层解释
├── README.md
├── conf 配置文件夹
│ └── dev 开发环境配置文件(多环境是创建对应其它环境的配置文件)
│ ├── base.toml
│ ├── mysql_map.toml
│ └── redis_map.toml
├── controller 控制器
│ └── demo.go
├── dao DB数据层
│ └── demo.go
├── docs swagger文件层
├── dto 输入输出结构层
│ └── demo.go
├── go.mod
├── go.sum
├── main.go 入口文件
├── middleware 中间件层
│ ├── panic.go
│ ├── response.go
│ ├── token_auth.go
│ └── translation.go
├── public 公共文件
│ ├── log.go
│ ├── mysql.go
│ └── validate.go
└── router 路由层
│ ├── httpserver.go
│ └── route.go
└── services 逻辑处理层
开始使用
1. 配置开启go mod 功能
参考:go 基础入门二十九 go mod
2. 下载 安装 gin_scaffold
- 执行clone命令下载gin_scaffold到本地,地址: git clone git@github.com:e421083458/gin_scaffold.git
- 下载完毕后跳转到下载的gin_scaffold目录下,
- 执行"go mod tidy"命令,安装gin_scaffold相关类库
- 使用ide编辑工具打开下载的gin_scaffold,找到"conf/mysql_map.toml","conf/redis_map.toml"配置文件,配置连接自己的mysql,redis
- 执行go run main.go 启动测试一下
实际就是拉取一个空的服务baes,测试启动一下,如果能够启动,在这个base上进行业务开发
3. 整合 golang_common
- gin_scaffold 下整合了golang_common包,在该包中提供了一下功能:
- 支持多环境运行设置,比如:dev、prod。
- 支持mysql、redis 多套数据源配置。
- 支持默认和自定义日志实例,自动滚动日志。
- 封装了 mysql(gorm.io/gorm v1.22.4)、redis(redigo)、http.client整合及常用方法
- 支持 mysql(gorm.io/gorm v1.22.4)、redis(redigo)、http.client 请求链路日志输出
- 官方参考文档
- 注意在使用golang_common中提供的相关方法时,对应组件可能要提供对应的配置文件,要将这些配置文件放到项目中,官方提供了相关配置文件示例
- 实际就是gin_scaffold 整合了一个golang_common工具包,这个工具包中封装了一些针对mysql,redis,httpclient的相关方法,整合了以后,如果在项目中使用mysql,redis时,直接通过golang_common中提供的方法使用即可,在golang_common中整合mysql,redis,httpclient时可能会用到相关的配置文件,将对应的配置文件放到项目中即可
- 注意: 如果拉下来的gin_scaffold脚手架项目中没有golang_common这个包,或者已经存在的项目后续开发中整合golang_common时:
- 项目中创建golang_common文件夹,访问官方文档地址,获取lib,log放到新建的golang_common文件夹下,
- 获取相关配置文件,放入conf目录下,
- 执行 “go get -v github.com/e421083458/golang_common” 拉取golang_common包
- 使用golang_common下提供的方法即可
4. 测试链路
- 执行sql脚本,创建测试数据
CREATE TABLE `area` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`area_name` varchar(255) NOT NULL,
`city_id` int(11) NOT NULL,
`user_id` int(11) NOT NULL,
`update_at` datetime NOT NULL,
`create_at` datetime NOT NULL,
`delete_at` datetime NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8 COMMENT='area';
INSERT INTO `area` (`id`, `area_name`, `city_id`, `user_id`, `update_at`, `create_at`, `delete_at`) VALUES (NULL, 'area_name', '1', '2', '2019-06-15 00:00:00', '2019-06-15 00:00:00', '2019-06-15 00:00:00');
- 请求接口,查看响应,其中"trace_id"就是链路唯一id
curl 'http://127.0.0.1:8880/demo/dao?id=1'
{
"errno": 0,
"errmsg": "",
"data": "[{\"id\":1,\"area_name\":\"area_name\",\"city_id\":1,\"user_id\":2,\"update_at\":\"2019-06-15T00:00:00+08:00\",\"create_at\":\"2019-06-15T00:00:00+08:00\",\"delete_at\":\"2019-06-15T00:00:00+08:00\"}]",
"trace_id": "c0a8fe445d05b9eeee780f9f5a8581b0"
}
- 查看链路日志
//gin_scaffold.inf.log下存放了所有正常info级别日志
tail -f gin_scaffold.inf.log
//gin_scaffold.wf.log下存放了所有的异常日志
tail -f gin_scaffold.wf.log
5. swagger文档生成
- 下载对应操作系统的执行文件到
G
O
P
A
T
H
/
b
i
n
下面
(
下载后解压获取到
b
i
n
文件夹下的
"
s
w
a
g
"
文件
,
复制到
GOPATH/bin下面(下载后解压获取到bin文件夹下的"swag"文件,复制到
GOPATH/bin下面(下载后解压获取到bin文件夹下的"swag"文件,复制到GOPATH/bin下)
//如
➜ gin_scaffold git:(master) ✗ ll -r $GOPATH/bin
total 434168
-rwxr-xr-x 1 niuyufu staff 13M 4 3 17:38 swag
- 编写接口,增加swagger注释,参考一下示例
// ListPage godoc
// @Summary 测试数据绑定
// @Description 测试数据绑定
// @Tags 用户
// @ID /demo/bind
// @Accept json
// @Produce json
// @Param polygon body dto.DemoInput true "body"
// @Success 200 {object} middleware.Response{data=dto.DemoInput} "success"
// @Router /demo/bind [post]
- 执行命令生成接口文档: swag init, 生成docs文件夹,生成swagger相关文件
- 启动服务, swagger整合完毕后访问接口文档页面: “http://127.0.0.1:8880/swagger/index.html”
- gin 集成swagger参考博客
- swagger命令报错问题
6. gin_scaffold 下conf 下配置文件
- base.toml 配置文件
# This is base config
[base]
debug_mode="debug"
time_location="Asia/Chongqing"
[http]
addr =":8880" # 监听地址, default ":8700"
read_timeout = 10 # 读取超时时长
write_timeout = 10 # 写入超时时长
max_header_bytes = 20 # 最大的header大小,二进制位长度
allow_ip = [ # 白名单ip列表
"127.0.0.1",
"192.168.1.1"
]
[session]
redis_server = "127.0.0.1:6379" #redis session server
redis_password = ""
[log]
log_level = "trace" #日志打印最低级别
[log.file_writer] #文件写入配置
on = true
log_path = "./logs/go_gateway.inf.log"
rotate_log_path = "./logs/go_gateway.inf.log.%Y%M%D%H"
wf_log_path = "./logs/go_gateway.wf.log"
rotate_wf_log_path = "./logs/go_gateway.wf.log.%Y%M%D%H"
[log.console_writer] #工作台输出
on = false
color = false
[cluster]
cluster_ip="127.0.0.1"
cluster_port="8080"
cluster_ssl_port="4433"
[swagger]
title="go_gateway swagger API"
desc="This is a sample server celler server."
host="127.0.0.1:8880"
base_path=""
- mysql_map.toml配置文件
# this is mysql config
[list]
[list.default]
driver_name = "mysql"
data_source_name = "root:123456@tcp(127.0.0.1:3306)/go_gateway2?charset=utf8&parseTime=true&loc=Asia%2FChongqing"
max_open_conn = 20
max_idle_conn = 10
max_conn_life_time = 100
- redis_map.toml 配置文件
# this is redis config file
[list]
[list.default]
proxy_list = ["127.0.0.1:6379"]
conn_timeout = 500
password = ""
db = 0
read_timeout = 1000
write_timeout = 1000
max_active = 200
max_idle = 500
7. gin_scaffold main方法启动服务解释
- 初始化路由,启动服务方法,与关闭服务方法
import (
"context"
"go_test/golang_common/lib"
"github.com/gin-gonic/gin"
"log"
"net/http"
"time"
)
var (
HttpSrvHandler *http.Server
)
//1.初始化路由,以及监听端口启动服务
func HttpServerRun() {
//1.默认在debug下运行
gin.SetMode(lib.GetStringConf("base.base.debug_mode"))
//2.初始化路由(自定义函数在下方)
r := InitRouter()
//3.创建server服务
HttpSrvHandler = &http.Server{
Addr: lib.GetStringConf("base.http.addr"),
Handler: r,
ReadTimeout: time.Duration(lib.GetIntConf("base.http.read_timeout")) * time.Second,
WriteTimeout: time.Duration(lib.GetIntConf("base.http.write_timeout")) * time.Second,
MaxHeaderBytes: 1 << uint(lib.GetIntConf("base.http.max_header_bytes")),
}
//4.启动服务
go func() {
log.Printf(" [INFO] HttpServerRun:%s\n",lib.GetStringConf("base.http.addr"))
if err := HttpSrvHandler.ListenAndServe(); err != nil {
log.Fatalf(" [ERROR] HttpServerRun:%s err:%v\n", lib.GetStringConf("base.http.addr"), err)
}
}()
}
//2.关闭服务方法
func HttpServerStop() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := HttpSrvHandler.Shutdown(ctx); err != nil {
log.Fatalf(" [ERROR] HttpServerStop err:%v\n", err)
}
log.Printf(" [INFO] HttpServerStop stopped\n")
}
- main
import (
"flag"
"go_test/golang_common/lib"
"go_test/router"
"os"
"os/signal"
"syscall"
)
func main(){
//1.设置环境变量,如果第一个参数configPath为空,则会通过命令行读取"-config=./conf/prod/"下的文件
//整合是注意此处的路径是否正确
config := flag.String("config", "./config/dev/", "input config file like ./config/dev/")
//lib是golang_common包下的
lib.InitModule(*config)
//2.退出
defer lib.Destroy()
//3.调用上面的初始化router,开启服务方法
router.HttpServerRun()
//4.监听系统的停止信号(也就是监听syscall.SIGINT与syscall.SIGTERM)
quit:=make(chan os.Signal)
signal.Notify(quit,syscall.SIGINT, syscall.SIGTERM)
<-quit
//5.当拿到监听的信号后说明需要关闭服务,调用上面的关闭服务方法
router.HttpServerStop()
}
8. gin_scaffold 中初始化路由简单解释
func InitRouter(middlewares ...gin.HandlerFunc) *gin.Engine {
//读取conf下配置文件中自定义配置数据,
//通过gin官方提供的NewRedisStore,在redis获取session
//store, err := sessions.NewRedisStore(10, "tcp", lib.GetStringConf("base.session.redis_server"), lib.GetStringConf("base.session.redis_password"), []byte("secret"))
//if err != nil {
// log.Fatalf("sessions.NewRedisStore err:%v", err)
//}
//1.设置swagger相关
docs.SwaggerInfo.Title = lib.GetStringConf("base.swagger.title")
docs.SwaggerInfo.Description = lib.GetStringConf("base.swagger.desc")
docs.SwaggerInfo.Version = "1.0"
docs.SwaggerInfo.Host = lib.GetStringConf("base.swagger.host")
docs.SwaggerInfo.BasePath = lib.GetStringConf("base.swagger.base_path")
docs.SwaggerInfo.Schemes = []string{"http", "https"}
//2.获取到默认的router
router := gin.Default()
router.Use(middlewares...)
//3.绑定默认提供的/ping接口
router.GET("/ping", func(c *gin.Context) {
c.JSON(200, gin.H{
"message": "pong",
})
})
//4.绑定swagger相关接口
router.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler))
//5.设置group前缀
adminLoginRouter := router.Group("/admin_login")
//路由注册
adminLoginRouter.Use( //设置中间件(下方只是示例)
//sessions.Sessions("mysession", store), //session相关中间件
middleware.RecoveryMiddleware(), //用来捕获panic异常的中间件
middleware.RequestLog(), //打印链路日志相关的中间件
middleware.IPAuthMiddleware(), //权限相关中间件
middleware.SessionAuthMiddleware(), //校验session是否登录的中间件
middleware.TranslationMiddleware()) //多语言翻译中间件
{
//6.调用controller下的接口
controller.AdminLoginRegister(adminLoginRouter)
}
}
9. gin_scaffold 中controller简单示例
import (
"encoding/json"
"fmt"
"github.com/e421083458/go_gateway/dao"
"github.com/e421083458/go_gateway/dto"
"github.com/e421083458/go_gateway/middleware"
"github.com/e421083458/go_gateway/public"
"github.com/e421083458/go_gateway/golang_common/lib"
"github.com/gin-gonic/contrib/sessions"
"github.com/gin-gonic/gin"
)
//1.创建controller结构体
type AdminController struct{}
//2.AdminController 接口体上绑定的业务方法(下方的注释时swagger相关的)
// AdminInfo godoc
// @Summary 管理员信息
// @Description 管理员信息
// @Tags 管理员接口
// @ID /admin/admin_info
// @Accept json
// @Produce json
// @Success 200 {object} middleware.Response{data=dto.AdminInfoOutput} "success"
// @Router /admin/admin_info [get]
func (adminlogin *AdminController) AdminInfo(c *gin.Context) {
//1.获取session
sess := sessions.Default(c)
sessInfo := sess.Get(public.AdminSessionInfoKey)
//2.在session中读取接收到的数据转换为结构体
adminSessionInfo := &dto.AdminSessionInfo{}
if err := json.Unmarshal([]byte(fmt.Sprint(sessInfo)), adminSessionInfo); err != nil {
middleware.ResponseError(c, 2000, err)
return
}
//3.设置响应
out := &dto.AdminInfoOutput{
ID: adminSessionInfo.ID,
Name: adminSessionInfo.UserName,
}
middleware.ResponseSuccess(c, out)
}
//业务方法2
// ChangePwd godoc
// @Summary 修改密码
// @Description 修改密码
// @Tags 管理员接口
// @ID /admin/change_pwd
// @Accept json
// @Produce json
// @Param body body dto.ChangePwdInput true "body"
// @Success 200 {object} middleware.Response{data=string} "success"
// @Router /admin/change_pwd [post]
func (adminlogin *AdminController) ChangePwd(c *gin.Context) {
//1.读取并校验请求参数
params := &dto.ChangePwdInput{}
if err := params.BindValidParam(c); err != nil {
middleware.ResponseError(c, 2000, err)
return
}
//2.在session读取用户信息到结构体
sess := sessions.Default(c)
sessInfo := sess.Get(public.AdminSessionInfoKey)
adminSessionInfo := &dto.AdminSessionInfo{}
if err := json.Unmarshal([]byte(fmt.Sprint(sessInfo)), adminSessionInfo); err != nil {
middleware.ResponseError(c, 2000, err)
return
}
//3.从数据库中读取 adminInfo
//进行数据库操作时,先获取到默认的gorm连接池
tx, err := lib.GetGormPool("default")
if err != nil {
middleware.ResponseError(c, 2001, err)
return
}
//执行gorm操作
adminInfo := &dao.Admin{}
adminInfo, err = adminInfo.Find(c, tx, (&dao.Admin{UserName: adminSessionInfo.UserName}))
if err != nil {
middleware.ResponseError(c, 2002, err)
return
}
//生成新密码 saltPassword
saltPassword := public.GenSaltPassword(adminInfo.Salt, params.Password)
adminInfo.Password = saltPassword
//执行数据保存
if err := adminInfo.Save(c, tx); err != nil {
middleware.ResponseError(c, 2003, err)
return
}
middleware.ResponseSuccess(c, "")
}
//3.当前controller注册到Group(也就是controller中第5步骤调用的方法)
func AdminRegister(group *gin.RouterGroup) {
adminLogin := &AdminController{}
group.GET("/admin_info", adminLogin.AdminInfo)
group.POST("/change_pwd", adminLogin.ChangePwd)
}
10. gin_scaffold 操作数据库的dao示例
import (
"github.com/e421083458/go_gateway/dto"
"github.com/e421083458/go_gateway/public"
"github.com/e421083458/gorm"
"github.com/gin-gonic/gin"
"github.com/pkg/errors"
"time"
)
type Admin struct {
Id int `json:"id" gorm:"primary_key" description:"自增主键"`
UserName string `json:"user_name" gorm:"column:user_name" description:"管理员用户名"`
Salt string `json:"salt" gorm:"column:salt" description:"盐"`
Password string `json:"password" gorm:"column:password" description:"密码"`
UpdatedAt time.Time `json:"update_at" gorm:"column:update_at" description:"更新时间"`
CreatedAt time.Time `json:"create_at" gorm:"column:create_at" description:"创建时间"`
IsDelete int `json:"is_delete" gorm:"column:is_delete" description:"是否删除"`
}
func (t *Admin) TableName() string {
return "gateway_admin"
}
func (t *Admin) LoginCheck(c *gin.Context, tx *gorm.DB, param *dto.AdminLoginInput) (*Admin, error) {
adminInfo, err := t.Find(c, tx, (&Admin{UserName: param.UserName, IsDelete: 0}))
if err != nil {
return nil, errors.New("用户信息不存在")
}
saltPassword := public.GenSaltPassword(adminInfo.Salt, param.Password)
if adminInfo.Password != saltPassword {
return nil, errors.New("密码错误,请重新输入")
}
return adminInfo, nil
}
func (t *Admin) Find(c *gin.Context, tx *gorm.DB, search *Admin) (*Admin, error) {
out := &Admin{}
err := tx.SetCtx(public.GetGinTraceContext(c)).Where(search).Find(out).Error
if err != nil {
return nil, err
}
return out, nil
}
func (t *Admin) Save(c *gin.Context, tx *gorm.DB) error {
return tx.SetCtx(public.GetGinTraceContext(c)).Save(t).Error
}
11. gin_scaffold/middleware/下的中间件示例
- 捕获panic异常的中间件示例
import (
"errors"
"fmt"
"github.com/e421083458/go_gateway/public"
"github.com/e421083458/go_gateway/golang_common/lib"
"github.com/gin-gonic/gin"
"runtime/debug"
)
// RecoveryMiddleware捕获所有panic,并且返回错误信息
func RecoveryMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
defer func() {
if err := recover(); err != nil {
//先做一下日志记录
fmt.Println(string(debug.Stack()))
public.ComLogNotice(c, "_com_panic", map[string]interface{}{
"error": fmt.Sprint(err),
"stack": string(debug.Stack()),
})
if lib.ConfBase.DebugMode != "debug" {
ResponseError(c, 500, errors.New("内部错误"))
return
} else {
ResponseError(c, 500, errors.New(fmt.Sprint(err)))
return
}
}
}()
c.Next()
}
}
- 打印请求日志中间件示例
import (
"bytes"
"github.com/e421083458/go_gateway/public"
"github.com/e421083458/go_gateway/golang_common/lib"
"github.com/gin-gonic/gin"
"io/ioutil"
"time"
)
// 请求进入日志
func RequestInLog(c *gin.Context) {
traceContext := lib.NewTrace()
if traceId := c.Request.Header.Get("com-header-rid"); traceId != "" {
traceContext.TraceId = traceId
}
if spanId := c.Request.Header.Get("com-header-spanid"); spanId != "" {
traceContext.SpanId = spanId
}
c.Set("startExecTime", time.Now())
c.Set("trace", traceContext)
bodyBytes, _ := ioutil.ReadAll(c.Request.Body)
c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(bodyBytes)) // Write body back
lib.Log.TagInfo(traceContext, "_com_request_in", map[string]interface{}{
"uri": c.Request.RequestURI,
"method": c.Request.Method,
"args": c.Request.PostForm,
"body": string(bodyBytes),
"from": c.ClientIP(),
})
}
// 请求输出日志
func RequestOutLog(c *gin.Context) {
// after request
endExecTime := time.Now()
response, _ := c.Get("response")
st, _ := c.Get("startExecTime")
startExecTime, _ := st.(time.Time)
public.ComLogNotice(c, "_com_request_out", map[string]interface{}{
"uri": c.Request.RequestURI,
"method": c.Request.Method,
"args": c.Request.PostForm,
"from": c.ClientIP(),
"response": response,
"proc_time": endExecTime.Sub(startExecTime).Seconds(),
})
}
func RequestLog() gin.HandlerFunc {
return func(c *gin.Context) {
//todo 优化点4
if lib.GetBoolConf("base.log.file_writer.on") {
RequestInLog(c)
defer RequestOutLog(c)
}
c.Next()
}
}
- 封装响应的中间件示例
package middleware
import (
"encoding/json"
"fmt"
"github.com/e421083458/go_gateway/golang_common/lib"
"github.com/gin-gonic/gin"
"strings"
)
type ResponseCode int
//1000以下为通用码,1000以上为用户自定义码
const (
SuccessCode ResponseCode = iota
UndefErrorCode
ValidErrorCode
InternalErrorCode
InvalidRequestErrorCode ResponseCode = 401
CustomizeCode ResponseCode = 1000
GROUPALL_SAVE_FLOWERROR ResponseCode = 2001
)
type Response struct {
ErrorCode ResponseCode `json:"errno"`
ErrorMsg string `json:"errmsg"`
Data interface{} `json:"data"`
TraceId interface{} `json:"trace_id"`
Stack interface{} `json:"stack"`
}
func ResponseError(c *gin.Context, code ResponseCode, err error) {
trace, _ := c.Get("trace")
traceContext, _ := trace.(*lib.TraceContext)
traceId := ""
if traceContext != nil {
traceId = traceContext.TraceId
}
stack := ""
if c.Query("is_debug") == "1" || lib.GetConfEnv() == "dev" {
stack = strings.Replace(fmt.Sprintf("%+v", err), err.Error()+"\n", "", -1)
}
resp := &Response{ErrorCode: code, ErrorMsg: err.Error(), Data: "", TraceId: traceId, Stack: stack}
c.JSON(200, resp)
response, _ := json.Marshal(resp)
c.Set("response", string(response))
c.AbortWithError(200, err)
}
func ResponseSuccess(c *gin.Context, data interface{}) {
trace, _ := c.Get("trace")
traceContext, _ := trace.(*lib.TraceContext)
traceId := ""
if traceContext != nil {
traceId = traceContext.TraceId
}
resp := &Response{ErrorCode: SuccessCode, ErrorMsg: "", Data: data, TraceId: traceId}
c.JSON(200, resp)
response, _ := json.Marshal(resp)
c.Set("response", string(response))
}
12. 将gin_scaffold脚手架修改为自己的指定服务
- 复制一个gin_scaffold副本出来,重命名为自己的项目,
- 新项目中去除拉取gin_scaffold是的git文件,以及".ide"一类的相关文件
- 通过ide开发工具打开, 设置go module
- 将原项目中gin_scaffold文件名,文件路径等等全局替换掉
- 去除一下无用的controller,dao等等