参考原文档:在 AlertManager 报警通知中展示监控图表Promoter 是一个用于 AlertManager 通知的 Webhooks 实现,支持在消息通知中展示实时报警图表,也支持定制消息通知模板。https://mp.weixin.qq.com/s/Wcp7ltEbnHpUlbaF9JDgZg项目源码:GitHub - cnych/promoter: AlertManager Webhook Service With Alerts images.https://github.com/cnych/promoter该项目能够实现在Altermanager报警通知中展示监控的图表,这里用到一个promoter程序,作为接收Altermanager告警信息的Webhook,这里的核心部分是如何渲染监控图表,通过 Prometheus 的API接口来获取查询的指标数据,然后将获取的指标绘制出来,上传到Amazon S3(或者阿里云 OSS 也可以),此外消息通知展示样式支持模板定制,该功能参考自项目 https://github.dev/timonwong/prometheus-webhook-dingtalk,默认模板位于 template/default.tmpl,也可以根据自己需求定制通知模板。
部署:
部署源文件参考:https://github.com/cnych/promoter/blob/main/deploy/kubernetes/promoter.yaml,以下是我的部署Yaml:
$ cat promoter.yml
apiVersion: v1
kind: ConfigMap
metadata:
name: promoter-conf
namespace: kube-mon
data:
config.yaml: |
global:
prometheus_url: # prometehus的url(http://10.20.30.40:9090)
dingtalk_api_token: # 钉钉群机器人的token
dingtalk_api_secret: # 钉钉群机机器人的secret
s3:
access_key: # 云平台阿里云或AWS的 <ak>
secret_key: # 云平台阿里云或AWS的 <sk>
endpoint: # 对象存储访问的endpoint 我用的是AWS上的S3(s3.cn-north-1.amazonaws.com.cn)
region: # 地域 (cn-north-1)
bucket: # 存储桶的名称
receivers:
- name: rcv1 #接收人
dingtalk_configs:
- message_type: markdown
at:
isAtAll: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: promoter
namespace: kube-mon
labels:
app: promoter
spec:
selector:
matchLabels:
app: promoter
template:
metadata:
labels:
app: promoter
spec:
volumes:
- name: config
configMap:
name: promoter-conf
- name: timezone
hostPath:
path: /etc/localtime
containers:
- name: promoter
image: cnych/promoter:main
imagePullPolicy: IfNotPresent
command: ["/bin/sh","-ce","/bin/promoter --config.file=/etc/promoter/config.yaml"] # 启动promoter
ports:
- containerPort: 8080
volumeMounts:
- mountPath: "/etc/promoter"
name: config
- mountPath: /etc/localtime
name: timezone
readOnly: true
---
apiVersion: v1
kind: Service
metadata:
name: promoter
namespace: kube-mon
labels:
app: promoter
spec:
selector:
app: promoter
ports:
- port: 8080
$ kubectl apply -f promoter.yaml
promoter部署成功后,给 AlertManager 配置一个 webhook ,在AlertManager的配置中增加一个路由接收器,指定 Webhook 地址即可,这里配置地址为:http://promoter:8080/rcv1/send,这个地址就是上面部署的 webhook 的接收 Service 地址:
apiVersion: v1
kind: ConfigMap
metadata:
name: alert-config
namespace: kube-mon
data:
config.yml: |-
global:
resolve_timeout: 5m
smtp_smarthost: 'smtp.163.com:465'
smtp_from: 'xxxxxxxxxx'
smtp_auth_username: 'xxxxxxxxxx'
smtp_auth_password: 'xxxxxxxxxx'
smtp_hello: '163.com'
smtp_require_tls: false
route:
group_by: ['alertname']
group_wait: 30s
group_interval: 30s
repeat_interval: 1h
receiver: default
routes:
- receiver: email
group_wait: 10s
match:
team: other
- receiver: webhook
group_wait: 10s
match:
team: node
receivers:
- name: 'default'
email_configs:
- to: 'xxxxxxxxxx@qq.com'
send_resolved: true
- name: 'email'
email_configs:
- to: 'xxxxxxxxxx@qq.com'
send_resolved: true
- name: 'webhook'
webhook_configs:
- url: 'http://promoter:8080/rcv1/send'
send_resolved: true
这会儿就都部署完了,触发告警后就可以收到告警通知了,通知效果如下:
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)