要将警报发送到您的 gmail 帐户,您需要在文件中设置警报管理器配置,例如alertmanager.yaml:
cat <<EOF > alertmanager.yml
route:
group_by: [Alertname]
# Send all notifications to me.
receiver: email-me
receivers:
- name: email-me
email_configs:
- to: $GMAIL_ACCOUNT
from: $GMAIL_ACCOUNT
smarthost: smtp.gmail.com:587
auth_username: "$GMAIL_ACCOUNT"
auth_identity: "$GMAIL_ACCOUNT"
auth_password: "$GMAIL_AUTH_TOKEN"
EOF
现在,当您使用 kube-prometheus 时,您将拥有一个名为alertmanager-main
这是默认配置alertmanager
。你需要创建一个秘密alertmanager-main
再次使用以下命令使用新配置:
kubectl create secret generic alertmanager-main --from-file=alertmanager.yaml -n monitoring
现在,您的警报管理器已设置为在收到来自普罗米修斯的警报时发送电子邮件。
现在您需要设置一个警报,您的邮件将根据该警报发送。您可以设置 DeadManSwitch 警报,该警报在每种情况下都会触发,并用于检查您的警报管道
groups:
- name: meta
rules:
- alert: DeadMansSwitch
expr: vector(1)
labels:
severity: critical
annotations:
description: This is a DeadMansSwitch meant to ensure that the entire Alerting
pipeline is functional.
summary: Alerting DeadMansSwitch
之后DeadManSwitch
警报将被触发,并应向您的邮箱发送电子邮件。
参考链接:
https://coreos.com/tectonic/docs/latest/tectonic-prometheus-operator/user-guides/configuring-prometheus-alertmanager.html https://coreos.com/tectonic/docs/latest/tectonic-prometheus-operator/user-guides/configuring-prometheus-alertmanager.html
EDIT:
deadmanswitch 警报应该放在普罗米修斯正在读取的配置映射中。我将在这里分享我的普罗米修斯的相关快照:
"spec": {
"alerting": {
"alertmanagers": [
{
"name": "alertmanager-main",
"namespace": "monitoring",
"port": "web"
}
]
},
"baseImage": "quay.io/prometheus/prometheus",
"replicas": 2,
"resources": {
"requests": {
"memory": "400Mi"
}
},
"ruleSelector": {
"matchLabels": {
"prometheus": "prafull",
"role": "alert-rules"
}
},
上面的配置是我的 prometheus.json 文件,其中包含要使用的 Alertmanager 名称和ruleSelector
这将根据以下条件选择规则prometheus
and role
标签。所以我的规则配置映射如下:
kind: ConfigMap
apiVersion: v1
metadata:
name: prometheus-rules
namespace: monitoring
labels:
role: alert-rules
prometheus: prafull
data:
alert-rules.yaml: |+
groups:
- name: alerting_rules
rules:
- alert: LoadAverage15m
expr: node_load15 >= 0.50
labels:
severity: major
annotations:
summary: "Instance {{ $labels.instance }} - high load average"
description: "{{ $labels.instance }} (measured by {{ $labels.job }}) has high load average ({{ $value }}) over 15 minutes."
更换DeadManSwitch
在上面的配置图中。