I was working on the helm chart for Prometheus operator. Everything was working fine apart from the alertmanager.
The way that alertmanager config works are to covert the config section into a secret and the alertmanager will reference that secret. Behind the scene, it actually covert the config into base64 format and the secret will be reference alertmanager.yaml : <base64 encode value>
So in this case, if the value is incorrect. alertmanager will refuse to be created by operator. Or in another form will be the config not able to re-load.
I checked logs on the operator and realised there were some issues with the configuration. Like below.
E1202 22:17:45.649708 1 operator.go:533] Sync “monitoring/prometheus-kube-prometheus-alertmanager” failed: provision alertmanager configuration: base config from Secret could not be parsed: yaml: unmarshal errors:
line 5: field send_resolved not found in type config.plain
After the investigation, I found it’s the typo of the yaml. I should put send_resolved one level down.
