K8s 自动更新证书失败
最近一段时间证书有点问题，记得上次证书过期还是手动更新的，这次3个月的证书又过期了，找了一下k8s证书自动管理方式，发现之前是用 Cert-Manager 来管理证书的，太久没动服务了都忘了。 没更新的原因是 blog.wanderto.top的 ingress 里的脚本太复杂了，好像是半年多前加的，这之前又声明了 acme.cert-manager.io/http01-edit-in-place: "true"，cert修改ingress时修复失败，解决方法：对应配置修改为acme.cert-manager.io/http01-edit-in-place: "false" 这样cert就会新增一个ingress来验证challenge，自动更新成功。 注：有一个问题，如果域名已有一个ingress是”/“路径，那么cert创建的ingress可能无法生成，又会冲突，一共3个域名，一个改false成功，另外两个改为true成功，false会失败，GG的很。如果更新失败不知道原因可以看certificate,然后根据events里的信息跟踪相关资源，最终会看到失败信息在challenge中
排查记录
kubectl get certificates -n blog-web
NAME READY SECRET AGE
blog-tls-secret-blog True blog-tls-secret-blog 22m
blog-tls-secret-blog-static False blog-tls-secret-blog-static 17m
[root@iZbp1605iwejf5qgem2c7hZ ~]# kubectl describe certificates blog-tls-secret-blog-static
Error from server (NotFound): certificates.cert-manager.io "blog-tls-secret-blog-static" not found
[root@iZbp1605iwejf5qgem2c7hZ ~]# kubectl describe certificates blog-tls-secret-blog-static -n blog-web
Name: blog-tls-secret-blog-static
Namespace: blog-web
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1
Kind: Certificate
Metadata:
Creation Timestamp: 2025-12-18T08:47:28Z
Generation: 1
Owner References:
API Version: networking.k8s.io/v1
Block Owner Deletion: true
Controller: true
Kind: Ingress
Name: my-static-blog-ingress
UID: c44283c8-0173-4f9e-bb99-0c751275000e
Resource Version: 114043476
UID: 12ed2f6d-fca5-4061-ae9f-4e70522e2786
Spec:
Dns Names:
static.wanderto.top
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-prod
Secret Name: blog-tls-secret-blog-static
Usages:
digital signature
key encipherment
Status:
Conditions:
Last Transition Time: 2025-12-18T08:47:28Z
Message: Certificate expired on Sun, 14 Dec 2025 23:27:26 UTC
Observed Generation: 1
Reason: Expired
Status: False
Type: Ready
Last Transition Time: 2025-12-18T08:47:28Z
Message: Renewing certificate as renewal was scheduled at 2025-11-14 23:27:26 +0000 UTC
Observed Generation: 1
Reason: Renewing
Status: True
Type: Issuing
Next Private Key Secret Name: blog-tls-secret-blog-static-d4l8c
Not After: 2025-12-14T23:27:26Z
Not Before: 2025-09-15T23:27:27Z
Renewal Time: 2025-11-14T23:27:26Z
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Issuing 17m cert-manager-certificates-trigger Renewing certificate as renewal was scheduled at 2025-11-14 23:27:26 +0000 UTC
Normal Reused 17m cert-manager-certificates-key-manager Reusing private key stored in existing Secret resource "blog-tls-secret-blog-static"
Normal Requested 17m cert-manager-certificates-request-manager Created new CertificateRequest resource "blog-tls-secret-blog-static-1"
[root@iZbp1605iwejf5qgem2c7hZ ~]# kubectl describe -n blog-web CertificateRequest blog-tls-secret-blog-static-1
Name: blog-tls-secret-blog-static-1
Namespace: blog-web
Labels: <none>
Annotations: cert-manager.io/certificate-name: blog-tls-secret-blog-static
cert-manager.io/certificate-revision: 1
cert-manager.io/private-key-secret-name: blog-tls-secret-blog-static-d4l8c
API Version: cert-manager.io/v1
Kind: CertificateRequest
Metadata:
Creation Timestamp: 2025-12-18T08:47:29Z
Generation: 1
Owner References:
API Version: cert-manager.io/v1
Block Owner Deletion: true
Controller: true
Kind: Certificate
Name: blog-tls-secret-blog-static
UID: 12ed2f6d-fca5-4061-ae9f-4e70522e2786
Resource Version: 114043497
UID: 70c916d4-708a-4e30-9018-24fe2fe6ebf4
Spec:
Extra:
authentication.kubernetes.io/pod-name:
cert-manager-6dc66985d4-tk7gm
authentication.kubernetes.io/pod-uid:
c571d844-c862-4fcd-b65d-b73f5bdaec21
Groups:
system:serviceaccounts
system:serviceaccounts:cert-manager
system:authenticated
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-prod
Request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2lUQ0NBWEVDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU55dwpBRXFNOXlHaHlxRzFzMXBZNytmYWorK2ZvTEE5OStTWWpQTTYzWFRmZXYxeDRXeCtGTEsrWDNaTE5mdERvNlJJCnF2a1VWenB6c0hqUmpjbXk5amQwWERzN0VKS2NGVWg4Wkx6Y293NEJrZmtycWVaTkdmSUJpbWNJMnl4N1JWUysKc1p4aWhtTkY4M0RvVGZ4K3B3QWZ3YlZGdzlMUTdvb01GejJ1blBnM3lRQnpCRjZvSFh4c1FSOTIrMzlHSFQyNApFMGR3SU4wbnBHbkxyOGJsdlI0d0RHYlZ6eHpwWHF3Z0FZNXZ3ZXdTa1ZOdngzZmdheGpURUQrbEVKeEh6V0owCkVSWEU5dE1TVElnUHBlNXFjeHNTTGxRRlRwbU9nWEREdWdycGl4RjRsaDRVeStjYVB1Nms4em1hYnhROEFyQTYKajQwbmZPUGxOdWlDVnQyMEdQa0NBd0VBQWFCRU1FSUdDU3FHU0liM0RRRUpEakUxTURNd0lRWURWUjBSQVFILwpCQmN3RllJVGMzUmhkR2xqTG5kaGJtUmxjblJ2TG5SdmNEQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0RRWUpLb1pJCmh2Y05BUUVMQlFBRGdnRUJBRUFPbWp2Z0M2S3dIZVdHTElaMVQzTVd1ZzNNUldOdjBDZDlsOVNTUjM1N3d1YTIKc2RVZU0yWHZFOUk2UFFNZlBQZkRiM0JZdWVLbkxIRXhaT0NOczJHallHRXBaY1RZWXNNeVZ2eTl2U1laUWJNcgpNcHlLL1F2Nm54VE1BcjBScVdYTW1sRENIOEJuKzJiTGk2c0c1VlhQNjZ4UEIyUWZBdCtycERySFFYaVpUYnNVClJvYkNxRDF4VFlkQWhCQ0VXMndLdFBNRExra21FdHZxdUFoZXRTd3JGUHpiV1p3azQ0d3pLaFlxNTMwbWttd2sKbHh0OHRkdTh2d285SnY4TUswRTMvVThEcVgxaWt2VzNtMmFJRGlKT2tOcEg1MElra3AxcjJ2SExHY2lwRXdNbQpGYzF1Z2Z2NDdYMytuc282RWJ0bXpCZk53UGxLZzF6UTV6RHdESkU9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=
UID: d9dfe0c7-4f28-484b-bcc2-6adbdcc020d8
Usages:
digital signature
key encipherment
Username: system:serviceaccount:cert-manager:cert-manager
Status:
Conditions:
Last Transition Time: 2025-12-18T08:47:28Z
Message: Certificate request has been approved by cert-manager.io
Reason: cert-manager.io
Status: True
Type: Approved
Last Transition Time: 2025-12-18T08:47:29Z
Message: Waiting on certificate issuance from order blog-web/blog-tls-secret-blog-static-1-2530046973: "pending"
Reason: Pending
Status: False
Type: Ready
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal WaitingForApproval 18m cert-manager-certificaterequests-issuer-vault Not signing CertificateRequest until it is Approved
Normal WaitingForApproval 18m cert-manager-certificaterequests-issuer-selfsigned Not signing CertificateRequest until it is Approved
Normal WaitingForApproval 18m cert-manager-certificaterequests-issuer-venafi Not signing CertificateRequest until it is Approved
Normal WaitingForApproval 18m cert-manager-certificaterequests-issuer-ca Not signing CertificateRequest until it is Approved
Normal WaitingForApproval 18m cert-manager-certificaterequests-issuer-acme Not signing CertificateRequest until it is Approved
Normal cert-manager.io 18m cert-manager-certificaterequests-approver Certificate request has been approved by cert-manager.io
Normal OrderCreated 18m cert-manager-certificaterequests-issuer-acme Created Order resource blog-web/blog-tls-secret-blog-static-1-2530046973
Normal OrderPending 18m cert-manager-certificaterequests-issuer-acme Waiting on certificate issuance from order blog-web/blog-tls-secret-blog-static-1-2530046973: ""
[root@iZbp1605iwejf5qgem2c7hZ ~]# kubectl describe -n blog-web order blog-tls-secret-blog-static-1-2530046973
Name: blog-tls-secret-blog-static-1-2530046973
Namespace: blog-web
Labels: <none>
Annotations: cert-manager.io/certificate-name: blog-tls-secret-blog-static
cert-manager.io/certificate-revision: 1
cert-manager.io/private-key-secret-name: blog-tls-secret-blog-static-d4l8c
API Version: acme.cert-manager.io/v1
Kind: Order
Metadata:
Creation Timestamp: 2025-12-18T08:47:29Z
Generation: 1
Owner References:
API Version: cert-manager.io/v1
Block Owner Deletion: true
Controller: true
Kind: CertificateRequest
Name: blog-tls-secret-blog-static-1
UID: 70c916d4-708a-4e30-9018-24fe2fe6ebf4
Resource Version: 114043499
UID: 45d79705-31e4-416d-8073-77ffbae13a12
Spec:
Dns Names:
static.wanderto.top
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-prod
Request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2lUQ0NBWEVDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU55dwpBRXFNOXlHaHlxRzFzMXBZNytmYWorK2ZvTEE5OStTWWpQTTYzWFRmZXYxeDRXeCtGTEsrWDNaTE5mdERvNlJJCnF2a1VWenB6c0hqUmpjbXk5amQwWERzN0VKS2NGVWg4Wkx6Y293NEJrZmtycWVaTkdmSUJpbWNJMnl4N1JWUysKc1p4aWhtTkY4M0RvVGZ4K3B3QWZ3YlZGdzlMUTdvb01GejJ1blBnM3lRQnpCRjZvSFh4c1FSOTIrMzlHSFQyNApFMGR3SU4wbnBHbkxyOGJsdlI0d0RHYlZ6eHpwWHF3Z0FZNXZ3ZXdTa1ZOdngzZmdheGpURUQrbEVKeEh6V0owCkVSWEU5dE1TVElnUHBlNXFjeHNTTGxRRlRwbU9nWEREdWdycGl4RjRsaDRVeStjYVB1Nms4em1hYnhROEFyQTYKajQwbmZPUGxOdWlDVnQyMEdQa0NBd0VBQWFCRU1FSUdDU3FHU0liM0RRRUpEakUxTURNd0lRWURWUjBSQVFILwpCQmN3RllJVGMzUmhkR2xqTG5kaGJtUmxjblJ2TG5SdmNEQU9CZ05WSFE4QkFmOEVCQU1DQmFBd0RRWUpLb1pJCmh2Y05BUUVMQlFBRGdnRUJBRUFPbWp2Z0M2S3dIZVdHTElaMVQzTVd1ZzNNUldOdjBDZDlsOVNTUjM1N3d1YTIKc2RVZU0yWHZFOUk2UFFNZlBQZkRiM0JZdWVLbkxIRXhaT0NOczJHallHRXBaY1RZWXNNeVZ2eTl2U1laUWJNcgpNcHlLL1F2Nm54VE1BcjBScVdYTW1sRENIOEJuKzJiTGk2c0c1VlhQNjZ4UEIyUWZBdCtycERySFFYaVpUYnNVClJvYkNxRDF4VFlkQWhCQ0VXMndLdFBNRExra21FdHZxdUFoZXRTd3JGUHpiV1p3azQ0d3pLaFlxNTMwbWttd2sKbHh0OHRkdTh2d285SnY4TUswRTMvVThEcVgxaWt2VzNtMmFJRGlKT2tOcEg1MElra3AxcjJ2SExHY2lwRXdNbQpGYzF1Z2Z2NDdYMytuc282RWJ0bXpCZk53UGxLZzF6UTV6RHdESkU9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=
Status:
Authorizations:
Challenges:
Token: 9rRExe76u5hMqnrLHnp_NNIYCnjUACY9Q9L7nHuTtbo
Type: dns-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall/1727082297/629248003796/ZJEx9A
Token: 9rRExe76u5hMqnrLHnp_NNIYCnjUACY9Q9L7nHuTtbo
Type: http-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall/1727082297/629248003796/oKKBTg
Token: 9rRExe76u5hMqnrLHnp_NNIYCnjUACY9Q9L7nHuTtbo
Type: tls-alpn-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall/1727082297/629248003796/sp6PDw
Identifier: static.wanderto.top
Initial State: pending
URL: https://acme-v02.api.letsencrypt.org/acme/authz/1727082297/629248003796
Wildcard: false
Finalize URL: https://acme-v02.api.letsencrypt.org/acme/finalize/1727082297/460044912416
State: pending
URL: https://acme-v02.api.letsencrypt.org/acme/order/1727082297/460044912416
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Created 20m cert-manager-orders Created Challenge resource "blog-tls-secret-blog-static-1-2530046973-1912455086" for domain "static.wanderto.top"
[root@iZbp1605iwejf5qgem2c7hZ ~]# kubectl describe -n blog-web Challenge blog-tls-secret-blog-static-1-2530046973-1912455086
Name: blog-tls-secret-blog-static-1-2530046973-1912455086
Namespace: blog-web
Labels: <none>
Annotations: <none>
API Version: acme.cert-manager.io/v1
Kind: Challenge
Metadata:
Creation Timestamp: 2025-12-18T08:47:30Z
Finalizers:
finalizer.acme.cert-manager.io
Generation: 1
Owner References:
API Version: acme.cert-manager.io/v1
Block Owner Deletion: true
Controller: true
Kind: Order
Name: blog-tls-secret-blog-static-1-2530046973
UID: 45d79705-31e4-416d-8073-77ffbae13a12
Resource Version: 114043521
UID: a8a0fd94-e835-401c-800c-2b0e3d387e73
Spec:
Authorization URL: https://acme-v02.api.letsencrypt.org/acme/authz/1727082297/629248003796
Dns Name: static.wanderto.top
Issuer Ref:
Group: cert-manager.io
Kind: ClusterIssuer
Name: letsencrypt-prod
Key: 9rRExe76u5hMqnrLHnp_NNIYCnjUACY9Q9L7nHuTtbo.haZXj8s8E83HGaYsxqYEpyixIqt_dgo9gPoTdm4qwSU
Solver:
http01:
Ingress:
Ingress Class Name: nginx
Token: 9rRExe76u5hMqnrLHnp_NNIYCnjUACY9Q9L7nHuTtbo
Type: HTTP-01
URL: https://acme-v02.api.letsencrypt.org/acme/chall/1727082297/629248003796/oKKBTg
Wildcard: false
Status:
Presented: true
Processing: true
Reason: Waiting for HTTP-01 challenge propagation: wrong status code '404', expected '200'
State: pending
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Started 21m cert-manager-challenges Challenge scheduled for processing
Normal Presented 21m cert-manager-challenges Presented challenge using HTTP-01 challenge mechanism
豆包挺好用的，顺着豆包的思路找到的问题，个人感觉比deepseek好用，豆包不好使用的时候我就用chatgpt的免费次数。