An ACME ClusterIssuer cannot be used with spec.tls.issuerConf

Report

Setting spec.tls.issuerConf to an existing ACME ClusterIssuer (Let's Encrypt, or any ACME server) does not work. The Operator resolves this to external issuer mode and points all leaf Certificate resources at that issuer, but two of them request identifiers that ACME can never issue:

Certificate	`commonName` and `dnsNames`
`<cluster>-replication-cert`	`_crunchyrepl`
`<cluster>-pgbackrest-client-cert`	`pgbackrest@<cluster-uid>`

cert-manager submits commonName and every dnsNames entry to the ACME server as identifiers requiring an HTTP-01 or DNS-01 challenge. Neither of these strings is a resolvable DNS name, so the Order can never be fulfilled.

Both are client certificates whose identity is consumed by CN matching, not by hostname verification:
The other four certificates (cluster, instance, repo host, PgBouncer) are fine with an ACME issuer, given an ACME server that issues the clientAuth EKU and a spec.clusterServiceDNSSuffix set to a domain that server will validate.

Steps to reproduce

Install cert-manager and create an ACME ClusterIssuer.
Set spec.tls.issuerConf to that issuer (kind: ClusterIssuer, group: cert-manager.io) and deploy a cluster.

Versions

not relevant

Anything else?

The in-flight cert-manager documentation for 3.1.0 (percona/k8spg-docs#414) lists ACME and Let's Encrypt as supported external issuers, and states that the configured issuer must sign every leaf certificate, including the replication and pgBackRest ones. That combination is not achievable, so the docs may need adjusting alongside the fix.

I dont know if I'll call it a bug but it does mean acme isnt supported with cert-manager yet

@yoav-katz

@yoav-katz I think this more a feature request than a bug, because I don't see any other way than:

Ideally the internal mTLS client certificates would be issued independently of spec.tls.issuerConf - either from the Operator's own CA or from a separately configurable issuer - since their identifiers are not DNS names and were never meant to be publicly validated.

@egegunes

@egegunes totally agree, the bug is that in the next release notes there is a note about supporting acme - which is not true.

On the implementation side, as always, I’d be happy to contribute.

What do you think about separating the Issuers for internal and external use cases?

Another approach I thought of is allowing the _crunchyrepl user to be configured with a DNS suffix (e.g. _crunchyrepl.example.com). However, I have some security concerns about this approach, so I’m not sure it would be a good direction.

Rather than adding an arbitrary suffix to _crunchyrepl user, I'd separate Issuers for internal and external but I'm not sure how the API should look like.

@egegunes

@egegunes What do you think about

spec:
  tls:
    issuerConf:
      name: letsencrypt
      kind: ClusterIssuer
      group: cert-manager.io

    internalIssuerConf:
      name: postgres-internal
      kind: Issuer
      group: cert-manager.io

internalIssuerConf would be optional. When it is not specified, issuerConf is used for all certificates, preserving the current behavior.

This would allow us to use a separate issuer for the internal client certificates (_crunchyrepl and pgBackRest) without making the API more complicated or changing their identity semantics.

For this, we’ll need to implement CA bundle concatenation so PostgreSQL can trust certificates issued by both CAs. Currently, the CA/chain is provided through tls.crt in the cert-manager-managed Secret, which we cannot modify. Therefore, we’ll need to create a new -ca-bundle Secret containing the combined CA bundle and use it for the relevant trust stores.

I confirmed this against the cert-manager source: an ACME-backed Certificate yields
only tls.crt and tls.key. It can't be recovered from tls.crt either - ACME
returns leaf + intermediate with no root, and PostgreSQL won't anchor on a bare
intermediate.

We treat ca.crt as mandatory: it's what becomes ssl_ca_file, the replication
sslrootcert, patroni.ca-roots and pgbackrest.ca-roots.

So ACME requires the user to supply the anchor themselves. We already have this for
pgBouncer via additionalTrustedCAs - since the anchor is cluster-wide rather than
proxy-specific, I'd add it as spec.tls.additionalTrustedCAs.

I will open 2 new PR's:

spec.tls.additionalTRustedCAs - for:

issuerConf	`ca.crt`	additionalTrustedCAs	bundle
CA-type (today)	present	empty	just `ca.crt` — unchanged
ACME	absent	set	just the user anchor
CA-type + extra anchors	present	set	both

spec.tls.internalIssuerConf - will follow the same pattern for injecting the issuerConf ca.crt to ca-bundle and will be exclusivly for _cruncyrepl and pgbackrest certificates - it is optional and can be set only when issuerConf is set.

@hors

I think adding spec.tls.internalIssuerConf makes sense, yes. @hors wdyt of adding this into other operators as well?

I got blocked on the exat same problems in psmdb-operator (for clustername-mongos, clustername-rs0,clustername-config, clustername-rs0.namespace etc etc). as for my understanding mongodb requires all certificate san's on both ssl and internal-ssl so it may be problematic - but not actually sure about that.
havent checked mysql

I have created:
#1784
for the additionalTrustedCAs

There is still one remaining problem with this design: issuerConf is still used to create certificates containing names such as clustername-primary, clustername-primary.namespace, clustername-replicas, etc.

I validated the code and found that only one place actually requires these non-FQDN names: the PgBouncer connection.
To make issuerConf fully compatible with ACME, I think we should use fully qualified DNS names for the generated certificates and add a flag that allows removing the short/non-FQDN names from the certificate.

This would allow ACME-backed issuerConf configurations to issue the primary and replica certificates without requesting invalid identifiers, while preserving the existing behavior for users who still rely on the short Kubernetes service names.

I haven't settled on the API yet, but my initial thought was something like spec.tls.fullFQDNOnly

TL;DR, we need to add:

spec.tls.additionalTrustedCAs K8SPG-1153 Add tls.additionalTrustedCAs #1784
spec.tls.internalIssuerConf
spec.tls.fullFQDNOnly

This is starting to become a much larger feature than I initially expected, with several API and certificate-management changes involved. So I completely understand if this scope is considered too large and the feature gets pushed back.

This is starting to become a much larger feature than I initially expected, with several API and certificate-management changes involved.

I agree. Also once we expose these APIs, we also need to support moving between them on running clusters. These kind of changes usually have a lot of edge cases and subtle bugs, so maybe it's better if this implemented by us so the developer and QA can work closer.

so maybe it's better if this implemented by us so the developer and QA can work closer

Really makes sense.
I already made some implementation for spec.tls.internalIssuerConf on my fork, if you want to take a look at it 😃

Another thing that I think would be a great addition on top of all this is an external.hostname field with ExternalDNS Operator integration (like in psmdb operator). I’ll open a separate issue for that.

@yoav-katz

@yoav-katz will you continue working on #1784

I'll create an Epic for ACME certificates

@egegunes

@egegunes happily!
As I said earlier there is a draft for the internalIssuerConf - will push it today if Ill get to it 😃

Epic: https://perconadev.atlassian.net/browse/K8SPG-1151

Navigation Menu

Repository navigation

An ACME `ClusterIssuer` cannot be used with `spec.tls.issuerConf` #1775

Report

More about the problem

Steps to reproduce

Versions

Anything else?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

An ACME ClusterIssuer cannot be used with spec.tls.issuerConf #1775

Description

Report

More about the problem

Steps to reproduce

Versions

Anything else?

Activity

yoav-katz commented on Aug 22, 2026

egegunes commented on Aug 24, 2026

yoav-katz commented on Aug 24, 2026

yoav-katz commented on Aug 24, 2026

egegunes commented on Aug 25, 2026

yoav-katz commented on Aug 25, 2026

yoav-katz commented on Aug 25, 2026

yoav-katz commented on Aug 25, 2026

egegunes commented on Aug 27, 2026

yoav-katz commented on Aug 28, 2026

yoav-katz commented on Aug 28, 2026

egegunes commented on Aug 31, 2026

yoav-katz commented on Aug 31, 2026

egegunes commented on Sep 2, 2026

yoav-katz commented on Sep 2, 2026

egegunes commented on Sep 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

An ACME `ClusterIssuer` cannot be used with `spec.tls.issuerConf` #1775