MIME-Version: 1.0
References: <OSZPR01MB63102C42A24D59FACF6D9CD6FD4A9@OSZPR01MB6310.jpnprd01.prod.outlook.com>
 <OS3PR01MB6275328379FBE5734B4585619E54A@OS3PR01MB6275.jpnprd01.prod.outlook.com>
 <ZIgf3qBvFoTsMhIc@paquier.xyz> <CAA4eK1J8WOK9VZ9RpQRNRhRYhFOKQCu=GLCu+iBOePNO3JwLbQ@mail.gmail.com>
 <ZIjmGhPWdoJUfjjE@paquier.xyz> <CAJpy0uDaubBHyqPc1k0OysuBYDOVdoUgTWG4jXDCYj-OVSU8hg@mail.gmail.com>
 <CAA4eK1+K8KMsB=+jJO6wDUSt7wF1RiXKtF-HN48nCOEOv-J-3Q@mail.gmail.com>
 <CAJpy0uDLLBYAOzCePYObZ51k1epBU0hef4vbfcujKJprJVsEcQ@mail.gmail.com>
 <CAJpy0uAhLjQZ0Dh0KWDFP8mrnG0rbx99_heavwn8Ke8ZuD-Umg@mail.gmail.com>
 <OS0PR01MB5716A47D23EFAF988475D4A99431A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
 <CAD21AoCXCAQ5QyXu9-xs30ViUHtUxQMmf-818d8GX--5pTmZ7g@mail.gmail.com>
 <OS0PR01MB57163E6487EFF7378CB8E17C9438A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
 <B38B7239-92B0-4212-A8BF-FE506C3B0F66@yandex-team.ru> <CAA4eK1JpAcvnfOqF2DQo79pf7Cqp5=3HU5UDwBonWXW4V9ot=w@mail.gmail.com>
 <080f9394-c127-4cef-865e-10f2f997125c@postgrespro.ru> <CAD21AoCzjzr5VqVkpVWkN7V6vrFhac=yXSNMz6rkSE+KOPPU0w@mail.gmail.com>
In-Reply-To: <CAD21AoCzjzr5VqVkpVWkN7V6vrFhac=yXSNMz6rkSE+KOPPU0w@mail.gmail.com>
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Date: Wed, 4 Feb 2026 17:14:10 +0530
Message-ID: <CAExHW5ufb1CAcEP1o03M3c=q6AgCzdSLOFaxbC-dJN-k7ymqaA@mail.gmail.com>
Subject: Re: Support logical replication of DDLs
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Vitaly Davydov <v.davydov@postgrespro.ru>, pgsql-hackers@lists.postgresql.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAExHW5ufb1CAcEP1o03M3c%3Dq6AgCzdSLOFaxbC-dJN-k7ymqaA%40mail.gmail.com>
Precedence: bulk

On Wed, Feb 4, 2026 at 1:10=E2=80=AFPM Masahiko Sawada <sawada.mshk@gmail.c=
om> wrote:
>
> Hi,
>
> On Tue, Feb 3, 2026 at 1:04=E2=80=AFAM Vitaly Davydov <v.davydov@postgres=
pro.ru> wrote:
> >
> > Dear Hackers,
> >
> > I see, that the primary idea in this thread is to capture DDL statement=
s with
> > the use of the utility hook, convert it into a json-like format, save i=
t in the
> > WAL (xl_logical_message) and send it to a peer using the logical replic=
ation
> > subsystem (walsender). This approach has a major challenge that was alr=
eady
> > highlighted: DDL statements may be too complex to convert it into a jso=
n string
> > (for example, unlogged/temporary objects in DDL statements).
> >
> > An alternative approach, that was highlighted in the discussion by Dili=
p Kumar
> > is to decode system catalog changes. It helps to deal with complex DDL
> > statements containing temp objects and to work with a final representat=
ion of
> > changes stored in the WAL. Personally, I like this approach.
> >
> > I would share the following idea:
> >
> > 1. Log into the WAL system catalog changes (tuples) suitable for logica=
l
> > decoding (introduce a new wal_level =3D logical_ddl). I think, not all =
system
> > catalog changes are needed for decoding (not sure, we have to decode pg=
_depend
> > changes).
> >
> > 2. Implement a decoder of system catalog changes, that can produce a pa=
rse tree
> > using existing structures from parsenodes.h.
> >
> > 3. Based on the decoded parse tree, we can convert it into json or DDL =
SQL
> > statements in the output plugin. ParseTree to DDL SQL converter can be =
built-in
> > into the core. Output plugin can decide which converter to use. DDL sql=
 can be
> > directly applied on the replica.
> >
> > 4. Another option is to create json/ddl-sql from system catalog changes=
 without
> > an intermediate representation, but, anyway, when we interpret system c=
atalog
> > changes we have to temporary save current data in some structures. Pars=
enodes
> > is the already existing solution for it.
>
> IIUC, one of the main challenges of the "deparsing DDL parse tree"
> idea is the maintenance burden. If we implement logic to deparse parse
> nodes back to SQL text, we would end up updating that deparsing code
> every time the underlying parse node definition changes (which happens
> frequently in internal structures). This introduces a substantial and
> ongoing maintenance cost.
>
> In that light, the idea proposed above seems to increase this burden
> rather than alleviate it. It requires implementing not only the DDL
> deparsing (converting parse nodes back to SQL query) at step 3, but
> also the reconstruction of system catalog changes into parse nodes at
> Step 2. This implies maintaining complex mappings in two places
> whenever internal structures change.
>
> I've been researching this area lately and experimenting with several
> approaches (including the idea of decoding system catalog changes).
> While I'm still not sure what is the best approach for DDL
> replication, even with the deparse approach I agree that performing
> this work at decoding time would be better than doing that at
> execution time using event triggers, as it allows all logical decoding
> plugins to handle DDLs consistently.
>
> > The open question: can we unambiguously decode system catalog changes?
>
> While it sounds challenging, it sounds promising. ISTM it is possible
> to infer the executed DDL command from catalog changes. The crucial
> point is how to implement this cleanly and to minimize the maintenance
> burden as much as possible, especially because this code would be used
> only for logical decoding. Instead of reconstructing the full internal
> parse nodes, we might want to construct a stable, abstract
> representation containing only the necessary information for
> replication.

Consider an example of ALTER TABLE tab ADD PRIMARY KEY (id) where id
is an existing column in table tab. This will add a constraint on the
table and also create an index. Thus effectively two DDLs will be
executed. If we decode catalog changes we need to make sure that only
the original DDL is replicated; else the apply worker downstream will
cause an ERROR, stalling the replication. Similar is the case with
CREATE TABLE ... AS ... - we should only replicate the CREATE TABLE
and let the regular replication handle data replication. I think we
need to somehow annotate the WAL containing catalog changes to
indicate whether those represent the original DDL or derived DDL and
decode only the WAL corresponding to the original DDL.

If there are multiple DDLs in the same user SQL, decoding only the
original DDL from the WAL would help. Also we will be able to apply
table/column based filtering appropriately.

--=20
Best Wishes,
Ashutosh Bapat