public inbox for pgsql-general@postgresql.org
help / color / mirror / Atom feedFrom: Hannu Krosing <hannuk@google.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Vitaly Davydov <v.davydov@postgrespro.ru>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Support logical replication of DDLs, take2
Date: Tue, 28 Apr 2026 21:55:07 +0200
Message-ID: <CAMT0RQQQ5DAE74GSFnEAtEOGN3tXcXYt==fR1gPZ+PGZD+waPg@mail.gmail.com> (raw)
In-Reply-To: <CAA4eK1LoUyyPGZExMRuhQ2g=_PvpE-NQra-dcE1OF=WeWnURZQ@mail.gmail.com>
References: <OSZPR01MB63102C42A24D59FACF6D9CD6FD4A9@OSZPR01MB6310.jpnprd01.prod.outlook.com>
<ZIjmGhPWdoJUfjjE@paquier.xyz>
<CAJpy0uDaubBHyqPc1k0OysuBYDOVdoUgTWG4jXDCYj-OVSU8hg@mail.gmail.com>
<CAA4eK1+K8KMsB=+jJO6wDUSt7wF1RiXKtF-HN48nCOEOv-J-3Q@mail.gmail.com>
<CAJpy0uDLLBYAOzCePYObZ51k1epBU0hef4vbfcujKJprJVsEcQ@mail.gmail.com>
<CAJpy0uAhLjQZ0Dh0KWDFP8mrnG0rbx99_heavwn8Ke8ZuD-Umg@mail.gmail.com>
<OS0PR01MB5716A47D23EFAF988475D4A99431A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAD21AoCXCAQ5QyXu9-xs30ViUHtUxQMmf-818d8GX--5pTmZ7g@mail.gmail.com>
<OS0PR01MB57163E6487EFF7378CB8E17C9438A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<B38B7239-92B0-4212-A8BF-FE506C3B0F66@yandex-team.ru>
<CAA4eK1JpAcvnfOqF2DQo79pf7Cqp5=3HU5UDwBonWXW4V9ot=w@mail.gmail.com>
<080f9394-c127-4cef-865e-10f2f997125c@postgrespro.ru>
<CAD21AoCzjzr5VqVkpVWkN7V6vrFhac=yXSNMz6rkSE+KOPPU0w@mail.gmail.com>
<CAExHW5ufb1CAcEP1o03M3c=q6AgCzdSLOFaxbC-dJN-k7ymqaA@mail.gmail.com>
<38690b0e-f91b-46fa-b72a-57775612e463@postgrespro.ru>
<CAD21AoCTdgHGcEQLwbqXR1E6aC1VyO8wOB4S4U6nU50bZQ+NzQ@mail.gmail.com>
<CAD21AoCzT3sytVbimRNdjRF=N3R-8ddaWKW95EzsdderXqcm4g@mail.gmail.com>
<CAFiTN-t1d3vzTMy4X=R_9m2AOmyVKpYQWKfZnOrat2voBi-K9g@mail.gmail.com>
<CAA4eK1LoUyyPGZExMRuhQ2g=_PvpE-NQra-dcE1OF=WeWnURZQ@mail.gmail.com>
My high-level understanding is, that we should first clearly answer these
two questions
1. What does Logical Decodng infrastructure make available to the decoding
plugin call-backs
2. What the callbacks themselves do in case of DDL
My answer to the 1. is "everything, as it should be the plugin that decides
what it needs". It does not mean that we should always prepack everything
with special logical decoding structures, but there should be a way to get
at anything that is available in the WAL at least. The result is in WAL,
and for DDL it is also in the system tables themselves, in proper
time-travel way.
For 2. I would prefer to "deparse" the DDL from actual system tables at
that snapshot. In logical decoding the system tables are special in that we
keep the actual table content and have real time travel capabilities on
them. This should allow us to use the code we already have in pg_dump for
extracting the "status quo DDL" meaning the DDL for creating everything
from scratch. The main thing missing is DDL for ALTER and DROP which would
need to be added. But that too should be in the plugin, not in the DDL side
.
On Tue, Apr 28, 2026 at 8:57 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Mon, Apr 27, 2026 at 11:45 AM Dilip Kumar <dilipbalaut@gmail.com>
> wrote:
> >
> > On Tue, Apr 21, 2026 at 4:45 AM Masahiko Sawada <sawada.mshk@gmail.com>
> wrote:
> > >
> > > The overall idea of the previous patch set was to implement DDL
> > > deparsing and utilize it for DDL replication. It converted a parse
> > > tree into a JSON string. For instance, if a user executes "DROP TABLE
> > > t1", the deparser generates from its parse tree:
> > >
> > > {DROPSTMT :objects (("t1")) :removeType 41 :behavior 0 :missing_ok
> > > false :concurrent false}
> > >
> > > to:
> > >
> > > {"fmt": "DROP TABLE %{objidentity}s", "objidentity": "public.t1"}
> > >
> > > This JSON string is self-documenting, meaning someone who gets it can
> > > easily reconstruct the original DDL with schema-qualified object
> > > names. In a dedicated event trigger for logical replication, we
> > > deparsed the parse tree of a DDL, wrote it into a WAL record, and then
> > > the logical decoding processed it similarly to DML changes.
> >
> > I think there was also a discussion on whether to use JSON vs the
> > existing infrastructure of converting nodes to strings. Although the
> > JSON is standard format and might provide more flexibility using
> > existing format avoid extra maintence burden.
> >
>
> Yes, the discussion related to node-to-string and the pros and cons of
> the deparse approach are detailed in email [1]. I think some of these
> points could be also related to the new approach as well.
>
> [1] -
> https://www.postgresql.org/message-id/OS0PR01MB571684CBF660D05B63B4412C94AB9%40OS0PR01MB5716.jpnprd0...
>
> --
> With Regards,
> Amit Kapila.
>
>
>
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: hannuk@google.com, amit.kapila16@gmail.com, dilipbalaut@gmail.com, sawada.mshk@gmail.com, v.davydov@postgrespro.ru, ashutosh.bapat.oss@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Support logical replication of DDLs, take2
In-Reply-To: <CAMT0RQQQ5DAE74GSFnEAtEOGN3tXcXYt==fR1gPZ+PGZD+waPg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox