public inbox for pgsql-general@postgresql.org
help / color / mirror / Atom feedFrom: Amit Kapila <amit.kapila16@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Vitaly Davydov <v.davydov@postgrespro.ru>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Support logical replication of DDLs, take2
Date: Mon, 4 May 2026 17:53:04 +0530
Message-ID: <CAA4eK1K0sWdYqZx7bmbR8kz+gaPfKwsSJ+UO2a3b-1XGD3Z2zg@mail.gmail.com> (raw)
In-Reply-To: <CAD21AoC_HwTLRSWmD0d=E0Adftu0LE=Ep9FWJf2m1BJyq7x4Gw@mail.gmail.com>
References: <OSZPR01MB63102C42A24D59FACF6D9CD6FD4A9@OSZPR01MB6310.jpnprd01.prod.outlook.com>
<ZIjmGhPWdoJUfjjE@paquier.xyz>
<CAJpy0uDaubBHyqPc1k0OysuBYDOVdoUgTWG4jXDCYj-OVSU8hg@mail.gmail.com>
<CAA4eK1+K8KMsB=+jJO6wDUSt7wF1RiXKtF-HN48nCOEOv-J-3Q@mail.gmail.com>
<CAJpy0uDLLBYAOzCePYObZ51k1epBU0hef4vbfcujKJprJVsEcQ@mail.gmail.com>
<CAJpy0uAhLjQZ0Dh0KWDFP8mrnG0rbx99_heavwn8Ke8ZuD-Umg@mail.gmail.com>
<OS0PR01MB5716A47D23EFAF988475D4A99431A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAD21AoCXCAQ5QyXu9-xs30ViUHtUxQMmf-818d8GX--5pTmZ7g@mail.gmail.com>
<OS0PR01MB57163E6487EFF7378CB8E17C9438A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<B38B7239-92B0-4212-A8BF-FE506C3B0F66@yandex-team.ru>
<CAA4eK1JpAcvnfOqF2DQo79pf7Cqp5=3HU5UDwBonWXW4V9ot=w@mail.gmail.com>
<080f9394-c127-4cef-865e-10f2f997125c@postgrespro.ru>
<CAD21AoCzjzr5VqVkpVWkN7V6vrFhac=yXSNMz6rkSE+KOPPU0w@mail.gmail.com>
<CAExHW5ufb1CAcEP1o03M3c=q6AgCzdSLOFaxbC-dJN-k7ymqaA@mail.gmail.com>
<38690b0e-f91b-46fa-b72a-57775612e463@postgrespro.ru>
<CAD21AoCTdgHGcEQLwbqXR1E6aC1VyO8wOB4S4U6nU50bZQ+NzQ@mail.gmail.com>
<CAD21AoCzT3sytVbimRNdjRF=N3R-8ddaWKW95EzsdderXqcm4g@mail.gmail.com>
<CAA4eK1KMDya3brZdgDwKCoLCKM11t=sqP4QNCEbLai_NRKQF=A@mail.gmail.com>
<CAD21AoACTBsVvyQUKuWuM7qs6-WV4of9SkdUvSv9KHyC_W+K1A@mail.gmail.com>
<CAA4eK1KXbXSMvkZnZunhHCwDF-tV2SWs3rbQN2-bJsRn813BVA@mail.gmail.com>
<CAD21AoC_HwTLRSWmD0d=E0Adftu0LE=Ep9FWJf2m1BJyq7x4Gw@mail.gmail.com>
On Fri, May 1, 2026 at 2:11 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Wed, Apr 29, 2026 at 9:44 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, Apr 29, 2026 at 3:19 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > >
> > > On Mon, Apr 27, 2026 at 11:32 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > >
> > > > Yes, there will be a maintenance cost of JSON-based deparsing
> > > > approach. But note that multiple senior people (Alvaro Herrera, Robert
> > > > Haas) [1] seems to favor that approach. So, I am not sure we can
> > > > conclude to abandon that approach without those people or some other
> > > > senior people agreeing to abandon it. To be clear, I am not against
> > > > considering a new/different approach for DDL replication but just that
> > > > it is not clear that old/existing approach can be ruled out without
> > > > more discussion on it,
> > >
> > > Thank you for pointing it out. Just to be clear, IIUC what they liked
> > > was to use JSON string representation of DDLs, but not JSON string
> > > representation of DDLs that are deparsed from parse nodes, no?
> > >
> >
> > As per my understanding, we built deparsing stuff with a goal of
> > supporting DDL replication and Alvaro was the original author of that
> > work, see [1]. The benefit it provides flexibility in terms of
> > filtering by decoding plugin, if any, or changing the DDL (like
> > schema-mapping) during apply. It is not clear to me if we can achive
> > similar level of flexibility with other approach.
>
> I think we can generate the same JSON-string representation of DDLs
> from catalog information, it would also require a lot of code, though.
> It would be independent from parse nodes and if we implement it as an
> option for pg_get_xxx_ddl() functionality it would be able to be
> reused by other tools too.
>
IIRC, this was discussed previously as well but we were not sure if we
can build all (especially some complex ones) without parsetree. See
discussion/emails around [1][2].
> >
> > >
> > > >
> > > > We would need to maintain the JSON serialization code whenever
> > > > > creating or modifying parse nodes, regardless of whether the changes
> > > > > were related to DDL replication. IIUC, this was the primary reason the
> > > > > feature didn't cross the finish line.
> > > > >
> > > > > Additionally, I think there is another design issue: it is not
> > > > > output-plugin agnostic. Since the deparsed DDL was written by a
> > > > > logical-replication-specific event trigger, third-party logical
> > > > > decoding plugins cannot easily detect DDL events.
> > > > >
> > > >
> > > > Why RmgrId like RM_LOGICALDDLMSG_ID and XLOG_LOGICAL_DDL_MESSAGE wal
> > > > info is not sufficient for this? Decoder will add a message like
> > > > REORDER_BUFFER_CHANGE_DDL which can be used to detect DDL message, no?
> > >
> > > Right, but I'm not sure this is a good developer experience that
> > > additional steps are required to capture DDL events for other plugins
> > > while changes of INSERT/UPDATE/DELETE/TRUNCATE are passed from the
> > > logical decoding by default.
> > >
> >
> > Yes, there could probably be additional steps for plugins but they
> > must be doing a few things already which are defined at publication
> > level like column lists, row filtering, something related to RI, etc.
>
> I think those publication-level features operate at a somewhat
> different layer than the fundamental mechanism of capturing DDL
> events. Plugins filter rows or columns based on configuration, but the
> logical decoding itself guarantees that the DML events are reliably
> passed to them. Given that the TRUNCATE in logical replication already
> works so, I guess DDL should have the same fundamental guarantee.
>
> it's unclear to me how plugins could reliably manage these event
> triggers. While a plugin might create an event trigger during the
> startup callback if it doesn't exist, it cannot drop it during the
> shutdown callback. We also cannot establish a dependency between an
> event trigger and a logical replication slot. We would likely need to
> invent a new plugin callback specifically invoked at slot drop time
> just to clean it up. Also, if different plugins want to capture DDL
> events, they could end up registering different event triggers,
> emitting multiple DDL WAL records for the same DDL event.
>
Why would different plugins end up registering different event
triggers? I mean if they are already registered by the first plugin
what is the need to re-register.
[1] - https://www.postgresql.org/message-id/20150221212017.GD2037@awork2.anarazel.de
[2] - https://www.postgresql.org/message-id/20150224021018.GH30784@awork2.anarazel.de
--
With Regards,
Amit Kapila.
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: amit.kapila16@gmail.com, sawada.mshk@gmail.com, v.davydov@postgrespro.ru, ashutosh.bapat.oss@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Support logical replication of DDLs, take2
In-Reply-To: <CAA4eK1K0sWdYqZx7bmbR8kz+gaPfKwsSJ+UO2a3b-1XGD3Z2zg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox