public inbox for pgsql-general@postgresql.org
help / color / mirror / Atom feedFrom: Hannu Krosing <hannuk@google.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Vitaly Davydov <v.davydov@postgrespro.ru>
Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Support logical replication of DDLs, take2
Date: Wed, 29 Apr 2026 10:07:04 +0200
Message-ID: <CAMT0RQThNce3+0MEzbqNWov7uZZQbiyFrhrod3g+MeKoAqjRbw@mail.gmail.com> (raw)
In-Reply-To: <CAFiTN-vG4z03ar5PMYT3ehe+yzKH8+nPR4uqhNEhnax0DmOq_g@mail.gmail.com>
References: <OSZPR01MB63102C42A24D59FACF6D9CD6FD4A9@OSZPR01MB6310.jpnprd01.prod.outlook.com>
<ZIjmGhPWdoJUfjjE@paquier.xyz>
<CAJpy0uDaubBHyqPc1k0OysuBYDOVdoUgTWG4jXDCYj-OVSU8hg@mail.gmail.com>
<CAA4eK1+K8KMsB=+jJO6wDUSt7wF1RiXKtF-HN48nCOEOv-J-3Q@mail.gmail.com>
<CAJpy0uDLLBYAOzCePYObZ51k1epBU0hef4vbfcujKJprJVsEcQ@mail.gmail.com>
<CAJpy0uAhLjQZ0Dh0KWDFP8mrnG0rbx99_heavwn8Ke8ZuD-Umg@mail.gmail.com>
<OS0PR01MB5716A47D23EFAF988475D4A99431A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAD21AoCXCAQ5QyXu9-xs30ViUHtUxQMmf-818d8GX--5pTmZ7g@mail.gmail.com>
<OS0PR01MB57163E6487EFF7378CB8E17C9438A@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<B38B7239-92B0-4212-A8BF-FE506C3B0F66@yandex-team.ru>
<CAA4eK1JpAcvnfOqF2DQo79pf7Cqp5=3HU5UDwBonWXW4V9ot=w@mail.gmail.com>
<080f9394-c127-4cef-865e-10f2f997125c@postgrespro.ru>
<CAD21AoCzjzr5VqVkpVWkN7V6vrFhac=yXSNMz6rkSE+KOPPU0w@mail.gmail.com>
<CAExHW5ufb1CAcEP1o03M3c=q6AgCzdSLOFaxbC-dJN-k7ymqaA@mail.gmail.com>
<38690b0e-f91b-46fa-b72a-57775612e463@postgrespro.ru>
<CAD21AoCTdgHGcEQLwbqXR1E6aC1VyO8wOB4S4U6nU50bZQ+NzQ@mail.gmail.com>
<CAD21AoCzT3sytVbimRNdjRF=N3R-8ddaWKW95EzsdderXqcm4g@mail.gmail.com>
<CAFiTN-t1d3vzTMy4X=R_9m2AOmyVKpYQWKfZnOrat2voBi-K9g@mail.gmail.com>
<CAA4eK1LoUyyPGZExMRuhQ2g=_PvpE-NQra-dcE1OF=WeWnURZQ@mail.gmail.com>
<CAMT0RQQQ5DAE74GSFnEAtEOGN3tXcXYt==fR1gPZ+PGZD+waPg@mail.gmail.com>
<CAFiTN-vG4z03ar5PMYT3ehe+yzKH8+nPR4uqhNEhnax0DmOq_g@mail.gmail.com>
On Wed, Apr 29, 2026 at 5:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> I am trying to understand your idea. If we are trying to deparse from
> an actual system table using a snapshot, why don't we just use the
> WAL? I mean, the WAL should contain the actual catalog modifications
> it has made.
We have the full data in the catalog and we would likely need catalog
queries for any change, even when de-parsing the tree.
And we should not add the extra load on the original DDL side, just as
we don't for DML.
At most we could just serialize the statement tree into the WAL,
though even that may be an overkill if we can get the change from
existing records.
- insert new row in pg_class --> extract the CREATE TABLE (or INDEX, or ...)
- update row in pg_class or insert, update or delete a row in
pg_attribute --> extract ALTER TABLE
- except when it just updates relfilenod --> extract TRUNCATE
- delete row in pg_class --> DROP TABLE
- dml on pg_constraint --> ALTER TABLE
... etc
> Although converting the catalog changes into a deparse
> representation of the DDL could be complex no?
Both de-parsing the tree and converting the catalog change could be complex.
The advantage of using the catalog is that we already have decades of
experience doing this via pg_dump.
> Another question is
> what we would do with those deparsed representations: will we convert
> them to SQL on the subscriber and execute, or do something else?
Current pg_dump approach is logically equivalent to "doing it on the
subscriber", pg_dump is designed to dump schemas from all older
database versions in format that is compatible with the version the
pg_dump is written for.
This brings us back to the uncomfortable discussion of needing to
back-port some changes to older versions contrary to general
PostgreSQL development principles of adding new features to only the
latest version.
Or we could enable exporting the catalog snapshot from logical
replication stream so that subscriber could use that snapshot in a
"callback connection: to extract the catalog state at that snapshot
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: hannuk@google.com, dilipbalaut@gmail.com, amit.kapila16@gmail.com, sawada.mshk@gmail.com, v.davydov@postgrespro.ru, ashutosh.bapat.oss@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Support logical replication of DDLs, take2
In-Reply-To: <CAMT0RQThNce3+0MEzbqNWov7uZZQbiyFrhrod3g+MeKoAqjRbw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox