Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wHoWo-007XAd-2Q for pgsql-hackers@arkaria.postgresql.org; Tue, 28 Apr 2026 19:55:31 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wHoWn-000t9e-0Y for pgsql-hackers@arkaria.postgresql.org; Tue, 28 Apr 2026 19:55:29 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wHoWm-000t9W-2R for pgsql-hackers@lists.postgresql.org; Tue, 28 Apr 2026 19:55:28 +0000 Received: from mail-qt1-x82b.google.com ([2607:f8b0:4864:20::82b]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1wHoWk-00000003Emp-0Av4 for pgsql-hackers@lists.postgresql.org; Tue, 28 Apr 2026 19:55:27 +0000 Received: by mail-qt1-x82b.google.com with SMTP id d75a77b69052e-50d864c23bdso64831cf.1 for ; Tue, 28 Apr 2026 12:55:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1777406126; cv=none; d=google.com; s=arc-20240605; b=BBEiRZ3XV6Yeg2ML0inNwgdRm9NCva9rARJdAxf6xv2BN4nKCEWS+mOrCqLOwAUk4x JYWcvpTP9g1KYIFQq+WfhUCSLHZNOcFheUkxvdFAPle5R9I7iZx4pPkf190nQskqFWI8 OWgg1QVx2oYQyFktMLo0F7wsBAJtDBNRBLYXDHozmmEcJSVwI0oDg1IIox7m+/JTbTUS PvVhoohMcxYdPc7Rlmv4PBTixxSrPf3uBqwpi3duhBkplyPGP6yqH+wvJukVrzfoj6EW 5LRFwH24BlkXkWoQUIh360JzEyVWD7zEbMMaJzKjaYyGe0JsF2L++7J597IbS/y0TH9y jxhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=YtRNs+X2O+rGOIt76wBr6V+skEBgKJxVvQSnnQa5Dh8=; fh=xbFziBF6rFLZWELd08DLMCSBygCqsnWYpq1h4kEi2dA=; b=AmEJXQQ5Jsm0KhJjgbLG+eUIDTGaXyuDa0hkZ1TG50yYFTczxjPjFM49o5dNlaHMwR 9hXZU00cQ0qpP7nHZ05dDuRl6/jctrf75S/pn/GfslRAQgFQ6cboUQ+UyElx3hc27ON7 mhy/2P23AcX2cj7RjpwHaNhZbtmzg7dR7Hz2BDK1m/w2oGV56/gKynXXX+wqSVPhhk57 NgISz/sCaxbvQDwdkOBXWSObScGc7XMjdLqoEDRhA02vwuJJZ2SLK50jnrWDXTj/lTm8 5ktIXn+akaUd3Xb2cHXg12V6ZLGubehDc5GIBbHA0UmFk1GZrxB3mn6H6ngVr3xrdUxH S6BQ==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777406126; x=1778010926; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=YtRNs+X2O+rGOIt76wBr6V+skEBgKJxVvQSnnQa5Dh8=; b=dKrmQ0gd31jAa4pQ0N2s53XwU++5aR3MPCqFV/JccLIKqjzNxKW41J9VGokw+2+2M9 /jrsVKfoxQFDZN93rmKEhQxSaNFkZjcyIJVh7LUzMXDw6IfSWBgYcgL1OtzAEYC2CBYt eT0uR50TFSWf3QOa9aq9xEQAWMwL/i16eecSAW0QcLpjHRIWJCej6APvuy2/ozyZmDVG SIABDtEaA6JtCsYzoF7cfHAyCT9bkmJPlZzbhEUkiQ8HrS4JyaHUk6b5ls0seCH+QCS6 pRUzQTxWosopxAF5sL7tCy0xYTFwLMGxdvHI9oTPTVlLPkg87BMw0IyVadeAT1W/b2vk 9OmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777406126; x=1778010926; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=YtRNs+X2O+rGOIt76wBr6V+skEBgKJxVvQSnnQa5Dh8=; b=BhMdu/gPIA0FPhDGSoWVgTrPaEyPv2XZDbCtNdManfr8+a7Pna+ELZkR/Omt/2dBBl nDOMJlTo/D71h6QH1lPael07qBM3J8ANGiXUJSpYaU9q3dsL9ft5nI1Ap77/VT5zJmiz GRTMkmzThd3UXoNMdTFdEzJaAp1cZfTzYU5zklD26UKZyVK54xbYpjdNGX1xP6lubUjJ qxpYXAs7e9KOn8v6a37MIhIOP/7QUR+5UIuGJ9oVQpXvgKfqGyfxfYjeu/Ldj3iJNknN x6j8TWYe6Wiy19Bcq5qCDrGEB6lc6if2yyhpVtUWpeJvse+jRieSklUXTVKmKvjvVJd5 Iz0w== X-Forwarded-Encrypted: i=1; AFNElJ9CW14RuBd/lo6HZu+vaoJhZmZAfB2lt0frEUK4RTB8oVbcwcX+pB4pbzLfGjHlcTfON1mrmDg2qFbhRZsS@lists.postgresql.org X-Gm-Message-State: AOJu0Yyk5e9fyHF5sr9IdMpPZYQlrai1TrWHpHc6nYVgliBRIFPrjIkw O+lfUDFUFLsTzQlFFEcy8DnUM0s/D5O/YS6cII54ZhcN6wyBRqQEBlVL+A9tzDWgkFEWfmW0JKr mQnGa6cnKs1IJeIpaefUYCzJNKnibC5dJcC5g1pZO X-Gm-Gg: AeBDies/uZGJrTv79WtDWz7J3Wm5BnhbjYKM4Qz8Ilb7pDzofLNHVp7IoiFC6pkh0v3 8IOCyUvIgpzRq+VCE3Tqx5HpoQyP2Rk8sraKDnZdcu39zF8U329WofCRQNWqo9zDZ5O9A2HqJ0v qGFnQjtmZBqcevW7beCY5NVl+OQxiyAvBicNxhTrngA1Ujqbg/+z92DgsJNZxN3NHZB7cZ+l7wh 1uGlZhVDWAAFXZazcAuVWN5jrptB9CmNNhNtxaeRYl0OXft0jiYTyntKm4K3TkTjl63jqBmm7ty p9hlOXnD7hU/pbB9pi4EW3ZxqDHd+CifeSxIxw6dE8yROMXh6n4WEaaNtGOI X-Received: by 2002:a05:622a:5a9a:b0:50e:5c9d:49c7 with SMTP id d75a77b69052e-5101a5e352fmr129601cf.7.1777406125161; Tue, 28 Apr 2026 12:55:25 -0700 (PDT) MIME-Version: 1.0 References: <080f9394-c127-4cef-865e-10f2f997125c@postgrespro.ru> <38690b0e-f91b-46fa-b72a-57775612e463@postgrespro.ru> In-Reply-To: From: Hannu Krosing Date: Tue, 28 Apr 2026 21:55:07 +0200 X-Gm-Features: AVHnY4KXrTTqphQJ3WezQvdh9TkgC2oysQV3epjLTM2p9_w_2QBvkSgrYJ1meWU Message-ID: Subject: Re: Support logical replication of DDLs, take2 To: Amit Kapila Cc: Dilip Kumar , Masahiko Sawada , Vitaly Davydov , Ashutosh Bapat , PostgreSQL Hackers Content-Type: multipart/alternative; boundary="000000000000b728a506508a9c54" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000b728a506508a9c54 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable My high-level understanding is, that we should first clearly answer these two questions 1. What does Logical Decodng infrastructure make available to the decoding plugin call-backs 2. What the callbacks themselves do in case of DDL My answer to the 1. is "everything, as it should be the plugin that decides what it needs". It does not mean that we should always prepack everything with special logical decoding structures, but there should be a way to get at anything that is available in the WAL at least. The result is in WAL, and for DDL it is also in the system tables themselves, in proper time-travel way. For 2. I would prefer to "deparse" the DDL from actual system tables at that snapshot. In logical decoding the system tables are special in that we keep the actual table content and have real time travel capabilities on them. This should allow us to use the code we already have in pg_dump for extracting the "status quo DDL" meaning the DDL for creating everything from scratch. The main thing missing is DDL for ALTER and DROP which would need to be added. But that too should be in the plugin, not in the DDL side . On Tue, Apr 28, 2026 at 8:57=E2=80=AFAM Amit Kapila wrote: > On Mon, Apr 27, 2026 at 11:45=E2=80=AFAM Dilip Kumar > wrote: > > > > On Tue, Apr 21, 2026 at 4:45=E2=80=AFAM Masahiko Sawada > wrote: > > > > > > The overall idea of the previous patch set was to implement DDL > > > deparsing and utilize it for DDL replication. It converted a parse > > > tree into a JSON string. For instance, if a user executes "DROP TABLE > > > t1", the deparser generates from its parse tree: > > > > > > {DROPSTMT :objects (("t1")) :removeType 41 :behavior 0 :missing_ok > > > false :concurrent false} > > > > > > to: > > > > > > {"fmt": "DROP TABLE %{objidentity}s", "objidentity": "public.t1"} > > > > > > This JSON string is self-documenting, meaning someone who gets it can > > > easily reconstruct the original DDL with schema-qualified object > > > names. In a dedicated event trigger for logical replication, we > > > deparsed the parse tree of a DDL, wrote it into a WAL record, and the= n > > > the logical decoding processed it similarly to DML changes. > > > > I think there was also a discussion on whether to use JSON vs the > > existing infrastructure of converting nodes to strings. Although the > > JSON is standard format and might provide more flexibility using > > existing format avoid extra maintence burden. > > > > Yes, the discussion related to node-to-string and the pros and cons of > the deparse approach are detailed in email [1]. I think some of these > points could be also related to the new approach as well. > > [1] - > https://www.postgresql.org/message-id/OS0PR01MB571684CBF660D05B63B4412C94= AB9%40OS0PR01MB5716.jpnprd01.prod.outlook.com > > -- > With Regards, > Amit Kapila. > > > --000000000000b728a506508a9c54 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
My high-level understanding is, that we should=C2=A0first = clearly answer these two questions

1. What does Logical Decodng infr= astructure make available to the decoding plugin call-backs

2. What= =C2=A0the callbacks themselves do in case of DDL

My answer to the 1.= is "everything, as it should be the plugin that decides what it needs= ". It does not mean that we should always prepack everything with spec= ial logical decoding structures, but there should be a way to get at anythi= ng that is available in the WAL at least. The result is in WAL, and for DDL= it is also in the system tables themselves, in proper time-travel way.
=
For 2. I would prefer to "deparse" the DDL from actual system= tables at that snapshot. In logical decoding the system tables are special= in that we keep the actual table content and have real time=C2=A0travel ca= pabilities on them. This should allow us to use the code we already have in= pg_dump for extracting the "status quo DDL" meaning the DDL for = creating everything from scratch. The main thing missing is DDL for ALTER a= nd DROP which would need to be added. But that too should be in the plugin,= not in the DDL side .




On Tue, Apr 28,= 2026 at 8:57=E2=80=AFAM Amit Kapila <amit.kapila16@gmail.com> wrote:
On Mon, Apr 27, 2026 at 11:45=E2=80=AFAM D= ilip Kumar <d= ilipbalaut@gmail.com> wrote:
>
> On Tue, Apr 21, 2026 at 4:45=E2=80=AFAM Masahiko Sawada <sawada.mshk@gmail.com&= gt; wrote:
> >
> > The overall idea of the previous patch set was to implement DDL > > deparsing and utilize it for DDL replication. It converted a pars= e
> > tree into a JSON string. For instance, if a user executes "D= ROP TABLE
> > t1", the deparser generates from its parse tree:
> >
> > {DROPSTMT :objects (("t1")) :removeType 41 :behavior 0 = :missing_ok
> > false :concurrent false}
> >
> > to:
> >
> > {"fmt": "DROP TABLE %{objidentity}s", "o= bjidentity": "public.t1"}
> >
> > This JSON string is self-documenting, meaning someone who gets it= can
> > easily reconstruct the original DDL with schema-qualified object<= br> > > names. In a dedicated event trigger for logical replication, we > > deparsed the parse tree of a DDL, wrote it into a WAL record, and= then
> > the logical decoding processed it similarly to DML changes.
>
> I think there was also a discussion on whether to use JSON vs the
> existing infrastructure of converting nodes to strings.=C2=A0 Although= the
> JSON is standard format and might provide more flexibility using
> existing format avoid extra maintence burden.
>

Yes, the discussion related to node-to-string and the pros and cons of
the deparse approach are detailed in email [1]. I think some of these
points could be also related to the new approach as well.

[1] - https://www.postgresql.org/message-id/OS0PR01MB5716= 84CBF660D05B63B4412C94AB9%40OS0PR01MB5716.jpnprd01.prod.outlook.com

--
With Regards,
Amit Kapila.


--000000000000b728a506508a9c54--