public inbox for pgsql-general@postgresql.org
help / color / mirror / Atom feedFrom: Amit Kapila <amit.kapila16@gmail.com>
To: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
Cc: shveta malik <shveta.malik@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>
Cc: Ajin Cherian <itsajin@gmail.com>
Cc: Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>
Cc: Runqi Tian <runqidev@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: li jie <ggysxcq@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Japin Li <japinli@hotmail.com>
Cc: rajesh singarapu <rajesh.rs0541@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Zheng Li <zhengli10@gmail.com>
Subject: Re: Support logical replication of DDLs
Date: Fri, 28 Apr 2023 17:11:03 +0530
Message-ID: <CAA4eK1LDyivFBt3w2bQcL=N_DfYA5yRSX2AeHG1Q-5mA4A6MTw@mail.gmail.com> (raw)
In-Reply-To: <OS0PR01MB57167AB450D6B3C5FF2C90EF94649@OS0PR01MB5716.jpnprd01.prod.outlook.com>
References: <CAAD30ULCxqOJp0sffm_y9jNC4BVPYv7Q7_va_JE8qyfRXkfu+g@mail.gmail.com>
<CAFPTHDaaewvYUznZD1YjUQnvHycgvgMKNK4w=V8Q-8MKTpVDrw@mail.gmail.com>
<OS3PR01MB6275025C397CC22E446B1F589EBC9@OS3PR01MB6275.jpnprd01.prod.outlook.com>
<OS0PR01MB5716765B2D38786DA3943E0294809@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAFPTHDbAJPccMcZnOraBy14hM6qBJxYqcRwK6iV6=gtL6VZQwQ@mail.gmail.com>
<CALDaNm3NUO8ofK64N7HMtNmUP=52R8_jWzrekqAm7m7wqZjwaQ@mail.gmail.com>
<CALDaNm3XUKfD+nD1AVvSuZyUY_zRk_eyz+Pt9t13N8WXViR6pw@mail.gmail.com>
<3032112.1679865718@sss.pgh.pa.us>
<CAA4eK1K3VXfTWXbLADcH81J==7ussvNdqLFHN68sEokDPueu7w@mail.gmail.com>
<CAA4eK1++Y7a2SQq55DXT6neghZgj3j+pQ74=8zfT3k8Tkdj0ZA@mail.gmail.com>
<OS0PR01MB571638B29E37EB9A48507DAE94889@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB571636F0F1307AB1A868B06A948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB57169AB2355A90C9FA87DE75948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CALDaNm2vBN8oMv-7G=DH5rR-u40JGbR9aP4B6nwr71qw17rPFA@mail.gmail.com>
<OS0PR01MB571614EE7EEDBF7049DA4EAE94939@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAJpy0uCSCtWwb_LyjXQT65c-UksYhp-4U0QpimtEho_wxjzkog@mail.gmail.com>
<OS0PR01MB57168840ABFAB9E9BB28A54B94969@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB571643602A8B7226B558DBAA94969@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1LyEqEq0XCeCsd9ZmTHyeshHbv=BiRTD=1mmk+YtxwpkQ@mail.gmail.com>
<OS0PR01MB5716A9E1433DA1FD3EE38069949C9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAJpy0uDb2mDJtLNFXzUY4911qRZOvj6Q8pu4xFh4BMYBeOSPow@mail.gmail.com>
<CAJpy0uAA0SQ0kPA5bXmrW=32p0bwFCifoKb5OSgteTjGggEkLA@mail.gmail.com>
<CAJpy0uB7f2GxPNor5iTT-30JuD-p-gvnsMZG9tiiHN+DHJj0RQ@mail.gmail.com>
<OS0PR01MB57167AB450D6B3C5FF2C90EF94649@OS0PR01MB5716.jpnprd01.prod.outlook.com>
On Tue, Apr 25, 2023 at 9:28 AM Zhijie Hou (Fujitsu)
<houzj.fnst@fujitsu.com> wrote:
>
I have a few high-level comments on the deparsing approach used in the
patch. As per my understanding, we first build an ObjTree from the DDL
command, then convert the ObjTree to Jsonb which is then converted to
a JSON string. Now, in the consecutive patch, via publication event
triggers, we get the JSON string via the conversions mentioned, WAL
log it, which then walsender will send to the subscriber, which will
convert the JSON string back to the DDL command and execute it.
Now, I think we can try to eliminate this entire ObjTree machinery and
directly from the JSON blob during deparsing. We have previously also
discussed this in an email chain at [1]. I think now the functionality
of JSONB has also been improved and we should investigate whether it
is feasible to directly use JSONB APIs to form the required blob.
The other general point is that one of the primary reasons to convert
DDL into JSONB blob is to allow replacing the elements. For example,
say on the publisher, the table is in Schema A and then on the
subscriber the same table is in Schema B, so, we would like to change
the Schema in the DDL before replaying it, or we want to change the
persistence of table to UNLOGGED before replaying the DDL on the
subscriber. Is it possible to have such an API exposed from this
module so that we can verify if that works? It can be a separate patch
though.
[1] - https://www.postgresql.org/message-id/CAB7nPqRX6w9UY%2B%3DOy2jqTVwi0hqT2y4%3DfUc7fNG4U-296JBvYQ%40ma...
--
With Regards,
Amit Kapila.
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: amit.kapila16@gmail.com, houzj.fnst@fujitsu.com, shveta.malik@gmail.com, vignesh21@gmail.com, itsajin@gmail.com, wangw.fnst@fujitsu.com, runqidev@gmail.com, smithpb2250@gmail.com, tgl@sss.pgh.pa.us, ggysxcq@gmail.com, dilipbalaut@gmail.com, alvherre@alvh.no-ip.org, sawada.mshk@gmail.com, japinli@hotmail.com, rajesh.rs0541@gmail.com, pgsql-hackers@lists.postgresql.org, zhengli10@gmail.com
Subject: Re: Support logical replication of DDLs
In-Reply-To: <CAA4eK1LDyivFBt3w2bQcL=N_DfYA5yRSX2AeHG1Q-5mA4A6MTw@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox