public inbox for pgsql-general@postgresql.org
help / color / mirror / Atom feedFrom: Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: shveta malik <shveta.malik@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>
Cc: Ajin Cherian <itsajin@gmail.com>
Cc: Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>
Cc: Runqi Tian <runqidev@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: li jie <ggysxcq@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Japin Li <japinli@hotmail.com>
Cc: rajesh singarapu <rajesh.rs0541@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Zheng Li <zhengli10@gmail.com>
Subject: RE: Support logical replication of DDLs
Date: Mon, 17 Apr 2023 12:02:15 +0000
Message-ID: <OS0PR01MB5716A9E1433DA1FD3EE38069949C9@OS0PR01MB5716.jpnprd01.prod.outlook.com> (raw)
In-Reply-To: <CAA4eK1LyEqEq0XCeCsd9ZmTHyeshHbv=BiRTD=1mmk+YtxwpkQ@mail.gmail.com>
References: <CAAD30ULCxqOJp0sffm_y9jNC4BVPYv7Q7_va_JE8qyfRXkfu+g@mail.gmail.com>
<CAFPTHDaaewvYUznZD1YjUQnvHycgvgMKNK4w=V8Q-8MKTpVDrw@mail.gmail.com>
<OS3PR01MB6275025C397CC22E446B1F589EBC9@OS3PR01MB6275.jpnprd01.prod.outlook.com>
<OS0PR01MB5716765B2D38786DA3943E0294809@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAFPTHDbAJPccMcZnOraBy14hM6qBJxYqcRwK6iV6=gtL6VZQwQ@mail.gmail.com>
<CALDaNm3NUO8ofK64N7HMtNmUP=52R8_jWzrekqAm7m7wqZjwaQ@mail.gmail.com>
<CALDaNm3XUKfD+nD1AVvSuZyUY_zRk_eyz+Pt9t13N8WXViR6pw@mail.gmail.com>
<3032112.1679865718@sss.pgh.pa.us>
<CAA4eK1K3VXfTWXbLADcH81J==7ussvNdqLFHN68sEokDPueu7w@mail.gmail.com>
<CAA4eK1++Y7a2SQq55DXT6neghZgj3j+pQ74=8zfT3k8Tkdj0ZA@mail.gmail.com>
<OS0PR01MB571638B29E37EB9A48507DAE94889@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB571636F0F1307AB1A868B06A948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB57169AB2355A90C9FA87DE75948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CALDaNm2vBN8oMv-7G=DH5rR-u40JGbR9aP4B6nwr71qw17rPFA@mail.gmail.com>
<OS0PR01MB571614EE7EEDBF7049DA4EAE94939@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAJpy0uCSCtWwb_LyjXQT65c-UksYhp-4U0QpimtEho_wxjzkog@mail.gmail.com>
<OS0PR01MB57168840ABFAB9E9BB28A54B94969@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<OS0PR01MB571643602A8B7226B558DBAA94969@OS0PR01MB5716.jpnprd01.prod.outlook.com>
<CAA4eK1LyEqEq0XCeCsd9ZmTHyeshHbv=BiRTD=1mmk+YtxwpkQ@mail.gmail.com>
On Monday, April 10, 2023 7:20 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, Apr 7, 2023 at 8:52 AM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
> > Sorry, there was a miss when rebasing the patch which could cause the
> > CFbot to fail and here is the correct patch set.
> >
>
> I see the following note in the patch: "Note: For ATTACH/DETACH PARTITION,
> we haven't added extra logic on the subscriber to handle the case where the
> table on the publisher is a PARTITIONED TABLE while the target table on the
> subscriber side is a NORMAL table. We will research this more and improve it
> later." and wonder what should we do about this. I can think of the following
> possibilities: (a) Convert a non-partitioned table to a partitioned one and then
> attach the partition; (b) Add the partition as a separate new table; (c) give an
> error that table types mismatch. For Detach partition, I don't see much
> possibility than giving an error that no such partition exists or something like
> that. Even for the Attach operation, I prefer (c) as the other options don't seem
> logical to me and may add more complexity to this work.
>
> Thoughts?
I also think option (c) makes sense and is same as the latest patch's behavior.
Attach the new version patch set which include the following changes:
- Disallow using non-immutable function when executing ALTER TABLE TYPE/ADD
COLUMN DEFALUT command which could cause table rewrite, Because it could
result in different data between publisher and subscriber. The previous
design of rewrite command is to first replicate the DDL command and then
convert the incoming rewrite insert to updates and replicate them to
subscriber to keep the data consistent, but this doesn't work if the column
type of replica identity column is changed which means we cannot find the old
value in the rewrite inserts.
- Don't log DROP if it's not an original deletion target.
- pg_dump handling of internal event triggers. Now it goes by.
pg_event_trigger's new column (evtisinternal) instead of name parsing.
- Reattach the test_deparser patches and split them into table, index and other
part. (TODO: need to write tests about the replication of index command)
- Fix a bug that the expression used in "alter type using expr" was not schema
qualified during deparsing.
- Restrict Alter (rename) event trigger operations on internally created event
triggers. We allow Alter enable/disable internal event trigger, because it's
necessary to enable the event trigger on replica mode so that it can be fired
in apply worker.
- Address comments from Amit[1][2].
- Address comments from Wang[3].
- Fix typos.
Thanks Shveta for helping address comments and improve the pg_dump handling.
In this version, to make the patch set easier for review and rebase, I didn't
include the patch related to the deparsing and replication of the rest ddl
commands(objects other than table and index). We can add it back later when the
main patches get into shape.
[1]https://www.postgresql.org/message-id/CAA4eK1JJYkwk1rz0O2J6OUK8qb3bZV5P7RwK933DKFkgu56nXQ%40mail.gma...
[2]https://www.postgresql.org/message-id/CAA4eK1Lmifb6-JeCiZFQisu2JTVGokvSLFEmx-cchpLLyKc8TA%40mail.gma...
[3]https://www.postgresql.org/message-id/OS3PR01MB62759906D280A4EEC42570D89E9A9%40OS3PR01MB6275.jpnprd0...
Best Regards,
Hou zj
Attachments:
[application/x-gzip] ddl-replication-2023_04_17.tar.gz (157.0K, 2-ddl-replication-2023_04_17.tar.gz)
download
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: houzj.fnst@fujitsu.com, amit.kapila16@gmail.com, shveta.malik@gmail.com, vignesh21@gmail.com, itsajin@gmail.com, wangw.fnst@fujitsu.com, runqidev@gmail.com, smithpb2250@gmail.com, tgl@sss.pgh.pa.us, ggysxcq@gmail.com, dilipbalaut@gmail.com, alvherre@alvh.no-ip.org, sawada.mshk@gmail.com, japinli@hotmail.com, rajesh.rs0541@gmail.com, pgsql-hackers@lists.postgresql.org, zhengli10@gmail.com
Subject: RE: Support logical replication of DDLs
In-Reply-To: <OS0PR01MB5716A9E1433DA1FD3EE38069949C9@OS0PR01MB5716.jpnprd01.prod.outlook.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox