MIME-Version: 1.0
References: <19354-eefe6d8b3e84f9f2@postgresql.org> <CA+TgmoaRGSezRaA7x00X495Qho8WGTzggbDSUt-JsruXceZWug@mail.gmail.com>
 <CA+zULE4L4rA2DLAcfy=eQL7w_ZexV4P5zpQRbP=_qrhJBEOzjg@mail.gmail.com> <2292889.1765846569@sss.pgh.pa.us>
In-Reply-To: <2292889.1765846569@sss.pgh.pa.us>
From: VASUKI M <vasukianand0119@gmail.com>
Date: Tue, 16 Dec 2025 11:53:48 +0530
Message-ID: <CAE2r8H5vaSyaC_t1FcpHBo-BB_=SrFj7GFnOC-SxC6WDf5c9VA@mail.gmail.com>
Subject: Re: BUG #19354: JOHAB rejects valid byte sequences
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Jeroen Vermeulen <jtvjtv@gmail.com>, Robert Haas <robertmhaas@gmail.com>, 
	pgsql-bugs@lists.postgresql.org
Content-Type: multipart/alternative; boundary="00000000000055a4eb06460bc48e"
Archived-At: <https://www.postgresql.org/message-id/CAE2r8H5vaSyaC_t1FcpHBo-BB_%3DSrFj7GFnOC-SxC6WDf5c9VA%40mail.gmail.com>
Precedence: bulk

--00000000000055a4eb06460bc48e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks all,That analysis makes a lot of sense.

Given the lack of a clear spec,the existence of multiple JOHAB variants,and
how long this has apparently been "working" without anyone noticing,IMHO
desupporting it does seem like the least risky option.At this point,trying
to fix JOHAB variants feels like opening a pretty big can of
worms,especially with the potential for dump/reload surprises or subtle
parsing/security issues.

I don't have additional data to add,but +1 on removal or deprecation being
a reasonable outcome here,given how obscure and effectively dead the
encoding is nowadays.

Thanks for digging into this.

Cheers,
Vasuki M

On Tue, Dec 16, 2025 at 11:46=E2=80=AFAM Tom Lane <tgl@sss.pgh.pa.us> wrote=
:

> Jeroen Vermeulen <jtvjtv@gmail.com> writes:
> > This bit worries me: "TlOther, vendor-defined, Johab variants also
> exist" =E2=80=94
> > such as an EBCDIC-based one and a stateful one!
>
> Yeah.  So what we have here is:
>
> 1. Our JOHAB implementation has apparently been wrong since day one.
>
> 2. Wrongness may be in the eye of the beholder, since there are
> multiple versions of JOHAB.
>
> 3. Your complaint is the first, AFAIR.
>
> 4. That wikipedia page says "Following the introduction of Unified
> Hangul Code by Microsoft in Windows 95, and Hangul Word Processor
> abandoning Johab in favour of Unicode in 2000, Johab ceased to be
> commonly used."
>
> Given these things, I wonder if we shouldn't desupport JOHAB
> rather than attempt to fix it.  Fixing would likely be a significant
> amount of work: if we don't even have the character lengths right,
> how likely is it that our conversions to other character sets are
> correct?  I also worry that if different PG versions have different
> ideas of the mapping, there could be room for dump/reload problems,
> and maybe even security problems related to the backslash issue.
>
>                         regards, tom lane
>
>
>
>
>

--00000000000055a4eb06460bc48e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><p>Thanks all,That analysis makes a=C2=A0lot of sense.<br>=
<br>Given the lack of a clear spec,the existence of multiple JOHAB variants=
,and how long this has apparently been &quot;working&quot; without anyone n=
oticing,IMHO desupporting=C2=A0it does seem like the least risky option.At =
this point,trying to fix JOHAB variants feels like opening a pretty big can=
 of worms,especially with the potential for dump/reload surprises or subtle=
 parsing/security issues.<br><br>I don&#39;t have additional data to add,bu=
t=C2=A0+1 on removal or deprecation being a reasonable outcome here,given h=
ow obscure and effectively dead the encoding is nowadays.<br><br>Thanks for=
 digging into this.</p><p>Cheers,<br>Vasuki M</p></div><br><div class=3D"gm=
ail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On T=
ue, Dec 16, 2025 at 11:46=E2=80=AFAM Tom Lane &lt;<a href=3D"mailto:tgl@sss=
.pgh.pa.us">tgl@sss.pgh.pa.us</a>&gt; wrote:<br></div><blockquote class=3D"=
gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20=
4,204,204);padding-left:1ex">Jeroen Vermeulen &lt;<a href=3D"mailto:jtvjtv@=
gmail.com" target=3D"_blank">jtvjtv@gmail.com</a>&gt; writes:<br>
&gt; This bit worries me: &quot;TlOther, vendor-defined, Johab variants als=
o exist&quot; =E2=80=94<br>
&gt; such as an EBCDIC-based one and a stateful one!<br>
<br>
Yeah.=C2=A0 So what we have here is:<br>
<br>
1. Our JOHAB implementation has apparently been wrong since day one.<br>
<br>
2. Wrongness may be in the eye of the beholder, since there are<br>
multiple versions of JOHAB.<br>
<br>
3. Your complaint is the first, AFAIR.<br>
<br>
4. That wikipedia page says &quot;Following the introduction of Unified<br>
Hangul Code by Microsoft in Windows 95, and Hangul Word Processor<br>
abandoning Johab in favour of Unicode in 2000, Johab ceased to be<br>
commonly used.&quot;<br>
<br>
Given these things, I wonder if we shouldn&#39;t desupport JOHAB<br>
rather than attempt to fix it.=C2=A0 Fixing would likely be a significant<b=
r>
amount of work: if we don&#39;t even have the character lengths right,<br>
how likely is it that our conversions to other character sets are<br>
correct?=C2=A0 I also worry that if different PG versions have different<br=
>
ideas of the mapping, there could be room for dump/reload problems,<br>
and maybe even security problems related to the backslash issue.<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 regards, tom lane<br>
<br>
<br>
<br>
<br>
</blockquote></div>

--00000000000055a4eb06460bc48e--