Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wQTYg-001aGc-17 for pgsql-admin@arkaria.postgresql.org; Fri, 22 May 2026 17:21:14 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1wQTYe-00Dqcy-15 for pgsql-admin@arkaria.postgresql.org; Fri, 22 May 2026 17:21:13 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1wQTYd-00Dqcp-2n for pgsql-admin@lists.postgresql.org; Fri, 22 May 2026 17:21:12 +0000 Received: from mail-oa1-x2f.google.com ([2001:4860:4864:20::2f]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1wQTYd-00000000Ii0-0EEE for pgsql-admin@lists.postgresql.org; Fri, 22 May 2026 17:21:11 +0000 Received: by mail-oa1-x2f.google.com with SMTP id 586e51a60fabf-43496e6a964so5509387fac.2 for ; Fri, 22 May 2026 10:21:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1779470470; cv=none; d=google.com; s=arc-20240605; b=FHHQCixl87PmeDSgkJKYL96BkeyrHfagAnLin6Y0VHSJwhRokLkKDuPVqkvYRLMD5c R7Qy4P2CEKmDmF2cgso+Py3QpxFANvjtiPNPnry/fGXI3m1+izp2Ab/HGOpY5FQdHb/2 3ZltuJwSBykNt1O5gys81VrL0vlD0pcUNeom5inKQzE0rc3OKJ1AgniNAw44T+HovnqM rR/N2FlSrkOOgtcBnaLWFAvVghrcqazU/sWgKrCQ04P6Oc7+8imYOF6AX1g6R0ttHhvn J6A2Iit1JG/vz4mhE9xz2NhYvHvBo+K8L2xOy4LusvdUtIbNaNR14O9OBIXUPYQAPDFV jrkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=1whsWpMh7MB2w+Qh7vmmfjYu871KR6uaC/jOx58yE60=; fh=3YqLbvwuA2UUCb/PfrFLKGpVXVeV0HCtGZNHU4149wU=; b=Jqg6TKVXa1B4wUc/vST101E1tml4Fzg4RSKFhfozCTw12Y/sq/iNKFD2ovcaBcrfVO DvuyTzZYzUZHeWT6llu2rjFgMenM7+H0lAaQcaQmP97o6lrNIeRzsJG78SeskpxhLdVq wuCvxy75/3gAQRb2NIYXSCHUv8C+mVtnPc8e9b8BeFvBj5U7EeiM4i9ZvnfIEgeBxbpP 6i9PU9MpywuxeAxGTKD+T5DnuHzkDvRpfSV08U/acpWAea58Q/8jhg+E5d0C5w9G4uQf CAOUyO1QPAuRdyzSGhukb1HbQToD5HafBUUKIx5DOp5RcZuAo3vtyBAC810LxLFnuRkb 0kzA==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1779470470; x=1780075270; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=1whsWpMh7MB2w+Qh7vmmfjYu871KR6uaC/jOx58yE60=; b=Ktgn7mRy8EOS1rTWGDHfcmXLLKREJ7EUdj6ToRyO103TaRVV+++KjqVuoX6QwbQ+iP K9AHlQfjShPVmnY05jjj6bgHgOo+K5bCa23rP/x7fy6hlDcKDmwD+n6NqorPEoYFvwLm keXF3bgSKOoSd3LJNeUx/sxFYK7S21m/p+c+bqCT6HPL+JAmzGaYXfXSmOwOmjYixWMK AoYbP3c48oXkot38tnizzw7sCZfutEDNW+27j0rQGUnk5KdlLTiUogDQIesJydNbcC9l dr38cfwQjoG6yGlWHsUoY+nc/JFaq1AA6xboh9ucMeq+4Epa6qqxZtg9D84+YaUWddlz G5Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779470470; x=1780075270; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=1whsWpMh7MB2w+Qh7vmmfjYu871KR6uaC/jOx58yE60=; b=HqveSm9ZxLGqR84LMW5uOKD9/K7ls4wbIqSeAyVP2rEgQzdpaGUxP3pf6i4p47m6eW dwSa65tgiaU0JRNrAxSXmTHOXxl+8evYJjW1yakj6JbIKzE3rdzrHIqtcftRHPV70+NA R3jihC1+RwD/cBMXgmXY64sEqyYMuQOfi76GXj7P7hQWutd4YucxCW8unsAPDS9KghPA VMTSUUVGuq+xCETsBl305DR+qjFYjYiKjbu6bsu2+TORjW68ASzgEhL1q+D1sZT4wKHn lHh/tzFFyhoGqfzjlc98om5BlzEqCEKsSTADhr0Y8SFIyri0qSe69LUC0zGkR4Mk35ZD ZgvQ== X-Gm-Message-State: AOJu0YyQJSji67Fejxp0VDI/CAExuF5ZLH9CbyCaaZ5Vvpwb1p5YUug3 rOceHlQVoManYWLncRTXDXghlN515Lx9MeGjYM+ih+4BwjlrufMNIC0dSTPdbfW5LkfE0lXq7jb pIKwj1RfKoflqRQJD5KhQZJbQhx6w6F1v3RYY X-Gm-Gg: Acq92OFv3r/StLjzjS87SBTKOUtItc0zTe7kLAoBzBVN0I7/exJ2TUNx+MrPStHQBlH C9BiJMEw74nMD1XA9V/t/EEtUnzgKR++xnHHUYwpgPiLLrmeziaTezle0ZGpYqEcuzYJ4tB7GHl DpnuHeY4VsnDfQJZf0n9NU8r3WReuydIZWN+w2n2nNfom9wGOJ0CEA888MBbbVfzI0GOy20Sf6u kevVspFWcUL8VtSqVs3MvYl0hoOU/tPesOv6rZGMPSKmuqJB8PgDFINdNsK53MGDEhS8EHnIAPy DYX3djExtePEEKdj+ws= X-Received: by 2002:a05:6870:90cb:b0:3e8:970e:d4f7 with SMTP id 586e51a60fabf-43b5aa4f734mr2490952fac.11.1779470470413; Fri, 22 May 2026 10:21:10 -0700 (PDT) MIME-Version: 1.0 References: <26493.1779468779@sss.pgh.pa.us> In-Reply-To: <26493.1779468779@sss.pgh.pa.us> From: Ron Johnson Date: Fri, 22 May 2026 13:20:58 -0400 X-Gm-Features: AVHnY4LxXEzvdf0wJ14MVZ0LldfUzjj6nO4MPUKlwKy3DUAo1ogf-9WvemSd7Os Message-ID: Subject: Re: Request For Feature: pg_dump To: Tom Lane Cc: Pgsql-admin Content-Type: multipart/alternative; boundary="00000000000047055a06526b41b5" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --00000000000047055a06526b41b5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, May 22, 2026 at 12:53=E2=80=AFPM Tom Lane wrote= : > Ron Johnson writes: > > In --format=3Ddirectory mode, remove .dat files with zero data records,= and > > mark that table's toc.dat entry that it's an empty table. > > > Justification: *lots* of empty tables means *lots* of teeny-tiny files = in > > the DB's dump directory. That unnecessarily bloats the fs, and makes "= du > > -c" really really slow. > > Evidence please? Most file systems that I've looked at optimize > zero-size files pretty well. > They aren't zero bytes. It's those pesky 5 (or 14 or whatever size that gzip and lz4 produces) byte files. 66 thousand tiny files plus 8 thousand files with data in them makes for a 2.4MB directory. That's big and slow. $ find . -size 14c | wc 66180 66180 1191240 $ zstd -dk 2115841.dat.zst 2115841.dat.zst : 5 bytes $ cat 2115841.dat \. $ dir | grep " 14 " | head -n20 -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115841.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115842.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115843.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115844.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115845.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115851.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115899.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115901.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115902.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115903.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115905.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115907.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115909.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115913.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115915.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115917.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115919.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115923.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115926.dat.zst -rw-r--r-- 1 postgres postgres 14 2026-05-22 00:50:30 2115931.dat.zst --=20 Death to , and butter sauce. Don't boil me, I'm still alive. lobster! --00000000000047055a06526b41b5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Fri, May 22, 2026 at 12:53=E2=80=AFPM = Tom Lane <tgl@sss.pgh.pa.us>= wrote:
Ron Johnson <ronljohnsonjr@gmail.com> writ= es:
> In --format=3Ddirectory mode, remove .dat files with zero data records= , and
> mark that table's toc.dat entry that it's an empty table.

> Justification: *lots* of empty tables means *lots* of teeny-tiny files= in
> the DB's dump directory.=C2=A0 That unnecessarily bloats the fs, a= nd makes "du
> -c" really really slow.

Evidence please?=C2=A0 Most file systems that I've looked at optimize zero-size files pretty well.

They aren&= #39;t zero bytes.
It's those pesky 5 (or 14 or whatever size = that gzip and lz4 produces) byte files.=C2=A0 66 thousand tiny files plus 8= thousand files with data in them makes for a 2.4MB directory.=C2=A0 That&#= 39;s big and slow.

$ find= . -size 14c | wc
=C2=A0 66180 =C2=A0 66180 1191240

$ zstd= -dk 2115841.dat.zst
2115841.dat.zst =C2=A0 =C2=A0 : 5 bytes=C2=A0 =C2= =A0 =C2=A0=C2=A0

$ cat 2115841.dat
\.

=
$ dir | grep " 14 " | head -n20
= -rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00= :50:30 2115841.dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2= =A0 =C2=A0 14 2026-05-22 00:50:30 2115842.dat.zst
-rw-r--r-- 1 postgres = postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115843.dat.zst=
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-2= 2 00:50:30 2115844.dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 = =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115845.dat.zst
-rw-r--r-- 1 postgr= es postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115851.dat.= zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-0= 5-22 00:50:30 2115899.dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2= =A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115901.dat.zst
-rw-r--r-- 1 po= stgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115902.= dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 20= 26-05-22 00:50:30 2115903.dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 = =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115905.dat.zst
-rw-r--r-- 1= postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 21159= 07.dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14= 2026-05-22 00:50:30 2115909.dat.zst
-rw-r--r-- 1 postgres postgres =C2= =A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115913.dat.zst
-rw-r--r= -- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2= 115915.dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2= =A0 14 2026-05-22 00:50:30 2115917.dat.zst
-rw-r--r-- 1 postgres postgre= s =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115919.dat.zst
-rw= -r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50= :30 2115923.dat.zst
-rw-r--r-- 1 postgres postgres =C2=A0 =C2=A0 =C2=A0 = =C2=A0 14 2026-05-22 00:50:30 2115926.dat.zst
-rw-r--r-- 1 postgres post= gres =C2=A0 =C2=A0 =C2=A0 =C2=A0 14 2026-05-22 00:50:30 2115931.dat.zst
=C2=A0
-- <= /span>
Death= to <Redacted>, and butter sauce.
Don't boil me, I'm stil= l alive.
<Redacted> lobster!
--00000000000047055a06526b41b5--