Feedback-ID: i76984098:Fastmail
Message-ID: <ecd7305e-888b-43bb-9e16-4297c93e4904@aklaver.com>
Date: Tue, 19 May 2026 07:44:57 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: scaling up from t1n to 60 million records
To: Martin Mueller <martinmueller@northwestern.edu>,
 "pgsql-general@postgresql.org" <pgsql-general@postgresql.org>
References: 
 <CY8PR05MB1010861EAD48ED098786C9690C4002@CY8PR05MB10108.namprd05.prod.outlook.com>
Content-Language: en-US
From: Adrian Klaver <adrian.klaver@aklaver.com>
In-Reply-To: 
 <CY8PR05MB1010861EAD48ED098786C9690C4002@CY8PR05MB10108.namprd05.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/ecd7305e-888b-43bb-9e16-4297c93e4904%40aklaver.com>
Precedence: bulk

On 5/19/26 7:27 AM, Martin Mueller wrote:
> I use Postgres with a GUI frontend (Aquafold) as a very large 
> spreadsheet on steroids that analyzes rare or defective spellings in a 
> corpus of 65,000 texts and1.5 billion words.  I typically extract  data 
> from the corpus with python scripts, turn them into tables and load them 
> into the database.
> 
> 
> On my Mac with 32 GB of memory performance is OK with queries that 
> typically within seconds extract data rows from tables  with up to ten 
> million rows.  If the result set is large, I suspect that most of time 
> machine's time is spent displaying result sets. I have used indexing 
> sparingly. While it helps, the time savings often don't matter much.

This is going to need more information:

1) Postgres version.

2) The table schema including indexes.

3) An example of the query.

4) Where you are measuring the time.

5) The client you are displaying the results in.

> 
> 
> I am thinking about scaling up to table with about 60 million rows.  Are 
> there things to do or watch out for? Or should I proceed on the 
> assumption that that 60 million records are within scope and that the 
> added timecost is roughly linear?
> 
> Martin Mueller
> 
> Professor emeritus of English and Classics
> 
> Northwestern University
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com