Resent-From: POSTGRES mailing list <postman@postgres.Berkeley.EDU>
Resent-Message-Id: <199403071522.HAA19378@nobozo.CS.Berkeley.EDU>
Sender: owner-postman@postgres.Berkeley.EDU
Date: Mon, 7 Mar 94 10:26:22 EST
From: suresh@nlm.nih.gov (Suresh Srinivasan)
Message-Id: <9403071526.AA11157@nls2.csb>
To: postgres@postgres.Berkeley.EDU
Subject: Core dump when copy'ing large file
Cc: rodgers@nls2.nlm.nih.gov
Resent-To: postgres-dist@postgres.Berkeley.EDU
Resent-Date: Mon, 07 Mar 94 07:22:22 -0800
Resent-XMts: smtp

Hi Postgres'ers:

I'm using the precompiled Sparc binary version 4.1 of
Postgres to index a largish image database.

There are about 60,000+ images in the collection with
cataloging information associated with each.  I'm trying
to create a word index on this catalog using the following:

  create Word (word = text, pos = int4, field = char, frame = int4) \g
  define index WordIndex on Word using btree (word text_ops) \g

Then, I load in the words from a pre-created file with:

  copy Word from "/foo/bar/Word.cl" \g

Note that even after removing stopwords, this is a large file
with well over 500,000 entries.
Also, there are large numbers of words that are repeated
several thousand times.  For example, here's the head of
the (reverse sorted) frequency file:

13254 hospital
8104 army
7774 full
7577 face
7487 head
7197 view
6744 shoulders
5043 pose
4627 france
4048 left
...

Well, after running for about 5 hrs on a Sparc 2, I get a
core dump (?() at ab058 using adb) with the following message
on the console:

  Error: No response from the backend, exiting..

The Word file in ~postgres/data/base/db when it died is 37M and the
WordIndex file is 20M.  All retrievals on Word return empty.
Before attempting another solution, I was hoping some of you
have seen this before.

Thanks in advance for any help..

--Suresh