Resent-From: POSTGRES mailing list <postman@postgres.Berkeley.EDU>
Resent-Message-Id: <199311221704.JAA19683@nobozo.CS.Berkeley.EDU>
Sender: owner-postman@postgres.Berkeley.EDU
From: John Peter Lee <jlee@cs.uml.edu>
Message-Id: <199311221704.AA29717@cs.uml.edu>
Subject: why my backend crashes ?
To: postgres@postgres.Berkeley.EDU
Date: Mon, 22 Nov 1993 12:04:42 -0500 (EST)
Content-Type: text
Content-Length: 1691      
Resent-To: postgres-dist@postgres.Berkeley.EDU
Resent-Date: Mon, 22 Nov 93 09:04:45 -0800
Resent-XMts: smtp

Hi:

  This is further along the lines of the problems I've been experiencing
  while trying to build a database. Here is some more info:

  I have a 17MB raw database that I am transferring to Postgres. Each
  append (through libPQ) consists of 49 attributes, all floating-point
  (float4) types. There are ~55,000 appends total that must be made,
  and I perform them in sequences of 18, because there are 18 data values
  for each attribute of the 3035 companies in the DB (3035 x 18 == ~55,000).

  I prepend each single append with PQexec ("begin"), and postpend with
  PQexec("end"). 

  Every set of 18 appends (an entire 18 years of data for a company)
  takes about 30 seconds, which translates to about 25 hours for
  the whole DB to be transferred into postgres.

  Postgres seems to crash after about 6 hours of this; whatever has
  been appended is still in the database. After re-running and monitoring
  the run-time size via "ps", I do notice that the run-time size is 
  increasing with time, which leads me to believe that there might be 
  some type of memory leak within Postgres. Could you possibly give 
  me some insight into this ???

  What I've done now is to perform a PQreset() after every 500 companies
  (9000 appends of 49 float4's), and hope for the best. Is this solution
  viable ?

  thanks.

  jp

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
JP Lee               Institute for Visualization and Perception Research	
jlee@cs.uml.edu                    University of Massachusetts at Lowell 
(508) 934-3384			      1 University Ave. Lowell, MA 01854

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~