agora inbox for postgres@postgres.berkeley.edu
help / color / mirror / Atom feedFailure of append
4+ messages / 2 participants
[nested] [flat]
* Failure of append
@ 1994-09-26 17:52 Raj Vadakkencherry <raj@CS.UCLA.EDU>
1994-09-26 19:39 ` Re: Failure of append Paul M. Aoki <aoki@cs.berkeley.edu>
0 siblings, 1 reply; 4+ messages in thread
From: Raj Vadakkencherry @ 1994-09-26 17:52 UTC (permalink / raw)
To: legacy
Hi,
I'm using libpq calls to append to a database. (PQexec("append ...")).
The append string passed to PQexec is rather long at times: 15000 chars
or more. (it is a string of approx. 1000 points, each point represented
as in 15 chars)
For such strings, the append fails.
Any suggestions on how to get around this problem ?
Raj.
==============================================================================
To add/remove yourself to/from the POSTGRES mailing list: send mail with
the subject line ADD or DEL to "postgres-request@postgres.Berkeley.EDU"
If this fails, send mail to "post_questions@postgres.Berkeley.EDU" and
a human will deal with it. DO NOT post to the "postgres" mailing list.
==============================================================================
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Failure of append
1994-09-26 17:52 Failure of append Raj Vadakkencherry <raj@CS.UCLA.EDU>
@ 1994-09-26 19:39 ` Paul M. Aoki <aoki@cs.berkeley.edu>
0 siblings, 0 replies; 4+ messages in thread
From: Paul M. Aoki @ 1994-09-26 19:39 UTC (permalink / raw)
To: Raj Vadakkencherry <raj@CS.UCLA.EDU>; +Cc: legacy
raj@CS.UCLA.EDU (Raj Vadakkencherry) writes:
> I'm using libpq calls to append to a database. (PQexec("append ...")).
> The append string passed to PQexec is rather long at times: 15000 chars
> or more. (it is a string of approx. 1000 points, each point represented
> as in 15 chars)
> For such strings, the append fails.
the parser seems to have some buffers that are currently hardwired to
8KB. look for "8192" in tmp/miscadmin.h and parser/scan.l and bump
those up to something larger.
--
Paul M. Aoki | University of California at Berkeley
aoki@CS.Berkeley.EDU | Dept. of EECS, Computer Science Division (#1776)
| Berkeley, CA 94720-1776
==============================================================================
To add/remove yourself to/from the POSTGRES mailing list: send mail with
the subject line ADD or DEL to "postgres-request@postgres.Berkeley.EDU"
If this fails, send mail to "post_questions@postgres.Berkeley.EDU" and
a human will deal with it. DO NOT post to the "postgres" mailing list.
==============================================================================
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Failure of append
@ 1994-09-26 21:09 Raj Vadakkencherry <raj@CS.UCLA.EDU>
1994-09-27 04:46 ` Re: Failure of append Paul M. Aoki <aoki@cs.berkeley.edu>
0 siblings, 1 reply; 4+ messages in thread
From: Raj Vadakkencherry @ 1994-09-26 21:09 UTC (permalink / raw)
To: legacy
> X-Mts: smtp
>
> Paul Brown <pbrown@postgres.Berkeley.EDU> writes:
> > I sent this guy a note with the following observation;
> > 8192 - ( <<n ) / ( 2 * sizeof(double)) = number of points per page.
> > At about 500 points - given that a point is two doubles - you
> > hit page size limits. He says he's got 1000 point polygons.
>
> that's a good point, though he won't have tripped over that problem
> yet.
>
> george hartzell once did a recursive grep for 8192, changing every
> instance of 8192 to something much larger, and that somehow worked.
>
> the better solution is large objects, which do not have to be loaded
> through a query interface and do not have to be stored on one page.
> --
> Paul M. Aoki | University of California at Berkeley
> aoki@CS.Berkeley.EDU | Dept. of EECS, Computer Science Division (#1776)
> | Berkeley, CA 94720-1776
>
Sorry for mis-stating my problem - I meant 1000 doubles or 500
x,y pairs. In other words all my polygons have <= 500 points,
so I guess the 8K limit should not apply.
I'm reading polygon data from the regional data set (S2K benchmark) and writing
it to another postgres class. The failure of append occurs for some
really long strings. The error message is "unterminated char/string
constant". Here are some examples, if you would excuse the voluminous
output :-)
Here's an append that works (append string length 216)
append foo "(landuse="11"::int4,location="(-1.95e+06,-1.55e+05,-1.55e+05,-1.95e+06,-1.95e+06,-1.55e+05,-1.55e+05,-1.95e+06,-1.95e+06,-1.55e+05,-1.55e+05,-1.95e+06,-1.95e+06,-1.55e+05,-1.55e+05,-1.95e+06)"::polygon)
Here's an append that dosen't work: (string length 8580)
Error msg: WARN:Sep 26 20:23:39:*** scanner error: Unterminated char/string constant
append foo (landuse="11"::int4,location="(-1.95e+06,-4.68e+05,-4.68e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.59e+05,-4.59e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+06,-4.6e+05,-4.6e+05,-1.95e+06,-1.95e+!
06,-4.6e+05,-4.6e+05,-1.95e+06,-1
Since there are fewer than 500 points I think I will not hit the 8K limit, in either case. What else is the problem ?
Thanks,
Raj
==============================================================================
To add/remove yourself to/from the POSTGRES mailing list: send mail with
the subject line ADD or DEL to "postgres-request@postgres.Berkeley.EDU"
If this fails, send mail to "post_questions@postgres.Berkeley.EDU" and
a human will deal with it. DO NOT post to the "postgres" mailing list.
==============================================================================
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Failure of append
1994-09-26 21:09 Re: Failure of append Raj Vadakkencherry <raj@CS.UCLA.EDU>
@ 1994-09-27 04:46 ` Paul M. Aoki <aoki@cs.berkeley.edu>
0 siblings, 0 replies; 4+ messages in thread
From: Paul M. Aoki @ 1994-09-27 04:46 UTC (permalink / raw)
To: Raj Vadakkencherry <raj@CS.UCLA.EDU>; +Cc: legacy
raj@CS.UCLA.EDU (Raj Vadakkencherry) writes:
> > Paul Brown <pbrown@postgres.Berkeley.EDU> writes:
> > > I sent this guy a note with the following observation;
> > > 8192 - ( <<n ) / ( 2 * sizeof(double)) = number of points per page.
> > > At about 500 points - given that a point is two doubles - you
> > > hit page size limits. He says he's got 1000 point polygons.
> Sorry for mis-stating my problem - I meant 1000 doubles or 500
> x,y pairs. In other words all my polygons have <= 500 points,
> so I guess the 8K limit should not apply.
8KB == 8192B is the limit for the whole page, including the page
header/trailer, the line pointers on the page, tuple header(s), plus
any padding space. the exact amount of this overhead depends on the
type of computer you have since the size of various C structures will
be different on different architectures. 500 points == 16B * 500 ==
8000B which is reasonably close to the limit. if you are at or under
500, you are probably safe.
> I'm reading polygon data from the regional data set (S2K benchmark)
> and writing it to another postgres class. The failure of append
> occurs for some really long strings. The error message is
> "unterminated char/string constant"
like i said, i would guess that you hit the 8KB query buffer limit.
edit the files mentioned before, recompile/reinstall and see what
happens.
--
Paul M. Aoki | University of California at Berkeley
aoki@CS.Berkeley.EDU | Dept. of EECS, Computer Science Division (#1776)
| Berkeley, CA 94720-1776
==============================================================================
To add/remove yourself to/from the POSTGRES mailing list: send mail with
the subject line ADD or DEL to "postgres-request@postgres.Berkeley.EDU"
If this fails, send mail to "post_questions@postgres.Berkeley.EDU" and
a human will deal with it. DO NOT post to the "postgres" mailing list.
==============================================================================
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~1994-09-27 04:46 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
1994-09-26 17:52 Failure of append Raj Vadakkencherry <raj@CS.UCLA.EDU>
1994-09-26 19:39 ` Paul M. Aoki <aoki@cs.berkeley.edu>
1994-09-26 21:09 Re: Failure of append Raj Vadakkencherry <raj@CS.UCLA.EDU>
1994-09-27 04:46 ` Paul M. Aoki <aoki@cs.berkeley.edu>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox