[wirelesstoronto-discuss] tech help needed!

[Rob Janes]

the indexes should be setup already.  i don't think there's any tuning
needed there.  gabe will have followed the isf install script.

i'm thinking one of the nodes is sending too much heartbeat.  that's why i
suggested looking in the logs.

you could check the postgres logs.  postgres gets real busy when it flushes
it's logs, and it can bog down the machine.  it shouldn't be doing that very
often, it's a bursty thing.  gabe is saying the meter is pegged at 4 or so.
if postgres was busying up the system on it's own, due to a misconfiguration
or misbalancing of the database files and the log files, it would show low
load most of the time, then flipping over to some high number.

postgres can be set to log all sql that comes in.  in a more verbose mode it
could indicate what the problem is.

On Wed, Apr 16, 2008 at 4:38 PM, Jon Pounder <Jon at pounder.com> wrote:

> Quoting Rob Janes <janes.rob at gmail.com>:
>
>  did you do vacuumdb -full, or VACUUM FULL;  that frees up empty deleted
> > records.  it won't speed up queries, which may be the problem.  VACUUM
> > FULL
> > and vacuumdb -full lock tables, so there will be an outage when you do
> > this.  best to do it early in the morning.
> >
>
> if there are tons dead rows it does speed things up significantly since
> the live rows fit into far less physical pages on disk so net result you
> reduced the disk reads needed to do a given amount of work.
>
> make sure you have it on automatic though and then you don't have to
> remember to do it.
>
> apache could also be showing busy depending how efficient the waits are on
> postgres in the particular postgres client driver you are using. apache and
> pg could also be competing for disk read bandwidth and both waiting on that
> when pg is doing something intense.
>
> I kind of got into this group late so not sure what the auth server is
> actually using, but if its not a tried and tested system, its possible it
> needs some indexes to fix access speed on tables that started out small and
> now are huge.
>
>
>
>
>
> > however, that apache is busy too suggests something akin to a dos.
> >  check
> > the apache logs.
> >
> > -rob
> >
> > On Wed, Apr 16, 2008 at 3:59 PM, Gabe Sawhney <gabe at pwd.ca> wrote:
> >
> >  Top seems to indicate that the problem is with apache and postgres, but
> > > I
> > > don't know what the numbers mean.  I took Jon's suggestion and ran a
> > > VACUUM
> > > ANALYZE again (I thought I'd done several times before), and it cooled
> > > it
> > > down a bunch... now the load avg is between 4 and 5.  That still
> > > strikes me
> > > as high, but it's a lot better.  I'll keep an eye on it in the next
> > > few
> > > days.
> > >
> > > The machine had 638 days uptime until I rebooted it today, trying to
> > > troubleshoot this.  (The reboot didn't help.)
> > >
> > > Gabe
> > >
> > >
> > >
> > > On Wed, Apr 16, 2008 at 3:39 PM, Rob Janes <janes.rob at gmail.com>
> > > wrote:
> > >
> > > > what does top say?
> > > >
> > > > do you want me to log on to the box?
> > > >
> > > > -rob
> > > >
> > > >
> > > > On Wed, Apr 16, 2008 at 2:50 PM, Jon Pounder <Jon at pounder.com>
> > > wrote:
> > > >
> > > > > Quoting Gabe Sawhney <gabe at pwd.ca>:
> > > > >
> > > > >  We need some help from someone with Linux performance tuning
> > > > > > experience.
> > > > > > Our auth server in the past few months has been getting really
> > > > > > slow... we
> > > > > > haven't been asking it to do any more work, but while before the
> > > > > > load
> > > > > > average sat around 1.1, now it's almost always between 6 and
> > > 12(!).
> > > > > >
> > > > > > Basically we're talking about Apache, PHP5 and Postgresql.
> > >  There's
> > > > > > also
> > > > > > MySQL and Mailman running on the box, but I doubt those have
> > > > > > anything to do
> > > > > > with it.
> > > > > >
> > > > > > If you've done stuff like this before, please get in touch!
> > > > > >
> > > > >
> > > > >
> > > > > make sure postgres is configured to autovacuum tables.
> > > > >
> > > > > if you don't have it, install pgadmin on your own desktop
> > > (windows)
> > > > > and connect to the pg db as the postgres user, then you can fine
> > > tune
> > > > > whatever you need on there.
> > > > >
> > > > > also look for any log files that have just gotten massive, and
> > > clean
> > > > > them out.
> > > > >
> > > > > if its not been rebooted in a while (like months), also do that
> > > which
> > > > > will force a check of the drives in case anything is haywire
> > > there.
> > > > >
> > > > >
> > > > > also - btw, the listserver for this mailing list is broken,
> > > messages
> > > > > are addressed to the list rather than the person when they arrive
> > > so
> > > > > replying if you have multiple email accounts is broken.
> > > > >
> > > > >
> > > > >
> > > > > > Gabe
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Jon Pounder
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > wirelesstoronto-discuss mailing list
> > > > > wirelesstoronto-discuss at wirelesstoronto.ca
> > > > >
> > > > >
> > > http://lists.wirelesstoronto.ca/cgi-bin/mailman/listinfo/wirelesstoronto-discuss
> > > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > wirelesstoronto-discuss mailing list
> > > > wirelesstoronto-discuss at wirelesstoronto.ca
> > > >
> > > >
> > > http://lists.wirelesstoronto.ca/cgi-bin/mailman/listinfo/wirelesstoronto-discuss
> > > >
> > > >
> > >
> > > _______________________________________________
> > > wirelesstoronto-discuss mailing list
> > > wirelesstoronto-discuss at wirelesstoronto.ca
> > >
> > >
> > > http://lists.wirelesstoronto.ca/cgi-bin/mailman/listinfo/wirelesstoronto-discuss
> > >
> > >
> > >
> >
>
>
>
>
> Jon Pounder
>
>
> _______________________________________________
> wirelesstoronto-discuss mailing list
> wirelesstoronto-discuss at wirelesstoronto.ca
>
> http://lists.wirelesstoronto.ca/cgi-bin/mailman/listinfo/wirelesstoronto-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.wirelesstoronto.ca/pipermail/wirelesstoronto-discuss/attachments/20080416/6a4e6593/attachment-0001.htm