Snippets from the reprap-admin list

Quote

On 12 April 2013 14:47, bill2or3 wrote:


I did a round of server tuning, here are the changes I made:

- added robots.txt files to the wiki and forums, to tell msnbot, bingbot
and YandexBot not to request > 2 pages per minute. Those 3 bots combined
were accounting for *thirty percent* of the hits to the wiki, but
refer only a handful of visitors. This has made a huge difference.

- one-month 'Expires:' headers for the icon images on forums.reprap.org.
This will let those icons persist in client browser caches, across browser
restarts. Saves hits to the server, speeds up load times for return visitors.

- made the forum 'update viewcount' queries LOW_PRIORITY, so that actual
useful SELECT queries will happen first.

- set apache ssl session cache to the minimum size, 8k. It was set to
512k before, entirely unused. I think we can disable mod_ssl entirely,
which should free up a few megs.

- tiny fix to ReCaptcha.php, to avoid about 9 megs of errors messages per day
hitting /var/log/messages

The Munin cpu and network graphs are pretty flat since these changes.
Before, they were very inconsistent, occasionally spiking up by %400 or
just missing big chunks where the server was too busy to collect the graph
data.

I'm guessing that what was happening was that bingbot came to visit,
from several different IP's at once. All those bingbot crawlers were hitting
forums, making lots of mostly identical requests. All those queries eventually
hit the 'messages' table, causing the sql table to lock, and grinding
everything to a crawl.

The graphs show that apache hits/apache traffic/ethernet traffic all dropped
by about 1/3, and CPU seems to have a lot more breathing room.

If nobody has any objections, I'd like to let it run for a few days, just to
be sure everything is stable, and then enable mod_gzip and disable mod_ssl,
in Apache.

Depending how it looks after that, perhaps investigate using APC php caching,
and/or parsing php code via fastcgi instead of with mod_php.

Overall, I'm pretty happy with these results, and expect to quit hearing
"is it down?" asked in IRC.

cheers,
Bill

Quote
Traumflug
Blogging time :-)

Notified by a lot of error messages in /var/log/forum_error_log I found images embedded into forum posts didn't work the way they should. Forum admins tended to disable the embed_images module, which doesn't match with some other part of the configuration.

The problem was, /usr/bin/convert insists on additionally writing a file when told to send the output to stdout. That's clearly a bug. As it didn't have privileges to write (what a luck!), it failed.

To fix this, I canged /data/www/forums/htdocs/includes/api/images.php to write this file to /dev/null. This workaround might break when 'convert' removes its bug. Nevertheless, it's the first time I see properly thumbnailed image boxes in our forum.

Time needed to find the right place and the right type of this one-line fix: about 6 hours. :-)

As always, I made a copy of images.php before patching. A person doing a wiki or forum upgrade may please watch out for files with a date appended (or "2013" in it's name), they're originals of patched files.

Quote
bill2or3
I've edited the forum file at /data/www/forums/htdocs/include/phorum_get_url.php to set it so the javascript.php requests don't have the forum id added.

before:
GET /javascript.php?225
after:
GET /javascript.php

The javascript doesn't vary by forum, so having that forum-id on the request was causing the file to be re-requested when a user switched between forums, rather than just using the javascript file it'd already downloaded.
[...]
A spot-check of yesterdays access_log shows that it transfered 3121 megs of javascript.php, but it'd be 1387 megs if there were no duplicate javascript.php traffic.