Snippets from the reprap-admin list April 12, 2013 08:37AM |
Registered: 13 years ago Posts: 7,616 |
Quote
On 12 April 2013 14:47, bill2or3 wrote:
I did a round of server tuning, here are the changes I made:
- added robots.txt files to the wiki and forums, to tell msnbot, bingbot
and YandexBot not to request > 2 pages per minute. Those 3 bots combined
were accounting for *thirty percent* of the hits to the wiki, but
refer only a handful of visitors. This has made a huge difference.
- one-month 'Expires:' headers for the icon images on forums.reprap.org.
This will let those icons persist in client browser caches, across browser
restarts. Saves hits to the server, speeds up load times for return visitors.
- made the forum 'update viewcount' queries LOW_PRIORITY, so that actual
useful SELECT queries will happen first.
- set apache ssl session cache to the minimum size, 8k. It was set to
512k before, entirely unused. I think we can disable mod_ssl entirely,
which should free up a few megs.
- tiny fix to ReCaptcha.php, to avoid about 9 megs of errors messages per day
hitting /var/log/messages
The Munin cpu and network graphs are pretty flat since these changes.
Before, they were very inconsistent, occasionally spiking up by %400 or
just missing big chunks where the server was too busy to collect the graph
data.
I'm guessing that what was happening was that bingbot came to visit,
from several different IP's at once. All those bingbot crawlers were hitting
forums, making lots of mostly identical requests. All those queries eventually
hit the 'messages' table, causing the sql table to lock, and grinding
everything to a crawl.
The graphs show that apache hits/apache traffic/ethernet traffic all dropped
by about 1/3, and CPU seems to have a lot more breathing room.
If nobody has any objections, I'd like to let it run for a few days, just to
be sure everything is stable, and then enable mod_gzip and disable mod_ssl,
in Apache.
Depending how it looks after that, perhaps investigate using APC php caching,
and/or parsing php code via fastcgi instead of with mod_php.
Overall, I'm pretty happy with these results, and expect to quit hearing
"is it down?" asked in IRC.
cheers,
Bill
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
Re: Snippets from the reprap-admin list - part 2. April 22, 2013 10:21PM |
Admin Registered: 14 years ago Posts: 17 |
Re: Snippets from the reprap-admin list April 23, 2013 02:20AM |
Admin Registered: 16 years ago Posts: 13,889 |
Re: Snippets from the reprap-admin list April 23, 2013 05:20AM |
Registered: 13 years ago Posts: 7,616 |
Re: Snippets from the reprap-admin list April 23, 2013 11:46AM |
Admin Registered: 14 years ago Posts: 17 |
Re: Snippets from the reprap-admin list June 02, 2013 09:45AM |
Registered: 13 years ago Posts: 7,616 |
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
Re: Snippets from the reprap-admin list June 03, 2013 06:29PM |
Registered: 14 years ago Posts: 1,092 |
Re: Snippets from the reprap-admin list June 29, 2013 07:14AM |
Registered: 13 years ago Posts: 7,616 |
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
Re: Snippets from the reprap-admin list June 30, 2013 09:19PM |
Registered: 14 years ago Posts: 1,092 |
Re: Snippets from the reprap-admin list July 01, 2013 04:14AM |
Admin Registered: 16 years ago Posts: 13,889 |
Re: Snippets from the reprap-admin list August 24, 2013 09:00AM |
Registered: 13 years ago Posts: 7,616 |
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
spam-post permissions adjustments. September 05, 2013 01:06PM |
Admin Registered: 14 years ago Posts: 17 |
Re: Snippets from the reprap-admin list October 13, 2013 10:49PM |
Moderator Registered: 13 years ago Posts: 7,616 |
Quote
Traumflug
Blogging time :-)
Notified by a lot of error messages in /var/log/forum_error_log I found images embedded into forum posts didn't work the way they should. Forum admins tended to disable the embed_images module, which doesn't match with some other part of the configuration.
The problem was, /usr/bin/convert insists on additionally writing a file when told to send the output to stdout. That's clearly a bug. As it didn't have privileges to write (what a luck!), it failed.
To fix this, I canged /data/www/forums/htdocs/includes/api/images.php to write this file to /dev/null. This workaround might break when 'convert' removes its bug. Nevertheless, it's the first time I see properly thumbnailed image boxes in our forum.
Time needed to find the right place and the right type of this one-line fix: about 6 hours. :-)
As always, I made a copy of images.php before patching. A person doing a wiki or forum upgrade may please watch out for files with a date appended (or "2013" in it's name), they're originals of patched files.
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
Wiki user cleanup June 22, 2014 01:01PM |
Registered: 13 years ago Posts: 7,616 |
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
Re: Snippets from^H^H^H^H for the reprap-admin list July 11, 2014 02:46PM |
Registered: 13 years ago Posts: 7,616 |
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
Re: Snippets from the reprap-admin list July 18, 2014 06:21AM |
Registered: 13 years ago Posts: 7,616 |
Quote
bill2or3
I've edited the forum file at /data/www/forums/htdocs/include/phorum_get_url.php to set it so the javascript.php requests don't have the forum id added.
before:
GET /javascript.php?225
after:
GET /javascript.php
The javascript doesn't vary by forum, so having that forum-id on the request was causing the file to be re-requested when a user switched between forums, rather than just using the javascript file it'd already downloaded.
[...]
A spot-check of yesterdays access_log shows that it transfered 3121 megs of javascript.php, but it'd be 1387 megs if there were no duplicate javascript.php traffic.
Generation 7 Electronics | Teacup Firmware | RepRap DIY |
Re: Snippets from the reprap-admin list July 20, 2014 09:49PM |
Registered: 14 years ago Posts: 1,092 |
Re: Snippets from the reprap-admin list July 21, 2014 06:07AM |
Registered: 13 years ago Posts: 7,616 |
Generation 7 Electronics | Teacup Firmware | RepRap DIY |