1 Topic by sam-the-man (edited by sam-the-man 2014-10-02 22:45:13)

  • sam-the-man
  • Member
  • Offline
  • Registered: 2011-07-24
  • Posts: 87

Topic: amavis, clamAV, MYSQL cluebringer, using HUGE ram/cpu/io on my server

==== Required information ====
- iRedMail version: 0.8.7
- Store mail accounts in which backend (LDAP/MySQL/PGSQL): LDAP
- Linux/BSD distribution name and version:  Deb 7
- Related log if you're reporting an issue: attached
====

How can I explain this? My server is very light load, but these services use HUGE HUGE RAM. See screenshot. Any help or advice? It's starving my system and my mail is not working properly.

My RAM is only 2gb. I have nearly identical configuration on other server and it only uses 300MB TOTAL. Why is this configuration so out of control when there is little to no traffic to this machine? Why are these services hogging all of my memory?

EDIT: Is my small server somehow sending and receiving lots of mail even though I don't know about it?!

http://i.imgur.com/fsk5bZ3.png

----

Spider Email Archiver: On-Premises, lightweight email archiving software developed by iRedMail team. Supports Amazon S3 compatible storage and custom branding.

2 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,788

Re: amavis, clamAV, MYSQL cluebringer, using HUGE ram/cpu/io on my server

sam-the-man wrote:

How can I explain this? My server is very light load, but these services use HUGE HUGE RAM. See screenshot.

What does your screenshots explain? Bash history? Nothing related to RAM usage.

You can try to reduce max servers preforked by Amavisd in its config file /etc/amavis/conf.d/50-user:

$max_servers = 4;

Then set the same number in /etc/postfix/master.cf for transport 'smtp-amavis':

smtp-amavis ... 4 smtp

Restart both Amavisd and Postfix services.

3 Reply by sam-the-man

  • sam-the-man
  • Member
  • Offline
  • Registered: 2011-07-24
  • Posts: 87

Re: amavis, clamAV, MYSQL cluebringer, using HUGE ram/cpu/io on my server

ZhangHuangbin wrote:
sam-the-man wrote:

How can I explain this? My server is very light load, but these services use HUGE HUGE RAM. See screenshot.

What does your screenshots explain? Bash history? Nothing related to RAM usage.

You can try to reduce max servers preforked by Amavisd in its config file /etc/amavis/conf.d/50-user:

$max_servers = 4;

Then set the same number in /etc/postfix/master.cf for transport 'smtp-amavis':

smtp-amavis ... 4 smtp

Restart both Amavisd and Postfix services.

Sir, that was RAM usage per process.

Now I'm getting these messages:

"Your Server has exceeded the notification threshold (90) for CPU Usage by averaging 124.8% for the last 2 hours."

Why iRedMail behaves like this on one install, but not another??

4 Reply by craig

  • craig
  • Member
  • Offline
  • From: Canada
  • Registered: 2013-09-04
  • Posts: 271

Re: amavis, clamAV, MYSQL cluebringer, using HUGE ram/cpu/io on my server

I'd be interested in what you learn. In my case, the server load is low and RAM usage is fine, but depite disabling certain functions my I/O is very high ... which is why I've disblaed some functions. If they were enabled my server would grind to a halt, and did several times before I disabled (among other things) Amavis.


Craig

5 Reply by sam-the-man

  • sam-the-man
  • Member
  • Offline
  • Registered: 2011-07-24
  • Posts: 87

Re: amavis, clamAV, MYSQL cluebringer, using HUGE ram/cpu/io on my server

craig wrote:

I'd be interested in what you learn. In my case, the server load is low and RAM usage is fine, but depite disabling certain functions my I/O is very high ... which is why I've disblaed some functions. If they were enabled my server would grind to a halt, and did several times before I disabled (among other things) Amavis.


Craig

Its so weird to me because I have an identical server, with the samw exact configuration, that has WAY more apache traffic, and only uses half the CPU, 1/3rd of RAM, and has never set off the IO/CPU warning notifications through my hosting provider.

The only difference between that server and the problematic one I'm inquiring about is that the problematic machine gets more email. But it receives what, 10 per day at the most? 20 sometimes? A 2gb ram server with this configuration should be able to send/receive thousands of messages without breaking a sweat. I have used 512mb RAM cPanel servers that ran all of the iRM services without any trouble. I also have iRM on a 512mb ram dev server that runs without any trouble at a fixes 370mb RAM. I'm using 2GB swap drives on each 2GB RAM machine.

What is wrong with this iRM instance? The config is almost completely oem. Why should I change amavis settings if this problem doesn't exist on other, identical, BUSIER machines? Where is this insane CPU/RAM usage coming from? What can I do to diagnose this Zhang? This is a production server and the resource starvation is causing some other systems to misbehave.

Thank you,
Sam

6 Reply by ZhangHuangbin

  • ZhangHuangbin
  • ZhangHuangbin
  • iRedMail Developers
  • Offline
  • Registered: 2009-05-06
  • Posts: 30,788

Re: amavis, clamAV, MYSQL cluebringer, using HUGE ram/cpu/io on my server

Did you try to upgrade packages with 'apt-get'? does it solve this issue?
Currently i have no idea why it happened without checking it myself and no enough evidence. sorry.

The solution is, try to figure out which component causes this issue, then dive into this component settings to find the root cause.

*) You can try to disable Amavisd integration in Postfix by commenting out 'content_filter =' line in /etc/postfix/main.cf, then restart/reload Postfix service. wait for some minutes to let Amavisd process existing injected emails. If CPU/RAM usage drops, it's caused by Amavisd + SpamAssassin + ClamAV.

*) Enable Amavisd integration in Postfix again, but disable SpamAssassin integration in Amavisd.
*) Keep Amavisd integration in Postfix, enable SpamAssassin integration in Amavisd again, but disable ClamAV integration in Amavisd.
...

Sometimes i cannot help without inspecting on your server, and it may take long time to help troubleshoot via forum. I ask for some config files or command output, wait for some hours, then you provide required evidence; wait for some more hours, and i ask some more, ...some hours..., you respond, ...

Hope you can understand this situation. Please try to figure it out yourself first.