Portal Home > Knowledgebase > Articles Database > Kill process that goes mad
Kill process that goes mad
Posted by BELLonline, 09-04-2008, 08:06 AM |
Hi,
I have a problem with one of my servers. What's happening is that a process on the server will suddenly go completely nuts, sometimes every day sometimes not for 2 weeks.
It is particularly annoying because it overloads the server so much that I'm unable to log in to SSH and see what it is causing the problem and I have to get the server rebooted just to get it back online (although it always pings).
I've got a couple of questions:
Is it possible to set SSH so that you can always log in? i.e set a very high priority.
Also, are there any scripts or anything out there that can run every couple of minutes or so and kill any process that uses up more than, say, 40% of the CPU for a period of time? I'm sure it could be possible to have a script that checks the top processes every so often but I've been unable to find anything.
Any help with that would be great
P.S. I run exacly the same software (Shoutcast/sc_serv) on lower spec servers with more customers and never have the problem. I've had the datacentre (UK2) check the hardware and they seem to think it's ok and just an sc_serv process causing the high load (although during normal operation, sc_serv uses no more than about 2% CPU).
|
Posted by slawek22, 09-04-2008, 08:16 AM |
You can run it as php script (eg. not from apache but from shell).
Use pipes to get >top command output, grep it every second or two, then you can >kill the problematic process and restart it, or >renice it so you can ssh to the server.
One difficulty may be that you may need to run the script as a root or copy the required files (kill, top, etc.) to some subdirectory and set sticky but so they'll run with root privledges.
or simply top > somefilename.txt , vmstat 1 > somefile2.txt to see what's the problem.
|
Posted by macooper, 09-04-2008, 08:17 AM |
These may help Process Resource Monitore which is a simple script that can monitor the resource usage of processes. Or alternativelySystem Integrity Monitor which is a little more comprehensive and allows you to monitor individual ports and services along with a server reboot at a defined load level if required. This is the better of the two, but also more complicated to configure.
|
Posted by BELLonline, 09-04-2008, 08:41 AM |
Thanks for your quick replies
I've installed PRM on the server - that's pretty much exactly what I was looking for. I just added an sc_serv config file to the rules folder and hopefully that should be all that's needed.
|
Add to Favourites Print this Article
Also Read