website eating too much bandwidth

Portal Home > Knowledgebase > Articles Database > website eating too much bandwidth

Posted by eltonpiko, 01-28-2016, 01:23 AM
hi i have a website im hosting for a client i allocated 10gb bandwidth per month to this site but this month it exceeded the 10gb. i check the website and the registration form was filed with spam users so i deactivate registration on the site to prevent spam bot from filling the form. so i increased the bandwidth for that user by another 5gb so the website can be back online. but again just yesterday it used the 5gb i just allocated. seems that there is something doing this increase traffic on the site and causing this problem. please help. from my raw access log im getting ton of this 66.249.66.56 - - [27/Jan/2016:07:08:19 -0500] "GET /3042/joy29841-19/.do HTTP/1.1" 200 54554 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
Posted by Andei, 01-28-2016, 01:31 AM
Put the website under CloudFlare or a similar service, it should save you a lot of bandwidth on static content. You should also be able to get a better idea and understanding of what's causing most of the traffic from AWstats (if you have it enabled). It should tell you which IPs access the site the most and which resources are mostly accessed, and what's causing the most bandwidth usage.
Posted by ashutoshvct, 01-28-2016, 01:34 AM
Here the link which will help you: Read the complete page http://help.tibetbridges.com/faq/pro...our_bandwidth/
Posted by eltonpiko, 01-28-2016, 01:47 AM
from awstats im getting this see image the top one i my servers ip and i dont know why its using that much bandwidth in a day
Posted by eltonpiko, 01-28-2016, 01:55 AM
i also getting this
Posted by ashutoshvct, 01-28-2016, 02:04 AM
Have you tried the link i gave you??? It will guide you how to detect and block BOTS.
Posted by gnusys, 01-28-2016, 03:24 AM
You can use nginx as a web proxy and enable limit_req limit_conn to limit abuse of your web infrastructure.
Posted by anuja9991, 01-28-2016, 01:37 PM
Is the site built with WordPress ? I had encountered a similar kind of issue with one of the WP sites in past.
Posted by eltonpiko, 01-29-2016, 02:24 AM
hi its a Joomla website.
Posted by eltonpiko, 02-04-2016, 02:08 AM
ok so after doing some digging i found that most bandwidth is getting eaten by my own server ip why is that and how do i stop it. in 3 days the site consumed 2.2gb
Posted by anoopv, 02-04-2016, 06:23 AM
Hi, As per the attached screen shot Google Bots have consumed 2.08GB bandwidth and other bad bots have also consumed the bandwidth. You can use robots.txt file under the public_html directory of this sites to prevent bad bots from crawling your website and also can limit Google bots to crawl the site. Using .htaccess rules you can block bad bots. From your first message I can see that, you can see too many spam posts in registration form, to prevent this please use a captcha in your registration form. Using your server firewall or .htaccess file you can block those two IP's which consuming more bandwidth from accessing this site.
Posted by eltonpiko, 02-04-2016, 06:46 AM
Ok will try your suggestion.from my last awstat check i saw that my own server ip has used 520mb in one day any idea what causing this and how to prevent it? Last edited by eltonpiko; 02-04-2016 at 06:47 AM. Reason: Mistake
Posted by PowerUpHosting-Udit, 02-05-2016, 02:29 AM
This doesn't seem right. Maybe he is hosting some kind of media file and might be ranking on google for that file and users might be downloading those files. Did you try contacting your customer to ask him what's going on? Another reason why you should consider moving to CloudLinux OS.
Posted by eltonpiko, 02-05-2016, 02:42 AM
its really strange and its a nightmare in 5 days the bandwidth usage is already at 5.9gb considering ive set 10gb limit for this site for a month . just yesterday my own ip used 1gb.. i dont think i can deny my own ip from accessing the site. any idea please
Posted by eltonpiko, 02-05-2016, 03:51 AM
i came across this link http://www.allthingsdemocrat.com/hta...st-on-the-web/ and implemented their .htaccess file and instantly im getting tons off access deny in my error log
Posted by eltonpiko, 02-05-2016, 04:26 AM
one other problem i found is that on my search result on google its showing lots of random and foreign language. how to i correct this. 3周年記念で全商品ポイント2倍!!～9月30日19:59 。グローエ 2101900J アトリオ 2ハンドル洗面混合栓（引棒付き）クローム【GROHE】【02P23Sep15】
Posted by anoopv, 02-05-2016, 05:24 AM
Hi, Are you seeing this foreign languages while checking your site in Google. Please ask your hosting provider to run a scan on this account. Also please ask your website developer to check the database of this website. If the website looks clean please request Google to review your website files using Google webmaster tools.
Posted by roon2015, 02-05-2016, 05:43 AM
Some one might be trying to make your server down, might be a competitor, why you don't move to a host who give unlimtied bandwidth.
Posted by eltonpiko, 02-05-2016, 06:28 AM
The website is for a non profit. I own this vps server i limit this account bandwidth so it doesnt affect other site on the server
Posted by anoopv, 02-05-2016, 08:27 AM
Hi, As you updated you can see now too many access denied error message in your error log, it indicate that the connections from bad bots are now blocking the server. You can use the htaccess rules which mentioned in the URL http://www.javascriptkit.com/howto/htaccess13.shtml as well. Please check the bandwidth usage for the next couple of days and update this thread if you can't see any difference.
Posted by wiredhosting, 02-05-2016, 12:23 PM
Try this easy steps. 1) Put the website in cloudflare. will save you some banndwidth on static content and also will help you identify any security breach. 2) ask your hosting provider to scan your home directory with maldet 3) edit your .htaccess to have google review your website every 24hrs, not more..
Posted by netdepotmitch, 02-05-2016, 02:49 PM
That's a great link to save and send to people who come across the same problem.
Posted by eltonpiko, 02-06-2016, 12:40 AM
even if awstat is showing all does activities in my raw access log its filled with only this and i dont know what it is. but the part joy13534-19/.do if you put in google search will show results of a lot of website with the same foreign language as im getting im my search result. Is this an infection? 192.81.168.79 - - [05/Feb/2016:07:03:27 -0500] "GET / HTTP/1.1" 200 73944 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:27 -0500] "GET /3048/joy13534-19/.do HTTP/1.1" 200 65512 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:31 -0500] "GET /eco/ HTTP/1.1" 404 1152 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:29 -0500] "GET /eco/3042/joy26517-19/.do HTTP/1.1" 200 49550 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:32 -0500] "GET / HTTP/1.1" 200 73946 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:32 -0500] "GET /3048/joy14041-19/.do HTTP/1.1" 200 65031 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:35 -0500] "GET / HTTP/1.1" 200 73938 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:34 -0500] "GET /3048/joy1337-19/.do HTTP/1.1" 200 65777 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:38 -0500] "GET / HTTP/1.1" 200 73944 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:37 -0500] "GET /3048/joy33007-19/.do HTTP/1.1" 200 66383 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:40 -0500] "GET / HTTP/1.1" 200 73938 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:40 -0500] "GET /3048/joy29340-19/.do HTTP/1.1" 200 64173 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:43 -0500] "GET / HTTP/1.1" 200 73946 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:42 -0500] "GET /3041/joy43769-19/.do HTTP/1.1" 200 58618 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:45 -0500] "GET / HTTP/1.1" 200 73944 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:45 -0500] "GET /3048/joy15311-19/.do HTTP/1.1" 200 67838 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:48 -0500] "GET /adv/ HTTP/1.1" 404 1152 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:47 -0500] "GET /adv/3042/joy31549-19/.do HTTP/1.1" 200 47561 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:50 -0500] "GET / HTTP/1.1" 200 73938 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:50 -0500] "GET /3055/joy25618-19/.do HTTP/1.1" 200 53445 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:53 -0500] "GET / HTTP/1.1" 200 73945 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:52 -0500] "GET /3048/joy46037-19/.do HTTP/1.1" 200 70205 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:55 -0500] "GET / HTTP/1.1" 200 73938 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:55 -0500] "GET /3058/joy6935-19/.do HTTP/1.1" 200 46050 "-" "SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:58 -0500] "GET / HTTP/1.1" 200 73943 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:57 -0500] "GET /3048/joy45951-19/.do HTTP/1.1" 200 69964 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:04:00 -0500] "GET / HTTP/1.1" 200 73943 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:04:00 -0500] "GET /3048/joy30166-19/.do HTTP/1.1" 200 67225 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:04:02 -0500] "GET / HTTP/1.1" 200 73944 "-" "WHR"
Posted by anoopv, 02-06-2016, 01:28 AM
Hi, 66.249.69.27 this is Google IP, can you try to Google bots from crawl this website. Also do a scan on this website files also please ask the website developer to check whether there is any database level hack occurred for this site.
Posted by eltonpiko, 02-06-2016, 01:47 AM
hi i developed this website.from my analytic i got his on the 25th january Hacked content detected on (domianname) mydomain/3042/joy1015-19/.do dont know exactly where to start looking but the strange thing is that is coming from google ip.
Posted by anoopv, 02-06-2016, 01:51 AM
Hi, Is this files/directory still there. /3042/joy1015-19/.do is a hidden file/directory.
Posted by eltonpiko, 02-06-2016, 02:16 AM
from my root i cannot see any file or or directory with such name and from the look of my raw access log the number and the joy thing is always changing so it looks like this is getting generated by some script or something. i see the .do but dont know to do what looks like its calling on something. i can restore a backup before all this happen but im affraid it will happen again if i dont find the source. 192.81.168.79 - - [05/Feb/2016:07:03:27 -0500] "GET / HTTP/1.1" 200 73944 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:27 -0500] "GET /3048/joy13534-19/.do HTTP/1.1" 200 65512 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 192.81.168.79 - - [05/Feb/2016:07:03:31 -0500] "GET /eco/ HTTP/1.1" 404 1152 "-" "WHR" 66.249.69.27 - - [05/Feb/2016:07:03:29 -0500] "GET /eco/3042/joy26517-19/.do HTTP/1.1" 200 49550 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
Posted by eltonpiko, 02-06-2016, 09:48 AM
i came accross this site and its giving me a good idea where to check but still have not found the culprit. http://top.gabucis.com/categories/jo...-cloaking.html
Posted by alphavm_net, 02-06-2016, 10:44 AM
i think that that amount of data used for the website is too much for the normal usage, maybe try to check if there are some unauthorized access that uses your resources

Add to Favourites Print this Article

Knowledgebase

website eating too much bandwidth

Our Services

Client Menu

Legal