Portal Home > Knowledgebase > Articles Database > Server setup
Server setup
| Posted by thehostx, 02-19-2014, 04:43 AM |
| Hi, I am in the process of launching a site that lets users upload videos and host files.
The uploading of videos will take a lot of cpu power and was wondering what kind of setup I can use to keep everything up and running regarding cpu, memory, and storage space. I recently had a problem where the hard disks on my server filled up and the server would no longer boot.
So I was wondering what we can do to setup something that allows cpu and memory to be shared within the 3 servers. I am also going to be using the servers hard drives rather than having an storage device.
Any thoughts on this?
Thanks!
|
| Posted by david510, 02-19-2014, 06:20 AM |
| If you have database driven sites, you can setup remote mysql server and access the database remotely. We can use mysql for this configuration.
|
| Posted by CDNsun, 02-19-2014, 10:18 AM |
| Hello,
filled disk is always bad. You can try using disk quota to avoid this problem.
As for CPU, mem and disk: in general I would not combine SQL with storage-server. For SQL you need fast CPU, a lot of mem and a little of disk space. For storage server you need normal CPU with normal memory (for caching) and a lot of disk space.
But if combined together I would at least ensure that the CPU will have a lot of cores because typically disk operations takes the whole one core for itself.
Regards
|
| Posted by fabin, 02-19-2014, 12:10 PM |
| Have you thought of moving to a cloud services platform like Amazon?
|
| Posted by thehostx, 02-19-2014, 03:14 PM |
| Thanks for the replies.
I looked into openstack to create my own cloud. Even companies like amazon they must be able to pool all their servers together to achieve something like this no?
|
| Posted by CDNsun, 02-19-2014, 03:41 PM |
| There is a general problem-solving rule: divide et impera (divide and rule) so I would not aim putting them together but rather divide them cleverly.
I'm betting my right hand that Amazon is able to help with this
|
| Posted by Mr Terrence, 02-19-2014, 03:43 PM |
| You can also look into VPDC that way all the resources are dedicated to you, of course this all depends on how much you are willing to spend.
|
| Posted by simon_enzu, 02-19-2014, 03:58 PM |
| Cloud based hosting would be the best solution for you. There are lot of providers with cloudstack/openstack. Main advantage is their scalability. Depending on your requirement, you can increase the usage.
|
| Posted by fabin, 02-19-2014, 04:10 PM |
| Setting up your own cloud might be an overkill. Try cloud hosting providers who can scale resources automatically like amazon.
|
| Posted by Server Management, 02-19-2014, 08:20 PM |
| Like a file sharing service?
Surely you have the ability to add multiple storage nodes within the application your using?
|
| Posted by thehostx, 02-19-2014, 08:33 PM |
| It's a mix of youtube and file storage service.
Basically the video part is going to take up a lot of cpu power to do conversions etc... and both the video and file hosting will take up a lot of storage.
I looked into openstack and they allow data to be stored on multiple servers but was in need of something that will pool the resources of all the servers such as cpu and memory as well.
|
| Posted by sarathpv, 02-19-2014, 08:44 PM |
| The conversion can take up a lot of cpu power but only if there are multiple conversions simultaneously. In fact you can schedule the conversions and reduce the cpu usage. You would require a large storage size which gonna cost you high.
|
| Posted by eth00, 02-21-2014, 12:13 PM |
| So you want to pool the resources of 3 servers but then you also want them to be separate? It sounds like you just need to get a dedicated server and create 3 VMs with it. If you do that just remember you are putting all of your eggs in one basket.
To prevent the disk issues you could always create a partition (or use physical disks/arrays) for each VM. That would provide a hard limit.
|
| Posted by thehostx, 02-22-2014, 12:08 AM |
| Hey, I want to pool the resources of 3 servers which will be running 1 site. Nothing will really be separate here.
|
| Posted by SchnellNode, 03-06-2014, 09:40 AM |
| What are you looking for is called HA clustering with load balancing. Load balancing assures that each web client will use another web server in round robin fashion. The database is shared amongst the servers and all the servers run the same queries in sync. If you want to share the HDD over the netwrok is also possible but bare in mind that you will need 10GB network connections between the server in order to preserve the SATA III speed which is 6GB.
|
| Posted by thehostx, 03-06-2014, 08:32 PM |
| Hi, thanks for the reply. I am using a switch that I have all the server connect to so that should not be a problem.
So with this sort of setup if one server is using too much cpu and memory another server will kick in to help process that information?
|
| Posted by SchnellNode, 03-07-2014, 01:41 AM |
| This means that all the resources of the 3 servers are being used almost equally. There's no 'kick-in' process. The balancing handles the web visitors connections. Every new web visitor is sent to the a different webserver or to the webserver that is handeling the less visitors a the moment. This way you will never have one server that will be using ALL resources and another one that will just wait.
Make sure your switch is 10GB - NOT 1GB !!! - you will need a switch with at least 3 10GB ports - they are very expensive. For the start you could do the same with a 24 1GB port switch and 2 x 1GB 4port NICs for each server. I think this would be cheaper. If you need more details please contact me on PM.
|
| Posted by eth00, 03-09-2014, 03:00 PM |
| Before doing something like this you should really sit down and figure out exactly how you are doing to do everything.
Likely you will want to dedicated servers to particular tasks, database, storage, processing, and webserver.
You may consider using virtualization to ease new server deployments and help keep things separated for security and load. You can create multiple VMs on each server and then use that to help partition the resources on the server. If you need more or less on different services you can adjust the processor and RAM on them.
Unless you are going to be doing a LOT of video processing and storage or file replication on your storage you should not need the 10Gbps for awhile. That equipment can be expensive and likely is not important initially.
|
| Posted by thehostx, 03-10-2014, 10:54 PM |
| Thanks for the replies
I am already doing a lot of video processing and storage so was looking to expand into the cluster setup.
So if I do this setup what will determine which server will hold the database? How about if I want the database to be throughout all servers? Also what about storage I recently had a problem when my drives got full and my server crashed how can we prevent this?
The switch I currently have is a Cisco Catalyst 4948-10GE
|
Add to Favourites 
Print this Article
Also Read
Error 403 (Views: 744)
Shared Hosting
Shared Hosting
