Amir has written a
blog post about "How to create very reliable web services" and I think he is definitely right with his tips. Every WebOps, DevOps or Admin should read this post to get an idea of whats possible and much better: Whants needed!
I would solve some of the tasks with
Splunk, but that is up to you!