QoS in the cloudy world

[Caged]

Hopefully people have some experience with this, if not we can all lament the problems together

Let's say you have an office with 60 employees in. Your CRM system is a web app hosted in Amazon's cloud - but all file attachments and static content is stored in S3. You have a department that receives large artwork files from external collaborators, and they use one of the many file transfer services that is effectively a nice interface to an S3 bucket.

In this world where everything is HTTPS to an Amazon endpoint, how are you supposed to maintain quality of service to your line-of-business applications? Maybe you can rely on the DNS request being appropriate to each application, but with S3 this isn't always the case. Keeping track of which services are hosted at which IP address is a lot of work and will often involve manually figuring it all out since most services won't tell you because of the nature of managing public cloud services, and again doesn't really apply to S3.

Can newer firewalls work out that a particular flow is a download based on the volume of data being transferred in a certain time, and throttle it accordingly?

How are people dealing with this, or is it a "just buy a bigger pipe" type of scenario?

 

[Caged]

That make as much sense written out as it does in my head?

Maybe I'm more trying to discuss it as a problem that doesn't seem to have a real solution outside of buying direct connections to the services you want to use (lol). I think you're on the right path though.

We've had some Checkpoint firewalls in recently and they rate limit based on source/destination. It'd be nice if our web proxy could do this, but it only offers the option for YouTube, oddly..

I'm aware of rate limiting based on endpoints and also the application templates that firewalls can apply to things, but when everything is Amazon Web Services that's not hugely helpful. Perhaps the fix is going to be something along the lines of each AWS application having a unique tag on the traffic and the priority can be set accordingly?

 

[Caged]

I wasn't aware it was possible to do that unless you MITM the SSL(TLS) and push a new root CA key to your devices? Otherwise that could be a good place to start.

Dogers makes a good point though - I could just set a per-client limit for traffic related to S3 and the small attachments won't be affected because the throttle will never kick in before the file has loaded.

 

[Caged]

Certificate pinning is going to break that horribly though, so watch out for it if you're doing that sort of thing on your networks.