Oh boy, what a mess!
Before I went to bed last night, there were nearly half a million tasks waiting to go out, now we have, err, nothing (really nothing)! I don't believe for a minute that they will keep the draconian download limit in place as they have said themselves that they have 'pretty well given up on 24/7 operation' - and they must surely know that they will loose a considerable amount of processing capacity if these measures continue.
I thought the download limit was a fairly clever way of managing the network flow - and maybe it was. I did have a look at the Cricket graph for network utilisation during the period immediately after the servers came up, and it was impressive; a few hours of heavy traffic then a nice tailing off (its since gained a couple of spikes ). What I actually find interesting (god I'm sad - obviously a throw back to my old job), is that historically the inbound network traffic is where the greatest amount of data flows - which is the opposite of what you would expect (since a result file is far smaller than a task file):
http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;view=Octets;ranges=w
not sure why......
Unfortunately, the server crash wasn't something they could figure into their plans, and this put paid to what I think they were going to do, which was to gently increase the limit. This way, the small contributors get a bash at grabbing their work, and the big crunchers slowly fill their tanks and perhaps by the time we got to the next outage, everybody would have been happy - ish. Also with this approach, the inbound network traffic 'should've' remained fairly steady (after the initial rush) as tasks from everyone's smaller caches were reported and they requested new work. I also noticed that in the run up to the last big planned outage, only 1 download server was running, so there was additional download capacity available - but perhaps not the network bandwidth to use it.
Anyway, I think their approach actually shows promise. I think this current period of instability will pass, but that we will see significant changes to the pattern of short-term work availability. We musn't loose sight of the fact that they are making untested changes to a live production system - which is always fraught with problems (when you cannot fully replicate the physical and logical infrastructure in a test environment and test changes before migrating them to production).
We must also remember that the project does not exist to serve us - its the other way around........
Yes we all like big numbers (around here anyway), but we have volunteered to process their data when it is available.
Keep the faith! (
)
