TSB Upgrade issues

Caporegime
Joined
29 Jan 2008
Posts
58,899
You would hope, but it is not in a lot of places I've seen sadly.

Oh you mean someone having to manually change it? I don't think that is applicable here, you're talking about a huge amount of data - it has to be an automated process.
 
Commissario
Joined
23 Nov 2004
Posts
41,851
Location
Herts
Oh you mean someone having to manually change it? I don't think that is applicable here, you're talking about a huge amount of data - it has to be an automated process.
Yeh manually using a spreadsheet/macro/whatever, but also people not knowing how to use the automated tools properly either!
 
Man of Honour
Joined
17 Nov 2003
Posts
36,743
Location
Southampton, UK
Waterfall planning is tried, tested and works very well when properly implemented, and in many large projects it is the only system that can have a high chance of success and meet all the stakeholders requirements.

Waterfall works well when you have work that is easily known, understood and done before. If you are doing anything new, it falls apart. It also very good at meeting contractual obligations that aren't actually what brings the most (or sometimes any) business value.

Only in some environments does it make sense to use customers as beta testers and let them find the bugs, exposing them to huge security or safety critical issues.

The only real way of writing safety critical software bug free is with formal methods, and that's incredibly hard at best. Good testing is not limited to only waterfall projects. Agile has requirements for this to be fast, and we like to automate as much as possible but ultimately this has less to do with the project management methodology.

Rather than fixing bugs quickly, it is much better simply not to let the bugs get through to production in the first place.

Waterfall is very good at giving you a false sense of security on this. Bugs will always get through and being able to find and fix quickly is by far the better approach. Agile methods don't preclude good testing, but it should be fast to get quick feedback.

Otherwise planes fall out of the sky and power stations blow up.

Again, a lot of this is a false sense of security. Take aircraft that are easily hackable due to the waterfall and certification methods which mean that vulnerable security software can't be updated without recertification. Google Chris Roberts or click link (https://edition.cnn.com/2015/05/17/us/fbi-hacker-flight-computer-systems/index.html)
 
Caporegime
Joined
18 Oct 2002
Posts
32,615
Waterfall works well when you have work that is easily known, understood and done before. If you are doing anything new, it falls apart. It also very good at meeting contractual obligations that aren't actually what brings the most (or sometimes any) business value.
This is the exact opposite of the situation. Agile works very well for small simple projects where the path to the goal is obvious and rapid iteration without planning will most likely lead to sufficient solutions, and failure is of little concern. For larger and more complex projects it fails, horrifically in some cases. Business value require meeting contractual obligations.

The only real way of writing safety critical software bug free is with formal methods, and that's incredibly hard at best. Good testing is not limited to only waterfall projects. Agile has requirements for this to be fast, and we like to automate as much as possible but ultimately this has less to do with the project management methodology.
Agile will tend to use clients as beta testers and things fail in production with the hope that it can be fixed quickly. Good quality software will simply fail less often.


Waterfall is very good at giving you a false sense of security on this. Bugs will always get through and being able to find and fix quickly is by far the better approach. Agile methods don't preclude good testing, but it should be fast to get quick feedback.
Waterfall, has lots of measures in place that reduce the chances of failure. Agile just doesn't both, and will be expected to fail in production. Being able to fix bugs quickly is not limited to Agile, that is simply a merit. As you said, agile looks at fast testing, thereby you admit it is more limited in scope.

Again, a lot of this is a false sense of security. Take aircraft that are easily hackable due to the waterfall and certification methods which mean that vulnerable security software can't be updated without recertification. Google Chris Roberts or click link (https://edition.cnn.com/2015/05/17/us/fbi-hacker-flight-computer-systems/index.html)

An anecdote is of little relevance. Meanwhile, Agile projects are failing repeatedly, time and time again, at great cost:
http://www.information-age.com/uk-wasting-37-billion-year-failed-agile-it-projects-123466089/
'British business is set to waste an estimated £37 billion on failed agile IT projects over the course of the next 12 months"


https://www.computerweekly.com/news...opment-an-IT-fad-that-risks-iterative-failure

Over half of CIOs have discredited agile development, according to a survey commissioned by London-based technology services firm, 6Point6.

Three-quarters of the 300 UK and US-based CIOs surveyed said they were no longer prepared to defend it. Half said they now think of agile as “an IT fad”.


https://michaelochurch.wordpress.com/2015/06/06/why-agile-and-especially-scrum-are-terrible/
 
Soldato
Joined
28 Oct 2006
Posts
12,456
Location
Sufferlandria
This is the exact opposite of the situation. Agile works very well for small simple projects where the path to the goal is obvious and rapid iteration without planning will most likely lead to sufficient solutions, and failure is of little concern. For larger and more complex projects it fails, horrifically in some cases. Business value require meeting contractual obligations.

I don't think Agile is the real problem. It's the misconception that anything which isn't a waterfall methodology is agile. People hacking stuff together with no real plan, making it up as they go and claim they are following an agile approach.
 
Caporegime
Joined
18 Oct 2002
Posts
32,615
I don't think Agile is the real problem. It's the misconception that anything which isn't a waterfall methodology is agile. People hacking stuff together with no real plan, making it up as they go and claim they are following an agile approach.


There is also the misconception that whatever is not agile is "waterfall', w and is reminiscent of the 1960s software process. traditional software development processes matured, and are quite diverse. nothing in the Agile manifesto was knew, it was just re-spun into a fad
 
Man of Honour
Joined
17 Nov 2003
Posts
36,743
Location
Southampton, UK
Agile will tend to use clients as beta testers and things fail in production with the hope that it can be fixed quickly. Good quality software will simply fail less often.
Good quality software will fail less often. Waterfall doesn't help give better quality software though.

This is the exact opposite of the situation. Agile works very well for small simple projects where the path to the goal is obvious and rapid iteration without planning will most likely lead to sufficient solutions, and failure is of little concern.

Agile working is all about doing small parts of a larger problem to help derisk it. It does not mean little or no planning.

For larger and more complex projects it fails, horrifically in some cases.
Large and complex projects often fail due to the fact they are large and complex, not because of the project management methodology. Software engineering is more like landscape gardening than civil engineering which so much of the methodologies are taken from.

Business value require meeting contractual obligations.
Yes, but the slow nature of legalities mean that these are often obsolete requirements as soon as they're agreed.

Business value require meeting contractual obligations.]An anecdote is of little relevance. Meanwhile, Agile projects are failing repeatedly, time and time again, at great cost:
http://www.information-age.com/uk-wasting-37-billion-year-failed-agile-it-projects-123466089/

It's interesting that it raises 6 points:
Distributed teams - irrelevant to the methodology, this is a problem in both waterfall and agile.
Scaling Agile - this is often a problem due to software architecture that many large organisations use. Most existing systems are monoliths and so scaling any team to be able to maintain that codebase is difficult. Again, this is less about project management methodology and more about loosely coupled, tightly bounded systems that make changes less risky as they can be understood in their entirety.
Documentation - I would comment on this, but I downloaded the report that that link you provided is based on. It doesn't explain why they thought lack of documentation was a causal issue to projects failing or even if lack of documentation in agile projects was a problem. In fact it doesn't really talk about it except in the conclusions. Agile does not mean no documentation nor to avoid updating docs. It just says working software is more important than having comprehensive docs.

The only thing that report actually usefully draws attention to is the lack of architecture that is used in system design. This is a problem in many organisations regardless or how they plan their projects and is key to getting the loosely coupled, tightly bounded systems that enable so much good practice.

I don't think Agile is the real problem. It's the misconception that anything which isn't a waterfall methodology is agile. People hacking stuff together with no real plan, making it up as they go and claim they are following an agile approach.

I think this is one of the key take aways. Agile is a value preference, not a methodology in itself. Many use SCRUM and agile interchangeably - incorrectly. I bet out of those companies that were part of the 6point6 report many think that splitting up a Gantt chart into 2 weekly sprints is 'Agile' and wonder why it's not working.
 
Soldato
Joined
31 May 2009
Posts
21,257
Surely thee comes a point they roll back, and add the last weeks transactions manually from their backups?

- such as over a bank holiday weekend where they have 72 hours of go time.
 
Soldato
Joined
17 Jul 2005
Posts
3,191
Surely thee comes a point they roll back, and add the last weeks transactions manually from their backups?

- such as over a bank holiday weekend where they have 72 hours of go time.

There comes a point in some upgrades where a rollback isn't feasible any more ... more so if it's significant re-platforming. Sounds they are in in a state where they just have to keep moving forwards. Am sure the execs asked at a very early stage for them to roll back and got the above answer.

It's unforgivable at this level however you've got to feel for the project team. The moment when things started to go south will have been particularly galling....
 
Man of Honour
Joined
13 Oct 2006
Posts
90,820
Looks like this isn’t going to be fixed anytime soon. Could be months according to some reports this morning.

Bit surprised at that given they supposedly called in IBM who have a pretty decent "hit team" who can pretty much implement a complete solution for this almost on the spot (even with all the data recovery/verification issues that will be needed here) - maybe they don't want to pay what that costs though.
 
Caporegime
Joined
29 Jan 2008
Posts
58,899
Bit surprised at that given they supposedly called in IBM who have a pretty decent "hit team" who can pretty much implement a complete solution for this almost on the spot (even with all the data recovery/verification issues that will be needed here) - maybe they don't want to pay what that costs though.

wat???

implement a complete solution on the spot... doubtful

having worked with IBM on a project (for IBM) the technical people I was exposed to left a very different impression. At least I got to stay in London while working on it, some people ended up in some bum**** town in New York State where 99% of the population are IBM employees.
 
Man of Honour
Joined
13 Oct 2006
Posts
90,820
implement a complete solution on the spot... doubtful

You are taking me too literal - hence the "almost" - obviously they aren't going to turn up, implement and be gone in an hour. Dunno my experience has been different to what you are talking about - though we are talking about a lot of money and Z series implementation maybe its different at smaller scale but they seemed capable of rapidly scaling a solution up at almost any level.

EDIT: Obviously I don't really have any experience behind the scenes, etc. and I wasn't directly involved but they seemed pretty effective at recovering from what was pretty much a disaster albeit it wasn't cheap.
 
Caporegime
Joined
29 Jan 2008
Posts
58,899
i really really hope more people switch accounts. There needs to be serious consequences for this sort of thing for senior management like their jobs disappearing/company failing not just losing one annual bonus and carrying on.
 
Back
Top Bottom