The software supportability and reliability site

Economic Impact of Software Loading at Suppliers

Ramón Somoza

A major misconception of systems and software engineers is that software loading "is no big deal". Well, after all, the software is just to be changed a couple of times over the life time of a product, so why bother implementing an expensive software loading system? Let the equipment supplier do it! Been there, done that. And boy, was I wrong when I stated such things back when I was still part of a software design team!

Well, I've learned a couple of things since that time. For example, that it may be ruinous to dismiss such a support decision with such a simple statement, specially if the equipment in question is a) expensive, b) long-living and c) there are lots of it. The very least one should consider is the life-cycle cost of such decision.

Let us perform a simple exercise around an aircraft, which is exactly the type of equipment that meets this kind of criteria. This is a real example, and I used it in the past to challenge a design decision that in my opinion would imply a significant (and expensive) support burden.

Statement of the problem

The parameters for this exercise are as follows:

We have an on-board equipment called the RDP. Each aircraft has four (4) of these aboard, and we expect to sell five hundred aircraft of this type during ten years of production. The equipment is not cheap (they rarely are), but it is not grossly expensive either - say 100,000$ apiece. It is not going to change very often either - we expect three (3) changes during the first five years after entry into service (you know, childhood problems), but afterwards only one change every 5 years. We expect that the three initial changes and half of the changes afterwards will be due to design problems, so we would be liable for those. The expenses of the three initial changes would be split 50/50 between the supplier and us.

The supportability engineers actually asked to make it loadable on-aircraft, but there is a slight problem: The modification will cost 4,000,000$, and will add an extra 3,000$ to the cost of each aircraft. That is, the total cost of implementing this apparently "nice to have" feature is 4,000,000 + 500 x 3,000 = 5,500,000$.

The RDP is pretty reliable, so we can do with around 5% of spares (which is not much, taking into account that we have many customers). On top of that, it is easy to install and uninstall - a mere 15 minutes for each operation. We assume that we will maintain this equipment through the whole aircraft life, which is usually 30 years, and that software updates will be performed until five years before the last aircraft is retired. The supplier has a 30-day turn-around time for a repair and/or software update, and we have another 10 days for transport in any direction.

We have a "good" deal with the supplier - he does not charge us anything for designing the software update, "but" he will charge us his standard repair rate for loading an RDP - a meager 5% of the equipment cost (so just 5,000$ for each software update - a pittance). It will cost us another 1,000$ to send the equipment to the supplier and get it back (but it includes insurance). Overall, it looks like a good bargain.

Is it? Let's have a look at the numbers. Possibly the number of equipment is irrelevant?

Affected equipment

a) During the initial entry into service (5 years) we have 3 software updates. In this period, we manufacture 50 x 5 = 250 aircraft. Assuming the software updates are evenly spread, it would affect half of them - just 125 aircraft, with 4 RDPs each, that's 500 equipment to worry about. If we add to this 5% of spares (which of course you need to keep with the latest configuration), that's an additional 25 equipment. Given that we have 3 software updates, that's 1,575 equipment to be loaded!

b) The complete in-service period takes 40 years (30 years after the last aircraft has been delivered from a production line that lasts only 10 years). Given that the initial entry into service has been already calculated above, and we are not going to provide software updates during the last 5 years, we have a mere 30 years of software updates. With one update every 5 years, that is 6 software updates to be performed to the whole fleet. That is: 6 x 500 x 4 = 12,000 RDPs to be loaded! If we add the spares on stock, that's another 600 loads. Total: 12,600 RDP software updates.

Well, 1,575 + 12,600 = 14,175 equipment loads can hardly be considered irrelevant!

Related direct maintenance costs

Keeping in mind that each time you send an item for uploading you have to disassemble it from the aircraft and assemble one from the stock, you have to perform (1,500+12,000)x2 = 27,000 operations (of course we do not disassemble what is in the warehouse - that is what we actually assemble on the aircraft!). As each operation takes 15 minutes, this means 6,750 hours of work on-aircraft. At, say, 60$/hour (dirt cheap!), this is 405,000$ simply in direct maintenance costs for the customers (and we don't even consider downtime cost, preparation time or admin costs!).

Direct loading cost

Given that the software load can be only performed at the supplier, we have to add the transport cost and what the supplier is going to charge us. That's 1,000$ + 5,000$ = 6,000$ per equipment and load. As we have a total of of 14,175 equipment loads, this means 14,175 x 6,000 = 85,050,000$.

Total software loading cost

If we add the pure maintenance cost to the direct loading cost, the REAL cost about this decision is 85,050,000 + 405,000 = 85,455,000$!!!

But forget a moment the customer. What does it cost us?

Our liabilities

We have stated we are liable for the three modifications during entry into service. This costs 1,575 x 6,000 = 9,450,000$. As this liability is shared 50/50 with the supplier, we will "only" pay half of it - 4,725,000$. However, we are also liable for half of the remaining updates, that is, for 6,300 uploads (calculate it!). That's another 6,300 x 6,000 =37,800,000$. Our bill is therefore 42,525,000$, and that assuming that the customers do not charge us for the extra maintenance time!

So, overall, these design "savings" of 5,500,000$ actually imply a support cost of 85,455,000$, of which WE would pay not less than 42,525,000$. That is, the savings are in reality a loss of 37,025,000$!

Conclusion

The customer(s) would be quite pissed off (who wouldn't, after losing 38,025,000$?), but the supplier would be delighted - after all, this design decision would give him an additional income of 66,150,000$, even after paying the penalties! (No wonder he is providing updates to the software design for free! Possibly he may even want to give us some "goodies" for free that imply re-loading software?)

I leave it to the reader to calculate the time that it takes to upgrade the whole fleet. Simply assume that you are going to keep only half of the spares on stock, and send the rest for "upgrading", sending the next batch when the first one returns. Assume that the supplier can process that much equipment at the same time. When the reader sees how long this takes, he can start worrying about fleet configuration management....

So careful with the design "savings" - ignoring support issues might be extremely expensive. And yes, I managed to convince my engineers... ;-)

(c) 2005 Ramón Somoza

P.D. I have received from some hardware-knowleable visitors to this site the comment that this type of approach is already classic in the ILS (integrated Logistic Support) world - it's called LORA (Level Of Repair Analysis). Yes, of course, I never pretended otherwise. But some of our visitors were surprised that a LORA could be actually applied to software, so I imagine that the article may have started some thinking...

Hosted by:
Exobits