Engineering Costing Techniques in ESA

D. Greves &: B. Schreiber

Cost Analysis Division, ESTEC, Noordwijk, The Netherlands

It has always been important for organisations to be able to estimate and control their costs properly, but it is particularly so today. In the commercial domain there has been a marked tendency towards increased competition with international trade restrictions being reduced or removed and the question of competitiveness having to be considered on a global rather than simply on a local, national or regional scale. In the public domain also, consideration of cost is also assuming much greater importance with the tightening of budgets and changing budget priorities, together with the general expectation that there should be better value for money from public expenditure.

The importance of the cost-estimating function

In a competitive situation, if a company's estimate of its costs is unrealistically low then it may obtain an order but risks making a financial loss. On the other hand, if its cost estimates are too high, it is likely to be uncompetitive on price and to lose the order. Of course, strategic considerations also play a role and a company may quote a market price based on the knowledge of its estimated costs and its assessment of the highest price at which it can secure an order, which could be significantly higher or lower than its estimated costs.

Where customers can award contracts on the basis of real competition, they may feel reasonably comfortable about accepting contractor prices as quoted, with minimal negotiation, subject to other key parameters such as technical performance, delivery schedule and contract conditions being satisfactory. Nevertheless, in a developmental environment and in the case of technically sophisticated items, it is still important to have a good understanding of the make-up of the price, not least to be able to assess effectively the cost of technical and programmatic changes that may subsequently have to be introduced.

It would be unrealistic to assume that companies will necessarily have the same pricing policy irrespective of whether they are quoting in a highly competitive commercial market or to a national or international space agency on a direct-negotiation basis. In the latter situation, therefore, it is essential that customers have the ability to make their own independent assessment of what is a fair price for a given product and there also has to be some general notion of a basis on which to determine whether the price is reasonable and acceptable or not in the particular circumstances, apart from whether it can be accommodated within a particular budget.

In Annex I of the ESA General Clauses and Conditions of Contract, in the general context of cost accounting matters and specifically concerning the allowability of costs, it is stated that in order to be deemed allowable, costs must be: reasonable, and expedient in nature and not exceeding that expended by a prudent organisation in the conduct of competitive business´. For ESA, which places a significant number of contracts on a direct-negotiation basis, the above serves as a working guideline against which to assess the acceptability of the prices it is quoted.

In practice, the situation is often more complicated, there being the need for some calibration of the reference price to reflect the particular circumstances, such as the nomination of a relatively inexperienced contractor for a particular task (e.g. on the basis of geographical return).

Basic approaches to cost estimation

Stated simply, all estimating depends to a greater or lesser degree on past experience or knowledge. We can therefore repeat the adage of the Chinese philosopher, Confucius, who advised that you should 'study the past if you wish to predict the future'.

Broadly speaking, there are four main approaches to the process of cost estimation:

• The first is the bottom-up or detailed estimate, which is widely used by organisations to build up an estimate from task or work-package level. The number of hours needed for each category of labour, with its appropriate hourly rate, is assessed and a detailed assessment made of the non-labour resources required. This is the most time-consuming and costly approach, requiring a very detailed knowledge of the organisation. However, it is usually necessary for internal planning purposes and, not least, to enable the organisation to comply with customer requirements to present detailed cost estimates down to work-package level. Finally, this level of detail is also usually necessary for cost-control purposes after contract award. Such estimates are usually made by the engineers nominated to perform the work, often overseen by professional estimating services to check for consistency.
• The second method, which can be described as a 'rule of thumb' approach, also relies on the expert judgement of engineers familiar with the tasks being estimated. The experience accumulated by an engineer can constitute a large but unstructured´ knowledge base from which to assess the resources needed for a specific task. This experience is then often translated into 'rules of thumb' which are applied by the engineer for a rough sizing´ of costs. The creation of these 'rules of thumb' does not always follow a systematic process, but the technique is extensively used.
• The third estimating approach uses the principle of analogy. This process is similar to the sort of expert judgement´ described above, but it relies extensively on the degree of similarity of the task to some existing or past task. It requires the collection of source information by the estimator for these analogous elements (usually stored in a computer database) and then comparison with the new task or item to be estimated. The effectiveness of this method depends heavily on an ability to identify correctly differences between the case in hand and those deemed to be comparable.
• The fourth general approach is a statistical and parametrical method. It involves not only the collection of historical cost and technical information, but also its analysis using mathematical techniques. The statistical method correlates costs and manpower information with parameters´ describing the system to be costed. It results in sets of formulae, called cost-estimation relationships´, which are obtained as a result of regression analysis and which can be applied to produce cost outputs for the different elements of an estimate. Such methods can involve considerable effort because of the systematic collection and revision process required to keep the cost-estimation formulae updated, but once this data is available estimates can be produced fairly rapidly. In addition, parametric models´ that have been developed to facilitate this process are now available commercially.

The above approaches are applied in estimating the costs of engineering activities - including design, management, assembly, integration and verification, and product assurance - and the production of hardware. However, software development represents an increasing portion of the space-project effort, and often more than 50% of the total cost of new ground-segment development.

Estimation of the software development effort has always been among the most difficult of processes. Firstly, the abstract nature of software makes it a difficult product to characterise, and design changes are often introduced into software before, during and after its pure code-production phase. Secondly, there are often too few projects in process within a particular organisation to provide a good basis for estimating the cost of new developments. Lastly, software production has evolved in recent years from being more of a black art into an engineering discipline in which there is a very rapid evolution of techniques. This makes the process of basing new estimates on anything but fairly recent developments somewhat precarious. Consequently, the collection of software metrics is now viewed as the best basis for increasing software production efficiency by providing rational ways of measuring and estimating development costs. It also allows the application of statistical estimation.

The changing role of the cost estimator

In Europe, estimating practices differ widely, ranging from relying totally on the traditional bottom-up approach implemented by project personnel, to supplementing this in some cases by a wide range of other approaches applied by professional cost engineers/ estimators.

Differences in the approaches adopted by different organisations are likely to increase in future because the process of cost estimating is developing so rapidly. This is partly through necessity as programmes become more complex, but also because the dramatic advances in computer technology and software are greatly facilitating both the gathering and analysis of data. A further factor is the changing circumstances of many organisations, which is tending to focus attention on cost cutting in general, and thereby on the cost-estimating process also.

Traditionally, the estimating of costs has been one of the last tasks in the process of proposal or programme budget preparation, with the estimates being based on detailed designs or plans. This is all very well, but if the final product of this approach is an unacceptably high cost estimate, time constraints may mean there is no possibility of repeating the exercise with an alternative, less costly design solution or programme approach. At this point, the organisation's only course is to impose an arbitrary percentage cut in the estimated costs, leaving the problem of any financial shortfall to be addressed later. Moreover, for large complex programmes leaving the estimating task to the end, when time is often pressing, invites errors and omissions, particularly in respect of technical and contractual interfaces.

It is therefore desirable to have the ability to estimate costs with a reasonable degree of confidence at an early stage in the process, based on preliminary/summary-type information. The cost implications of a particular design can then be considered and assessed at the outset, as well as progressively through the detailed design process. The design and estimating processes therefore become interactive, with the cost estimate being a vital input to the design solution adopted. This changes the nature of the cost-estimating function from being a purely passive activity to an active one that has a positive impact both on final cost and productivity.

Many organisations are now striving to increase quality/performance at the same time as reducing costs. Consequently, costs are being reduced not only as a result of rationalisation, but also as a result of looking critically at work practices and processes, as a result of encouraging closer teamwork, and from generally trying to create a culture of innovation and continuous improvement.

Activity-based costing is one way of trying to make a more accurate determination of the true time, cost and value of specific activities, and thereby evaluate their real contribution to meeting the overall objective. Some organisations are therefore starting to use this approach in formulating their cost estimates, rather than simply relying on the traditional cost-accounting elements. Through early involvement, the cost estimator can not only influence the final design by feeding in the relevant cost information, but can also actively contribute to cost reduction by identifying cost drivers and to highlight how, for instance, a relatively small increase in system performance can have a disproportionately heavy impact on final cost.

Cost estimation in ESA

There are a number of unique features of ESA and its multi-national operations which have a considerable bearing on its approach to the preparation of estimates and its analysis of costs:

• ESA has over 900 contractors in its 14 Member States and elsewhere, with whom it cooperates.
• In principle, there may be as many as 16 currencies involved for a particular programme.
• The Agency's Industrial Policy requirements can lead to: a high degree of fragmentation of the work; sometimes to non-optimal work allocation and interfaces; somewhat restricted competition; some duplication of developments; awarding of contracts to relatively inexperienced contractors for certain items and under certain circumstances; and political and technical uncertainties can give rise to a high degree of change in the formative stages of programmes when a lot of estimating activity is required.

There are then more general requirements or features which are likely to be common to the estimating functions in most organisations:

• It is desirable that the cost estimator be involved at the earliest possible stage in a programme in order to be well-informed of developments and receive information as soon as it becomes available.
• Estimates are often required quickly and sometimes with minimal inputs.
• There are limited available estimating resources to deal with conflicting priorities.
• Cost drivers and possible cost-reducing features should be identified.
• Risks need to be identified and assessed.
• Iterations of estimates may be required as the design evolves and more information becomes progressively available, thus facilitating the use of a wider range of available estimating tools.
• A comprehensive and well-documented estimating report will be required, including: estimates, explanation of methodology applied, details of technical and programmatic baseline, assumptions, qualifications, risk assessment, and pearls of wisdom´ (additional comments and judgements).

Generally speaking, it is not practicable for ESA to apply the detailed bottom-up estimating method to programme estimates due both to the large numbers of contractors involved and to the many different possible permutations of sources of supply, each using different cost accounting systems, etc. However, as in other similar organisations, most engineers in ESA are involved to some degree in the cost-estimating and analysis process, through their participation in comparing competitive bids, with reference to earlier similar bids and based on a bottom-up-type approach using prior engineering experience. The latter is particularly useful when considering the cost impact of modifications, as the estimate will depend on a close knowledge of the circumstances and timing of the introduction of the change (including any abortive work already performed), the re-planning that may be necessary, as well as awareness of potential impacts on other parties in the contractual chain.

In addition, ESA has at its disposal the skills of the Cost Engineering Section within its Cost Analysis Division at ESTEC. Its principal functions are to:

• produce independent cost estimates for potential new programmes and in anticipation of industrial proposals to be submitted
• evaluate and analyse industrial cost proposals submitted
• provide support services to contracts officers and project-control services for the negotiation of prices.

The centralising of this function offers advantages in terms of the development and availability of 'estimating tools', in making available for the benefit of the whole organisation the data gathering from and analysis of projects from all Directorates, and by providing an experienced, expert task force to support project teams on cost-related matters in all the stages leading up to contract signature.

Furthermore, the Cost Engineering Section has the benefit of direct access to the cost-accounting expertise and records that reside in the Industrial Cost Auditing Section of the Cost Analysis Division.

The tools and methods used

The Cost Engineering Section at ESTEC makes use of the following tools and methods:

ECOS
ECOS - the ESA COsting Software - is a standardised software package developed for the Agency, which permits the submission of cost proposals to ESA on diskettes or via telecommunications links. Its use necessitates a homogeneity of approach at different contractual levels of a proposal for a particular programme and between different programmes. This has led to the adoption of a strictly product-oriented breakdown ('product tree') in conjunction with the traditional Work Breakdown Structure, which tends to be discipline- and organisation-oriented.

ECOS facilitates the preparation and presentation of major proposals, permitting computerised tender integration at each contractual level. It allows analysis of those proposals by higher-level contractors and by the Agency, there being the capability to sort the information in various ways and to generate special reports and graphical presentations. Last but not least, ECOS facilitates the establishment of a cost database containing details of all of these electronically submitted proposals for future reference.

In estimating terms, ECOS's contribution is to automate repetitive processes, to introduce a standardised approach to task breakdown, and to permit the collection of technical and programmatic details to complement the cost details provided. This latter point is important because cost estimates should also be well-documented and, in particular, related to specific technical and programmatic base-lines. This point may seem self-evident but, in an environment in which time pressures are a permanent feature, this approach is some-times not pursued.

A further feature of the ESA procurement process for major programmes, particularly those on a direct negotiation basis, is the need, for various reasons, for a number of iterations in the proposals, often against a changed technical/programmatic base-line. In these circumstances, without well-documented estimates it may be difficult by say the third iteration of the estimate to recall the exact basis used for the first one and to follow the overall evolution of the estimate.

The ECOS package itself was more fully described in an earlier Bulletin article, in May 1993.

Analogy
The analogy method relies on the availability of a database as a reference for future estimates. The implementation of ECOS, and as a result of previous estimating exercises, the Cost Engineering Section has at its disposal an extensive, well-classified and well-documented database covering projects from all ESA Directorates. Its main value lies in the fact that, even for the apparently most innovative programmes, there is usually considerable utilisation of existing designs and technology. At equipment level in particular, items are frequently re-used for projects in different ESA Directorates.

Thus, whilst the Agency does not generally use a bottom-up approach from the work-package level, it does use it at unit and equipment level. Considerable efforts are therefore made to identify such items of equipment and to relate them back to experience acquired with previous proposals, with due regard for present development status and the degree of modification necessary, which implies an element of new development.

Parametric models
For many years, the Agency has used a commercial parametric cost model, called PCM. It has the advantage of being able to run with relatively few inputs. More recently, the Agency has also started to use another commercial model called PRICE (H), following the introduction of a new PC-based version.

Together with the analogy approach, parametric modelling is one of the most commonly used estimating methods within the Cost Engineering Section at ESTEC.

Software projects history database
In 1988, the Cost Engineering Section initiated a continuous cycle of gathering completed software-project development information from many European software producers and promulgating it to all participating companies. This process is now carried out in collaboration with the INSEAD international business school in Fontainebleau, (F)*.

ESA has also participated in a four-year development effort on a software cost-estimation system known as 'Mermaid', funded by the European Commission. It has resulted in a commercial estimating tool based on the application of algorithms to the metrics of past projects stored in a dedicated database. The Cost Engineering Section is making use of this database when preparing cost estimates for new ESA projects. The database is also regularly distributed to companies actively participating in the data collection.

ECOM: an integrated cost-estimating tool
Each cost-estimating approach has its own particular advantages or limitations in a specific situation. For instance, the development status and previous prices for frequently used items of equipment from qualified suppliers will be well-known. However, the cost of singular, atypical spacecraft elements can only be assessed in an approximate way by comparison with other comparable systems. The Cost Engineering Section therefore started to look for a more efficient means of building up a programme estimate using a mixture of cost-estimating techniques for the various elements of a space system and a system was conceived to optimise the use of the basic historical information held by the Agency and of the individual estimating tools available. This system is the ESA Costing Model, or ECOM.

ECOM, which is modular in structure, consists of (Fig. 1):

• An ECOS-derived database, drawn from previous proposals and costs incurred, and a database of previous estimates.
• A knowledge base of Cost-Estimating Relationships (CERS) derived from analysis by the Agency of previous proposals and of parametric input precalculations for each type of spacecraft equipment and subsystem costs.
• A database interrogation module.
• An escalation module to automatically convert different cost elements to the same economic conditions (exchange rates and inflation).
• A risk-analysis module, based on ESA statistical analyses and also using commercially available software.
• Proprietary parametric models.

ECOM therefore allows the combination in a single estimating calculation of any of the cost-prediction methods appropriate for particular elements of the space system being evaluated. Different design options can therefore be rapidly costed and documented in an unambiguous form, thereby allowing good reproducibility and traceability of results.

The system also records the technical and programmatic baseline details, as well as the method by which the inputs for the estimating techniques have been deduced. The price indices issued independently by the Wiesbaden Institute for the Agency´s budgetary updating are also incorporated. There is also an extensive satellite-component price/technical performance database included, which can display prices for any economic conditions, using the appropriate currency exchange rates to compare prices from different sources/countries.

The present ECOM development programme will be completed this year and, inkeeping with the Agency´s general objectives of promoting Europe´s space technology and industrial capabilities, the Agency will then consider making ECOM available to other companies and organisations that could benefit from its application.

The way ahead

As noted at the outset, the space/aerospace cost-estimating process is evolving extremely rapidly. The international nature of ESA and its programmes means that there are particular challenges in estimating the costs of its projects, the largest of which involve many contractors and subcontractors widely scattered geographically throughout the Agency´s Member States. ECOM constitutes a significant advance in terms of ESA´s cost-estimating, and therefore cost-constraining capabilities. In addition, its modular structure will facilitate future enhancements to take full advantage of new ideas and emerging technologies.

Figure 1. The modular structured ESA Costing Model (ECOM)

Figure 2. Sample interrogation of the ECOM database