The Boeing Company

<!-- image -->

United States Government Accountability Office Washington, DC 20548

Matter of: The Boeing Company

File:

B-412441

Date:

February 16, 2016

Comptroller General of the United States

The decision issued on the date below was subject to a GAO Protective Order and contained classified information. This redacted version has been approved for public release. Redactions marked as [DELETED] remove proprietary or source selection sensitive information. Redactions obscured by black line remove classified information per the contracting agency.

Scott M. McCaleb, Esq., Paul F. Khoury, Esq., Jon W. Burd, Esq., Moshe B. Broder, Esq., and Nicole J. Owren-Wiest, Esq., Wiley Rein LLP, and Lynda Guild Simpson, Esq., Suzette W. Derrevere, Esq., and Mark W. Reardon, Esq., The Boeing Company, for the protester.

Col. Robert J. Preston II, Lt. Col. Mark E. Allen, Jeffrey P. Hildebrant, Esq., Christine C. Piper, Esq., Alexis J. Bernstein, Esq., Maj. Michael G. King, Maj. Darren S. Gilkes, Maj. Bertha A. Diaz, Richard B. Clifford, Esq., Jessica S. Mailman, Esq., Steven M. Sollinger, Esq., Bryan O. Ramos, Esq., and Maj. Matthew G. Karas, Department of the Air Force, for the agency.

David A. Churchill, Esq., Kevin C. Dwyer, Esq., Daniel E. Chudd, Esq., W. Jay DeVecchio, Esq., Damien C. Specht, Esq., Kevin P. Mullen, Esq., J. Alex Ward, Esq., and James A. Tucker, Esq., Jenner &amp; Block LLP, and Maureen Del Duca, Esq., and William J. Colwell, Esq., Northrop Grumman Corporation, for the intervenor.

Eric M. Ransom, Esq., David Ashen, Esq., and Edward Goldstein, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

- 1Protest that the agency failed to give proper effect to the solicitation's technical acceptability criteria is denied where the evaluation was consistent with the stated criteria.

- 2.Protest that the agency improperly evaluated the awardee's proposal as technically acceptable is denied where the record shows that the evaluation was reasonable and consistent with the solicitation's terms.

- 3.Protest challenging the cost realism analysis is denied where the agency performed an evaluation of major cost categories utilizing well established realism methodologies and relevant historical cost information available to the agency.

The Boeing Company, of St. Louis, Missouri, protests the award of a contract to California, by the Department of the Air Force, Rapid Capabilities Office, under request for proposals (RFP) , for the Long Range Strike Bomber (LRS-B) engineering and manufacturing development (EMD) and low rate initial production (LRIP) contract. Boeing alleges that the Air Force failed to consider risks inherent in Northrop's approach that should have rendered Northrop's proposal unacceptable or significantly increased Northrop's evaluated cost. Boeing also alleges that the Air Force unreasonably rejected cost data supporting Boeing's proposed costs and employed an arbitrary and unreasonable cost realism methodology.

We deny the protest.

The Air Force began conducting market research concerning long range strike Approval, October 25, 2013, at 8.1 Responses to an initial 2004 request for information indicated that the only contractors capable of producing the required aircraft were Boeing, Northrop, and Lockheed Martin. Id. In 2007, the Air Force initiated the Next Generation Bomber program with Boeing and Northrop (Lockheed Bomber, the Air Force continued to provide funding to Boeing, Northrop, and Lockheed, under separate contracts for additional long range strike aircraft risk reduction and cost savings efforts . Id.; Protest at 24.

The Air Force initiated the LRS-B program in February 2011 in response to a approving a top-level capabilities statement for the system. AR, Tab 7, Secretary of Defense Memorandum February 18, 2011, at 1. The memorandum included an affordability attribute of an approximately $550 million average procurement unit

1 Citations to the Agency Report tabs refer to the page numbers reflected in the electronic .pdf files. Citations to other filings, such as the protest, comments, or final comments, reflect the page numbers of the filed copies.

cost per aircraft (2010 dollars), based on a fleet size of 80 to 100 LRS-B aircraft. Id. at 2. The memorandum further directed that the program "[I]everage demonstrated, mature, integration-ready technologies and systems as much as possible," and "[r]educe complexity and technological risk by integrating only those systems the government's intent was to "keep requirements stable, manageable, and tradable to ensure affordability."Id.

Pursuant to the Secretary of Defense Memorandum, in November 2011, the Air Force issued a justification and approval authorizing the award of contracts to Boeing, Northrop, and Lockheed, to begin technology development (TD) for the LRS-B program. AR, Tab 10, Justification and Approval, October 25, 2013, at 8. In March 2012, the Air Force awarded TD phase contracts to Boeing, Lockheed, and through completion of a Preliminary Design Review (PDR)." Contracting Officer Statement of Facts (COSF), at 6. The TD phase design development effort was to culminate in the PDR, and "the selection of a single contractor to execute Engineering and Manufacturing Development (EMD), Low Rate Initial Production, § 1.6.; COSF at 6. During the TD phase, Lockheed elected to team with Boeing as Boeing's principal subcontractor. Prior to the conclusion of the TD phase, in 2014, Boeing and Northrop demonstrated competing LRS-B preliminary designs, both of which successfully completed the Air Force's PDR. Protest at 13; Intervenor Comments at 78.

For general background information on TD and PDR, the contracting officer directs our Office to the Defense Acquisition Guidebook. COSF at 6 n.1. The Guidebook explains that "the primary objective of the Technology Development (TD) phase is to reduce technical risk and develop a sufficient understanding of the materiel solution to support sound investment decisions . . . regarding whether to initiate a formal acquisition program." Defense Acquisition Guidebook, September 16, 2013, at 209. The Guidebook provides that "[t]here are two key technical objectives in the TD phase: technical risk reduction and initial system development activity, culminating in preliminary design." Id. at 210. The PDR "ensures the preliminary design and basic system architecture are complete, and that there is technical confidence the capability need can be satisfied within cost and schedule goals." Id. at 261; COSF at 6 n.1. The Guidebook further provides that:

A successful PDR confirms that the system's preliminary design:

Satisfies the operational and suitability requirements of the draft [Capability Development Document], as documented in the system performance specification[;]

Is affordable, producible, sustainable, and carries an acceptable level of risk[;]

Is composed of technologies demonstrated in a relevant environment that can be integrated into a system with acceptable levels of risk[;]

Is complete and ready for detailed design . .

Defense Acquisition Guidebook, September 16, 2013, at 262.

On October 25, 2013, as the LRS-B TD contracts were ongoing, the Air Force would solicit LRS-B EMD and LRIP proposals to Boeing and Northrop. AR, Tab 10, Justification and Approval. After receiving approval from the Department of Defense, the Air Force released the RFP to Boeing and Northrop on July 9, 2014. AR, Tab 4, RFP, at 1. The EMD phase portion of the resulting contract (LRS-B) Weapon System requirements," including the manufacture of LRS-B EMD Aircraft. COSF at 6. The LRIP portion of the contract provides option pricing for five lots of LRIP aircraft (consisting of a baseline total of 21 post-EMD aircraft). Id. at 6-7. The contract types are to be cost-plus-incentive-fee for the EMD efforts, fixed-price incentive for the first four lots of LRIP, and not-to-exceed (NTE) (fixed-price incentive to be negotiated) for LRIP lot five. Id. at 7.

The contract was to be awarded on the basis of two factors: technical capability and cost/price. AR, Tab 4c, RFP Amendment 0003, at 43-44. The technical capability factor, which was to be evaluated only for acceptability, consisted of seven subfactors: (1) (2) (3)

(5)

(6)

and (7)

(4)

Id. at 43. Each subfactor consisted of several "Measures of Merit (MOMs)." Id. at 46-54.

According to the RFP, each technical capability subfactor would be evaluated as acceptable or unacceptable, and the overall technical capability factor would be acceptable only if all subfactors were acceptable. Id. at 43. The RFP advised that "[t]he Government will evaluate the offeror's proposal to determine whether it meets the Government's requirements," and "[n]o additional credit will be assessed for exceeding the requirements." Id. at 45. The RFP defined "acceptable" as follows:

For Subfactor 1, the proposal meets requirements. For Subfactors 2-7, the proposal meets requirements and risk of unsuccessful performance is no worse than moderate. Moderate risk means the proposed technical approach can potentially cause disruption of schedule, increased cost or degradation of performance. Special contractor emphasis and close Government monitoring will likely be able to overcome difficulties. Weaknesses may exist but will have little or no impact on contract performance.

Id. at 46. The subfactor evaluation criteria further established that subfactor acceptability required each of the subfactor MOMs to be met. Id. at 46-54.

Under subfactor 1, the RFP required the offerors to submit a chief executive officer (SOW), system requirements document (SRD), and capability verification plan (CVP) in their entireties. Id. at 37. The CEO certification letter was also required to represent either that the proposal was consistent with the LRS-B design presented at the PDR, or identify "all weapon system design characteristics that are both different from what was presented at the PDR and have a consequential effect on PDR." Id. at 46. In this case, as described above, both offerors successfully completed PDR during the TD phase of the procurement; therefore, both CEO certification letters identified that there were no LRS-B design changes from the PDR.

Regarding cost/price, the RFP provided that the proposed EMD costs would be evaluated for cost realism, and adjusted to reflect the most probable cost (MPC) of the offeror's approach. Id. at 55. The MPC would be added to the proposed fixed prices and NTE for the five LRIP lots and the prices for any proposed government furnished property (GFP), and the result of this addition would be the offeror's total evaluated price (TEP).2 Id. at 56. The cost/price evaluation also called for the

2 Also relevant here, the cost proposal instructions addressed decisions to absorb and/or no-cost government use of contractor assets), cost reduction initiatives, and commonalities with other programs. Specifically, the instructions provided that any cost savings or efficiencies proposed as resulting from these approaches would remain subject to cost realism analysis. Concerning investments specifically, the offerors were required to record any no-cost contribution of contractor acquired property or no-cost government use of contractor assets to document that investment as a contractual requirement. COSF at 11-12.

percent of the EMD MPC plus 100 percent of the LRIP price and GFP, and thereby significantly emphasized consideration of the fixed-price production phase of the contract over the EMD effort. Id.

With respect to the selection criteria, the RFP did not contemplate a best-value tradeoff source selection process. Instead, the RFP established a formula for determining best-value based on lowest evaluated price. Specifically, the RFP provided that between technically acceptable offerors, if the TEP of the higher priced proposal was greater than 103 percent of the TEP of the lower priced proposal, the lower TEP proposal would constitute the best value. Id. at 44. If, TEP of the lower priced proposal, then the Air Force would refer to the TWPs, and the proposal with the lower TWP would constitute the best value. Id.

In sum, the source selection criteria for this RFP reflected a lowest-price technically acceptable best-value approach, considering only the acceptability of proposals under the technical capability factor with no credit for exceeding the requirements, and requiring award to the low TEP or low TWP proposal. Federal Acquisition Regulation (FAR) § 15.101-2(b).

Boeing and Northrop each submitted their proposals by the RFP's October 7, 2014 initial closing date. Following the initial evaluation, both offerors' proposals were rated unacceptable under the technical capability evaluation factor. See AR, Tab 16, Initial Evaluation Results. The Air Force then established a competitive range including both offerors and conducted eight rounds of discussions. COSF at 15. Through the discussions process, each offeror resolved all technical deficiencies, and while weaknesses and related risks remained in each offeror's proposal, both proposals were rated acceptable under the technical capability evaluation factor. See AR, Tab 173, Proposal Analysis Report.

Concerning cost/price proposals, as described above, the RFP provided that the contractor investments (contractor decisions to absorb costs), resulting in the calculation of an EMD MPC. The realism evaluation relied heavily on independent were crosschecked with historical data at both the cost category level and the overall projected EMD cost.³ More specifically, the Air Force developed

In order to develop an IGE through which adjusted historical costs could be

compared directly to the offerors' proposals, the Air Force divided the EMD effort into cost categories: (1) air vehicle non-recurring; (2) airframe recurring; (3) weapons suspension and release equipment; (4) propulsion; (5) vehicle (continued...)

offeror-specific parametric estimates for each EMD cost category based on historical program data. These estimates were adjusted by Air Force subject matter experts to account for each offeror's unique approach. AR, Tab 124, Boeing Final Cost/Price Evaluation Summary; Tab 162, Northrop Final Cost/Price Evaluation Summary. The cost category estimates, combined, comprised the IGE total for each offeror, which also constituted the government's estimate of the realistic cost for each offeror's individual approach to perform the EMD effort assuming reasonable economy and efficiency. Id.

The cost/price evaluation team compared the respective IGEs to each offeror's EMD cost proposal, and this comparison served as the primary basis for identifying potentially unrealistic cost categories within each offeror's proposal. The evaluators compared the IGE to the proposed costs at both the EMD level, and at the level of each comparable cost category. Id. Proposed cost categories that were both significantly different from the iGE in the judgment of the cost/price evaluators, and insufficiently substantiated, were deemed potentially unrealistic. Id.

Upon comparing the offerors' initial EMD cost proposals to the offeror-specific IGEs, the agency found that each offeror's cost proposal was significantly lower than their respective IGE and, in the agency's view, appeared unrealistic with respect to multiple cost categories and the total EMD effort. AR, Tab 107, Initial Boeing Cost/Price Summary, February 20, 2015, at 7-8; Tab 138, Initial Northrop Cost/Price Summary, February 20, 2015, at 8-9. Overall, the initial EMD cost proposals and offeror-specific IGEs were as follows:

(...continued)

subsystems; (6) air vehicle software; (7) mission systems; (8) system agency's Boeing-Specific IGE did not include a cost category for air vehicle software. Each MPC also included consideration of state tax incentives to be credited to the Air Force. See AR, Tab 174, Comparative Evaluation report, at 5, 7.

| $[DELETED] | $[DELETED]4 |

|--------------|---------------|

| $[DELETED] | $[DELETED] |

Subsequent to the initial cost evaluation, the cost/price evaluators engaged in multiple rounds of cost discussions, generating evaluation notices for each potentially unrealistic cost category, and communicating in each case the difference between the proposed cost category and the IGE level. Ultimately, through eight rounds of discussions, the Air Force issued 38 cost proposal evaluation notices to Boeing, and 59 cost proposal evaluation notices to Northrop.5 COSF at 17-18; AR, Tab 25, Final Source Selection Briefing, at 9.

After each round of discussions, the cost/price evaluators iteratively refined the individual IGE for each offeror based on the offerors' discussions responses as well as independent review of the historical data and adjustments underpinning the IGE. However, where in the judgment of the cost/price evaluators an offeror's proposed costs nonetheless remained unrealistic, the evaluators adjusted the relevant cost category to the offeror-specific IGE amount, which then generated an additional evaluation notice for the next round of discussions.

Throughout the discussions process, Boeing pursued an approach of revising and supplementing the substantiation of its original cost proposal, without upwardly

4 Here, we note that throughout this evaluation, Northrop's proposed EMD costs were substantially lower than Boeing's due to Northrop's corporate investment decisions. In this regard, Northrop proposed [DELETED], at no cost to the government. AR at 7. This investment, already reflected in Northrop's proposed costs, reduced subsequent Northrop-specific IGE estimates by $[DELETED]. Id. In specific IGE estimates by only $[DELETED]. Id. Additionally, Northrop's lower in comparison to Boeing. Id. at 40-41. In fact, in the final cost evaluation, the Air Force calculated Northrop's IGE on the basis of [DELETED] million labor hours, while Boeing's IGE was based on just [DELETED] million hours. Nonetheless, due to Northrop's investments and labor rate advantages, Northrop's IGE was lowercost. Supplemental AR at 151 n.48.

5 Boeing declined to respond to cost evaluation notices in discussions rounds four and eight, and due to its decision not to respond in round four, did not receive evaluation notices in connection with round five. COSF at 17-18.

Summary, at 13. In contrast, Northrop, while also revising and supplementing the substantiation of its costs, upwardly revised its proposed EMD cost proposal by $[DELETED]. AR, Tab 162, Northrop Final Cost/Price Evaluation Summary, at 7-8. Ultimately, after the eighth round of discussions, the cost evaluators concluded that they had a complete understanding of each offeror's proposal, but that both offeror's categories and the overall EMD effort. Accordingly, for each cost category still deemed unrealistic, the evaluators adjusted the proposed cost to the revised offeror-specific IGE amount. These adjusted costs, combined with the offerors' proposed costs for those cost categories deemed realistic, resulted in the offerors' evaluated EMD MPCs. The final proposed costs and evaluated EMD MPCs were are follows:

<!-- image -->

AR, Tab 174, Comparative Analysis Report, at 5, 7.

In the source selection decision document (SSDD), the SSA concluded that while weaknesses were identified for both offerors under technical capability subfactors 2-7, there was no worse than moderate risk of unsuccessful contract performance and both offerors were technically acceptable. 6 AR, Tab 175, SSDD, at 2. Additionally, the SSA concluded that both offerors had submitted affordable, reasonable, and balanced cost proposals. Id. at 3. Concerning cost realism, the SSA recorded that:

I conclude neither Northrop Grumman's nor Boeing's proposed [EMD] cost is realistic for the work to be performed. While both Northrop Grumman and Boeing demonstrated in their proposals that they have a clear understanding of the solicitation requirements and submitted cost proposals that are consistent with their proposed technical elements, both offerors submitted cost proposals that I believe reflect aggressive attempts to

6 four weaknesses--three weaknesses under subfactor (2) ,and one weakness under subfactor (7) AR, Tab 124, Boeing Cost/Price Summary, at 107-109 (Cost/Technical Crosswalk). Northrop's proposal was acceptable with ten weaknesses--one weakness under subfactor (3) , two weaknesses under subfactor (4) , two weaknesses under subfactor (5) , and five weaknesses under subfactor (6) AR, Tab 162, Northrop Cost/Price Summary, at 160-163 (Cost/Technical Crosswalk).

achieve the lowest evaluated price in this competition. Neither offeror substantiated that it could accomplish all necessary Engineering and Manufacturing Development (EMD) efforts at its proposed cost for EMD.

Id. The SSA then reviewed the MPC for each offeror, concluding that "I understand and agree with the analyses, resulting cost adjustments, and MPC estimates developed [for each offeror]." Id. at 4. The EMD MPCs, combined with the offerors' fixed prices and GFP, resulted in the final TEPs/TWPs, as follows:

AR, Tab 174, Comparative Analysis Report, at 8.

<!-- image -->

Since both offerors were rated acceptable under the technical capability factor, the SSA made the selection decision based on low TEP/TWP. Specifically, in accordance with the RFP's selection criteria, if the higher TEP was greater than 103 percent of the lower TEP, the lower TEP proposal was the best value. Since Boeing's higher TEP of $ Lwas greater than 103 percent of Northrop's lower TEP of $ , Northrop's proposal was selected as the best value without consideration of the TWPs. AR, Tab 175, SSDD at 4.

The Air Force awarded the LRS-B EMD and LRIP contract to Northrop on peb soyo pg fe 'o g-s I s 'P 0 2e Jso debriefing on November 3. AR Tab 45, Boeing Debriefing; Tab 70, Northrop Debriefing. This protest followed.

7 The EMD MPC total does not exactly match the previously-discussed evaluated EMD MPCs due to the addition of several fixed-price EMD line items in this calculation, including technical studies, data packages, and AR, Tab 174, Comparative Analysis Report, at 8.

Boeing alleges that the Air Force's evaluation of Northrop's proposal under the technical capability factor was unreasonable, and that the Air Force failed to properly account for technical risks in its cost realism analysis of Northrop's proposal. Boeing also alleges that the Air Force's cost realism analysis of its own proposal was unreasonable and resulted in an unjustifiably high EMD MPC. We address these arguments in turn.8 8

Under the technical capability factor, Boeing alleges that the Air Force failed to give proper effect to the definition of acceptable set forth in the RFP, failed to evaluate risks in the proposed designs' compliance with all aspects of the RFP system requirements document (SRD) and statement of work (SOW), and unreasonably concluded that Northrop's proposal was acceptable under four of the seven technical capability subfactors:(2) ; (3) ; (4)

- and (6) Boeing also alleges that the Air Force failed to consider, in the technical evaluation, risks identified in the cost realism analysis-specifically, that Northrop proposed overuse of low skill positions and unrealistically low labor rates. In this connection, Boeing asserts that Northrop's failure to propose, and inability to recruit and retain, sufficiently high-level engineers will compound various technical risks alleged in the protest, and therefore should have been evaluated as increasing risk in the technical capability evaluation.

Our review of the record leads us to conclude that the Air Force's evaluation under the technical capability factor was reasonable and consistent with the terms of the RFP, and those requirements of the SRD and SOW identified for evaluation in connection with this competition. The evaluation of an offeror's proposal is a matter within the agency's discretion. IPlus, Inc., B-298020, B-298020.2, June 5, 2006, 2006 CPD 1 90 at 7, 13. In reviewing an agency's evaluation, our Office will not reevaluate proposals; instead we will examine the record to ensure that it was reasonable and consistent with the solicitation's stated evaluation criteria and applicable procurement statues and regulations. Metro Mach. Corp., B-402567.2, June 3, 2010, 2010 CPD 1 132 at 13; Urban-Meridian Joint Venture, B-287168, B-287168.2, May 7, 2001, 2001 CPD 1 91 at 2. A protestor's disagreement with the

3 We have reviewed each argument presented in Boeing's protest and supplemental protest, and conclude that none provide a basis on which Boeing prevails. We address all principal arguments advanced by Boeing in this decision. To the extent that certain arguments or elements of arguments are not discussed in this decision, we have reviewed the record and conclude that the arguments provide no basis to sustain the protest.

agency's evaluation, without more, is not sufficient to render the evaluation unreasonable. Ben-Mar Enters., Inc., B-295781, Apr. 7, 2005, 2005 CPD 1 68 at 7.

Force failed to give effect to the entire definition of "acceptable" as set forth in the RFP for the evaluation of technical capability subfactors 2-7. As described above, the RFP defined "acceptable" in that context as follows:

For Subfactors 2-7, the proposal meets requirements and risk of unsuccessful performance is no worse than moderate. Moderate risk means the proposed technical approach can potentially cause disruption of schedule, increased cost or degradation of performance. Special contractor emphasis and close Government monitoring will likely be able to overcome difficulties. Weaknesses may exist but will have little or no impact on contract performance.

AR, Tab 4c, RFP Amendment 0003, at 46. In particular, Boeing asserts that the Air Force failed to give effect to the final sentence of the definition. According to analyze the impact of Northrop's assessed weaknesses and record a conclusion that the weaknesses "will have little or no impact on contract performance." Id. (Emphasis added.)

Boeing contends that a proper evaluation under this definition of acceptable required the Air Force to first consider whether risks or weaknesses in a proposal "can potentially cause disruption of schedule, increased cost or degradation of monitoring will likely be able to overcome difficulties." AR, Tab 4c, RFP definition required, with respect to weaknesses, that the Air Force consider whether contract performance." Stated another way, Boeing contends that with respect to weaknesses, the definition required the Air Force to consider whether a weakness will have little or no impact on contract performance, assuming that "[s]pecial mitigation approaches fail. Under Boeing's interpretation, if a weakness in a proposal, fully realized, could have more than "little to no impact on contract performance" then the proposal contained greater than moderate risk, and was unacceptable. Boeing Supplemental Comments at 8-9.

The Air Force responds that, contrary to Boeing's interpretation, the natural reading of the definition taken as a whole provides that the final sentence is a summation,

marrying the proceeding concepts in a logical result. In the Air Force's view, the final sentence confirms that the RFP permits a proposal to be acceptable where special contractor emphasis and close government monitoring can mitigate the risks presented by a technical approach--and the attendant weaknesses--such that the risks "will have little or no impact on contract performance." AR, Tab 4c, RFP Amendment 0003, at 46.

we will resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all of its provisions; to be reasonable, and therefore valid, an interpretation must be consistent with the solicitation when read as a whole and in a reasonable manner. Alliance Technical Servs., Inc., B-410307, B-410307.3, Dec. 1, 2014, 2014 CPD 9 345 at 3. Here, we agree with the Air Force that Boeing's interpretation conflicts with a plain reading of the RFP. On our review of the RFP, Boeing's argument is premised on a strained interpretation of the definition which requires the introduction of additional language concerning the realization of risks, or an assumption that the Air Force intended to evaluate weaknesses to such a heightened standard as to cause an illogical result. 9

In this connection, the definition provides that a moderate risk approach is will likely be able to overcome difficulties," and that weaknesses may exist. AR, Tab 4c, RFP Amendment 0003, at 46. However, under Boeing's interpretation, a

Boeing additionally argues that, in fact, the record supports an assumption that the Air Force intended to evaluate weaknesses to a heightened standard under this RFP. Specifically, Boeing highlights that the Air Force sought a waiver of Department of Defense (DoD) Source Selection Procedures to permit an alternate definition of acceptable under this RFP. According to Boeing, the purpose of this incorporate the additional sentence-"[w]eaknesses may exist but will have little or no impact on contract performance"--to ensure that there is no possibility whatsoever that a weakness could have more than "little or no impact on contract performance." Boeing Supplemental Comments at 10. To the contrary, our review of the record demonstrates that the waiver was required to establish a more lenient definition of "acceptable," under which risks and weaknesses could exist. In this regard, the DoD Source Selection Procedures mandate for subfactors to be evaluated on an acceptable/unacceptable basis, a definition of acceptable that does not account for the existence of risks. DoD Source Selection Procedures, March 4, 2011, at 1, A-2. It is apparent from the record that this RFP modified the definition the need to record any evaluated technical capability weaknesses for the purpose of consideration in the EMD cost realism analysis--not for the purpose of composing a more stringent definition of "moderate risk."

moderate risk approach would be rendered unacceptable by the requirement to consider the impact of the risk assuming that mitigation, including "[s]pecial contractor emphasis and close Government monitoring" will fail. Id. This result stands in clear contrast to the plain reading of the definition, which explicitly acknowledges that moderate risk and weaknesses may exist in an acceptable proposal. We see nothing in the record to support Boeing's construction of the definition as requiring consideration of the impact of weaknesses in the event that all mitigation approaches fail, without respect to how remote or improbable such an occurrence would be in the judgment of the Air Force evaluators.

Boeing next alleges that the evaluation of Northrop's proposal was inconsistent with the RFP and unreasonable because the Air Force failed to evaluate risks with respect to all aspects of the SRD and SOW in accordance with the RFP's moderate risk acceptability criteria. In this regard, Boeing alleges that Northrop's approach is contemporaneous record demonstrates that the Air Force failed to recognize or evaluate these risks, which should have rendered Northrop's proposal unacceptable. Boeing asserts that evaluation of all risks in connection with all SRD and SOW requirements was mandated by the RFP's instructions, which provided that "the offeror's proposal . . . shall meet requirements as stated in the [SRD and SOW]." AR, Tab 4c, RFP Amendment 0003, at 6. Boeing also asserts that all data required to demonstrate compliance with the SRD and SOW was to be included in the proposal, and that Northrop's proposal's mere references to its LRS-B design's successful completion of the PDR were insufficient substantiation of its design performance. Boeing Final Comments at 15-17.

We disagree that the Air Force was required to evaluate risks in connection with every aspect of the SRD and SOW under this RFP. We also disagree that the Air Force was required to completely ignore the fact that both offerors in this competition presented matured aircraft designs that had successfully completed PDR--demonstrating technical confidence that the preliminary designs met the capability requirements.

On the basis of our review of the RFP, Boeing misreads the proposal instructions and overlooks the importance of the CEO certification letter requirement. The relevant general instructions for the preparation of proposals, citied in part above, state in full that:

The Government supplies the following general guidance for preparing proposals as well as specific instructions on the format and content of the proposal. The offeror's proposal must include all data and information requested by [these instructions] and must be submitted in accordance with these

instructions. The offer shall be compliant with [the instructions] and shall meet requirements as stated in the System Requirements Document (SRD), Capability Verification Plan (CVP), Engineering and Manufacturing Development (EMD) Statement of Work (SOW), Production SOW, Advance Procurement SOW, Contract Data Requirements List (CDRL), and Model Contract. Failure to comply with the terms and conditions of the solicitation may result in the offeror being ineligible for award. . . . Proposal content alone shall be sufficient to enable the Government to assess the information in accordance with Section M, Evaluation Factors for Award.

AR, Tab 4c, RFP Amendment 0003, at 6. With respect to "specific instructions on

(Emphasis added).

the format and content of the proposal," the RFP did not require that the proposal address every aspect of the SRD and SOW. In this regard, we note that the SRD contained requirements in over of LRS-B contract performance--from and , to and --only 9 of which were identified in connection with any evaluation factor that offerors were required to address. 10 See AR, Tab 4, RFP, at 589-636 (SRD), Tab 4c, RFP (2) , the instructions required that "[t]he offeror shall describe the approach to satisfactorily develop the LRS-B weapon system to meet the SRD and requirements." AR, Tab 4c, RFP Amendment 0003, at 13.

Concerning the SRD and SOW requirements generally, as described above, the RFP required offerors to submit a CEO certification letter. In this regard, the instructions provided that "[t]he offeror shall submit information on a number of specific issues of interest to the Government in assessing contractor compliance with the overall RFP and contract terms, conditions, and other requirements," including the CEO certification letter, an integrated master schedule, and a completed model contract. Id. With respect to the CEO certification letter, the RFP instructions explained that:

The offeror's proposal is required to meet all solicitation requirements including, but not limited to, terms and conditions, representations and certifications, and Statement of Work

Specifically, RFP sections L and M identify proposal content requirements and proposal evaluation criteria with respect to the following SRD requirements:

,

-

,

and

-

(SOW) requirements. The offeror's CEO shall sign and submit a certification letter using the mandatory template in Section L, Attachment 1 certifying acceptance of the System Requirements Document (Section J, Attachment 1), Capability Verification Plan (Section J, Attachment 4), and Statements of Work (Section J, Attachments 2, 3, and 4) without exception.

Id. at 37. Additionally, the instructions provided that:

The CEO letter shall also list and summarize all weapon system design characteristics that are both different from what was presented at PDR and have a consequential effect on performance required in the SRD (Section J, Attachment 1). Use the definition of consequential effect on performance given in Section M, Paragraph 3.1.1.1.

Id. Taken as a whole, our Office interprets the instructions to provide that the proposals shall not take exception to any aspect of the SRD and SOW and shall, in the CEO certification letter, certify acceptance of the SRD and SOW, without exception. Concerning items for evaluation, the "[p]roposal content alone shall be sufficient to enable the Government to assess the information in accordance with proposal content instructions--identifies certain SRD requirements for consideration under the technical capability subfactors. Id. at 6. Thus, only those elements of the SRD and SOW identified in the evaluation criteria were required to be assessed for no worse than moderate risk under the technical capability criteria.

Further, the CEO certification affirmed that the proposed aircraft design was the design presented at the PDR, except as specifically identified. As Northrop's preliminary design successfully completed the PDR, it's CEO certification letter did not identify any characteristics different from the design presented at the PDR. Accordingly, we see nothing unreasonable in the Air Force's acceptance of representations that Northrop's design was the same as presented at the PDR, and therefore had demonstrated preliminary compliance with the SRD requirements. In sum, we see no error in the Air Force's approach.1

11 To the extent Boeing contends that the RFP should have provided for evaluation of the offerors' approaches to meet all SRD and SOW requirements, the allegation constitutes an untimely challenge to the terms of the solicitation since it was not raised prior to the closing date for the receipt of initial proposals as required by our Bid Protest Regulations. See 4 C.F.R. § 21.1(a).

As described above, Boeing alleges that the Air Force's evaluation of Northrop's technical capability proposal was unreasonable, and that a reasonable evaluation would have found Northrop unacceptable with respect to four technical capability subfactors: (2) (3) , (4) , and (6)

Boeing also alleges that the Air Force erred in failing to evaluate technical risk stemming from its conclusion in the cost realism evaluation that Northrop had proposed the overuse of low skill positions and unrealistically low labor rates.

As an initial matter, we note that subsequent to the filing of this protest, the Air Force and intervenor requested dismissal of Boeing's allegations concerning speculative and factually unsubstantiated. Our Office concluded that Boeing's protest grounds would be allowed to proceed on the basis that Boeing had identified its substantial experience and knowledge of the aerospace industry and competitive playing field as the substantiation for its claims. However, during the development of this protest, several technical experts retained by Boeing were granted access to Northrop's technical proposal and, despite this access, Boeing subsequently failed to submit any expert technical opinion or engineering analysis in support of its claims of technical risks associated with Northrop's actual design approach. Instead, Boeing largely repeated its initial assertions that technical risks existed, supported only by reference to its initial protest and statements furnished by legal counsel. Accordingly, while our Office addresses Boeing's allegations on the merits in this decision, we note that Boeing's bare assertions of various "technical risks" inherent in Northrop's approach do not substantiate its allegations.

Subfactor (2)

The subfactor provided for an evaluation of "the offeror's approach to satisfactorily develop the LRS-B weapon system to meet the SRD

,and

requirements.

AR, Tab 4c, RFP Amendment O003, at 13. The subfactor included four sub-elements, each comprised of several MoMs (measures of merit):

and

Boeing's allegations under this subfactor relate to two MOMs under design approach. The first MOM required the Air Force to evaluate whether "[t]he offeror's key design features, to include any design differences from PDR, substantiate the overall aircraft design approach to meet the SRD requirement." AR, Tab 4c, RFP Amendment 0003, at 48. The second MOM required the Air Force to evaluate whether "[t]he offeror adequately explains and

characterizes its top risks [and] describes mitigation steps that are likely to succeed." Id.

<!-- image -->

<!-- image -->

<!-- image -->

acceptance of [DELETED]. In an affidavit, the Air Force's subfactor chief explains that [DELETED]. AR, Tab 218, Subfactor Evaluation Chief Affidavit, at 16. [DELETED]. Id. The subfactor chief further explains that the [DELETED] are unlikely to impact the aircraft performance." Id. at 17.

[DELETED]

We will not substitute our judgment for the considered judgments of the agency's technical experts unless their conclusions are shown to be arbitrary or otherwise unreasonable. R&amp;B Equip. Co., B-271194, May 22, 1996, 96-1 CPD ¶ 250 at 4. Here, we have no basis to question the reasonableness of the Air Force's evaluation.

<!-- image -->

of [DELETED]

[DELETED]

We disagree. First, as addressed above, the RFP did not require proposals to demonstrate, or require the Air Force to evaluate, approaches to meeting every aspect of the SRD and SOW. Rather, concerning general SRD and SOW compliance, the RFP required a CEO certification letter accepting the requirements without exception, and explaining any design differences from the PDR. AR, Tab 4c, RFP Amendment 003, at 13. Northrop's proposal provided the required CEO certification letter which accepted all RFP requirements and represented that the the PDR during the TD phase; indicating that the preliminary design complied with the SRD, including the . AR, Tab 173, Proposal Analysis Report, at 17. The Air Force was not required to revisit Northrop's approach to the SRD under the evaluation factors set forth in this RFP; RFP sections L and M include no reference whatsoever to the

Second, to the extent Boeing argues that consideration of the alleged risk was necessarily for consideration under the

The Air Force's evaluation

The Air Force reviewed Northrop's

AR Tab 166, approach element of the subfactor. Id. Additionally, as addressed above, despite access to Northrop's technical proposal, Boeing has failed to substantiate its initial protest allegations concerning high risks in Northrop's design approach with technical analysis of any kind. In this regard, Boeing offers no support for its contention that Northrop's actual design approach as described in Northrop's technical proposal, would require mitigation consistent with information Boeing learned during the LRS-B TD phase regarding We find no error in the Air Force's evaluation here.

considered Northrop's additional mitigation steps for

Subfactor Evaluation, at 24-25. On consideration of this approach, the Air Force concluded that the proposal "described mitigation steps that are likely to succeed." This evaluation meets the requirements of the relevant MoMs set forth under the

Subfactor (3)

<!-- image -->

Boeing first alleges that Northrop's proposal failed to meet a MOM under the of the subfactor. The MOM required the Air Force to evaluate whether "[t]he offeror's justification of the proposed demonstrates the specification satisfies the SRD AR, Tab 4c, RFP Amendment 0003, at 49.

<!-- image -->

<!-- image -->

Id.

Boeing alleges that this evaluation of Northrop's justification was unreasonable because the Air Force

[DELETED]

[DELETED]. [DELETED

[DELETED]

[DELETED]. [DELETED]. [DELETED]

[DELETED]

We disagree. While Boeing is correct that the RFP required the proposal content to be sufficient to enable evaluation, as previously discussed, we see no prohibition on those demonstrations, and the demonstrated performance levels. In fact, such a prohibition would be illogical, where the entire purpose of conducting a TD phase is "to reduce technical risk and develop a sufficient understanding of the materiel solution to support sound investment decisions" concerning an EMD phase contract. In this case, Northrop reasonably chose to justify that its

met the SRD requirements by establishing the relationship between the and Northrop's , which had been independently analyzed by the Air Force during the TD phase, and was known to surpass the . AR, Tab 167, Subfactor Evaluation, at 38-39. We see no inconsistency between Northrop's approach and the RFP instructions, and nothing unreasonable in the Air Force's conclusion that Northrop's justified that it met the based on its

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

AR, Tab 167, Subfactor Evaluation, at 36-37.

<!-- image -->

in the Air Force evaluation that was unreasonable or inconsistent with the RFP's evaluation requirements.

<!-- image -->

Boeing's criticism of the

conclusion that

represents

disagreement with evaluator technical judgments that does not provide a basis to sustain the protest. See R&amp;B Equip. Co., supra., at 4.

[DELETED]

This determination represents an engineering judgment that our Office will not question, absent clear evidence that it was unreasonable. Id. Boeing has provided no such evidence.

(...continued)

[DELETED]

[DELETED]

"[DELETED]

15

[DELETED]

[DELETED]

[DELETED]

[DELETED]

[DELETED]

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

[DELETED]

<!-- image -->

In sum, on our review of the evaluation record and Northrop's proposal, we again see nothing unreasonable in the agency's conclusions.

- Subfactor (6)

With respect to the subfactor, the agency was required to evaluate "the offeror's approach to execute sufficient to verify requirements." AR, Tab 4c, RFP Amendment 0003, at 53. The subfactor included three sub-elements:

and approach contained one MOM, which required the Air Force to evaluate:

The

The offeror's description of its proposed approach to verify that the realized LRS-B design meets the SRD requirements

(...continued)

RFP criteria. Applying the reasonable interpretation of the acceptability criteria--that a weakness may be assessed in light of the likelihood of disruption of schedule, increased cost or degradation of performance, and whether special contractor emphasis and close Government monitoring will likely be able to overcome difficulties--we see no error in the agency's evaluation here.

includes all supporting data and analyses requested in [the instructions]. The offeror's description includes credible data, and analyses demonstrate sound methodology with reasonable assumptions. The offeror depicts the key tasks associated with its proposed approach in the [integrated master plan] and [integrated master schedule] in a logical, well-ordered approach.

AR, Tab 4c, RFP Amendment 0003, at 53. The sub-element contained multiple MoMs, the second of which required the Air Force to evaluate whether ld.

Boeing alleges that the evaluation was unreasonable where the Air Force relied on two contradictory mitigation approaches to conclude that Northrop's proposed approach was no more than moderate risk. Boeing also alleges that the Air Force failed to reasonably consider the compounding effect of the evaluated weaknesses taken together. As described below, we conclude that the evaluation was reasonable.

<!-- image -->

In its initial evaluation of Northrop's proposal, the Air Force assessed multiple deficiencies and weaknesses, and concluded that Northrop's proposed approach represented a high risk of unacceptable contract performance. AR, Tab 170, Subfactor Evaluation, at 4. Northrop's first revised proposal addressed the assessed deficiencies by correcting errors and providing additional substantiation, however, Northrop did not fundamentally change its approach and retained all aspects of its as described above. Id. at 5-6.

As a result, the first revised proposal failed to resolve multiple weaknesses and the

approach continued to represent a high risk of unacceptable contract performance.Id.

As relevant, the Air Force assigned six weaknesses to Northrop's first revised proposal, which concerned: (1) ; (2)

: (3)

(4)

(5)

; and (6)

AR,

Tab 170, Test and Evaluation Subfactor Evaluation, at 5-6. Of these weaknesses, the agency found that the combined effect of the first five weaknesses presented a high risk of unsuccessful contract performance, while the sixth weakness--

--did not contribute to high risk.Id.

(DELETED])

[DELETED]

[DELETED]

[DELETED]

[DELETED]

<!-- image -->

<!-- image -->

<!-- image -->

<!-- image -->

We see no basis to conclude that the Air Force's evaluation under this subfactor was unreasonable. As described in small part above, the Air Force conducted a thorough evaluation of the approach and the substantiating documentation provided in Northrop's proposal. The evaluation was highly critical of Northrop's initial proposed approach in multiple areas, as demonstrated by the evaluation of multiple weaknesses that combined to reflect a high risk of unacceptable contract performance, even after initial rounds of discussions. However, the Air Force's evaluation of Northrop's revised approach, while recognizing that weakness remained, determined that the approach resulted in a risk of unsuccessful contract performance that was no worse than moderate.

We cannot conclude that this evaluation was unreasonable, or failed to recognize the scope of the risks presented in Northrop's approach. Rather, as evidenced by the multiple weaknesses persisting in the evaluation of Northrop's revised proposals, these risks were thoroughly considered and acknowledged to hold the potential, individually and in combination, to cause a disruption of schedule, increased cost, or degradation of performance. In fact, the evaluation record goes so far as to establish that, concerning the weaknesses in combination, there exists some level of Air Force expectation that disruption of schedule may occur. However, the Air Force nonetheless concluded that Northrop could still successfully execute its approach with a that would not appreciably increase the risk of unsuccessful contract performance. Boeing's disagreement with the Air Force's judgment concerning the combined impact of the weaknesses presented in Northrop's approach provides no basis to sustain the protest. R&amp;B Equip. Co., supra., at 4; Ben-Mar Enters., Inc., supra., at 7.

Finally, Boeing alleges that the technical capability evaluation of Northrop's proposal was unreasonable where it failed to capture the technical risks associated with the Air Force's conclusion, concerning Northrop's cost proposal, that Northrop proposed unrealistically low engineering labor rates, and to overuse low skill mix positions. In this regard, Boeing asserts that the Air Force failed to account for the potential that Northrop's lack of skilled engineers will hinder its attempts to resolve allegedly high-risk aspects of Northrop's technical approach resulting in significant disruption of schedule and increased cost.

that Northrop's revised proposal resolved the Air Force's concern that Northrop's Tab 162, Northrop Cost/Price Evaluation Summary, at 129-134. Accordingly, the record does not support Boeing's allegation that the technical evaluation ignored a cost concern regarding Northrop's proposed labor mix. Second, to the extent the revised cost evaluation cited a concern regarding Northrop's actual labor rates, the Air Force considered and accounted for this risk by making corresponding upward adjustments to Northrop's labor costs. Id. at 134-135. By accounting for this as a technical risk.

Boeing first alleges that the Air Force's upward adjustment to Northrop's proposed EMD costs failed to account for the many high risks that Boeing alleged were inherent in Northrop's technical approach. Boeing next alleges that the Air Force's EMD cost realism analysis unreasonably rejected proposed EMD costs that Boeing substantiated materials costs, and improperly normalized costs without respect to the offerors' unique approaches. 19

19 Boeing generally asserts, throughout its filings, that the Air Force's cost evaluation lost sight of the importance of affordability under the LRS-B program. Boeing argues that the Air Force penalized the offerors' attempts to "break the cost curve" where it rejected innovative cost reducing approaches and unreasonably substituted costs derived from historical programs that exemplified the cost overruns that the agency now seeks to avoid. Protest at 4-6. First, as reviewed below, we see no error in the Air Force's rejection of supporting cost data presented $[DELETED] over Boeing's proposed costs, would nonetheless represent the (continued...)

When an agency evaluates a proposal for the award of a cost-reimbursement contract, an offeror's proposed costs are not dispositive because, regardless of the costs proposed, the government is bound to pay the contractor its actual and allowable costs. FAR §§ 15.305(a)(1), 15.404-1(d); Exelis Sys. Corp., B-407673 et al., Jan. 22, 2013, 2013 CPD 1] 54 at 7; CGl Fed. Inc., B-403570 et al., Nov. 5, 2010, 2011 CPD 1 32 at 5 n.1. Consequently, an agency must perform a cost realism analysis to determine the extent to which an offeror's proposed costs are realistic for the work to be performed. FAR § 15.404-1(d)(1); DynCorp Int'I LLC, B-411465, B-411465.2, Aug. 4, 2015, 2015 CPD 1 228 at 8. An agency is not required to conduct an in-depth cost analysis, see FAR § 15.404-1(d)(1), or to verify each and every item in assessing cost realism; rather, the evaluation requires the exercise of informed judgment by the contracting agency. AdvanceMed Corp.: TrustSolutions. LLC, B-404910.4 et al., Jan. 17, 2012, 2012 CPD 9| 25 at 13. While an agency's cost realism analysis need not achieve scientific certainty, the methodology employed must be reasonably adequate and provide some measure of confidence that the rates proposed are reasonable and realistic in view of other evaluation. Metro Mach Corp., B-295744, B-295744.2, Apr. 21, 2005, 2005 CPD 1 112 at 10-11; Science Applications Int'l. Corp., B-290971 et. al., Oct. 16, 2002, 202 CPD 9 184 at 17. Our review of an agency's cost realism evaluation is limited to determining whether the cost analysis is reasonably based and not arbitrary. TriCenturion. Inc.; Safeguard Servs., LLC, B-406032 et al., Jan. 25, 2012, 2012 CPD 1 52 at 6. As described below, we have considered each of Boeing's arguments and conclude that none provide a basis to sustain the protest.

one must not lose sight of the significant cost advantages maintained by Northrop (such as Northrop's decision to contribute a $[DELETED] investment of [DELETED], as well as its notably lower labor rates and labor rate escalation amounting to an approximately $[DELETED] cost advantage in the final cost evaluation), which in large measure are unchallenged by Boeing. Accordingly, Boeing has a significant hurdle to overcome for the purpose of establishing a reasonable possibility that it

(...continued)

second lowest-cost new aircraft development effort in recent history--higher-cost than only the C-17 program development effort, and significantly lower-cost than development of the B-1A/B, B-2, or F-22 programs to which the LRS-B effort was compared. AR, Tab 124, Boeing Final Cost/Price Evaluation, at 29. We agree with the agency's analysis that, contrary to Boeing's assertion, its EMO MPC was at the low end of the range for modern new aircraft development costs, and was consistent with the LRS-B program approach--reflecting firm requirements, increased design maturity, reliance on non-developmental systems, and implementation of advanced manufacturing processes. Id.

suffered competitive prejudice as a consequence of the cost errors that it has alleged. In order to fully appreciate the cost advantage maintained by Northrop, we begin our discussion of the cost realism evaluation with additional relevant background information describing the significance of Northrop's substantial cost/price advantage--maintained from its initial proposed costs through the agency's final MPC (most probable cost) evaluation--under the award criteria in this RFP.

In this regard, as addressed above, between technically acceptable offerors, the RFP provided that if the higher TEP (total evaluated price) was more than 103 percent of the lower TEP, award was to be made to the lower TEP proposal. If instead, the higher TEP was within 103 percent of the lower TEP, then the award made to the lower TWP proposal. AR, Tab 4c, RFP Amendment 0003, at 44. As relevant, the offeror's final TEPs/TWPs were calculated as follows:

AR, Tab 174, Comparative Analysis Report, at 8.

<!-- image -->

was greater than 103 percent of Northrop's lower TEP of and that, therefore, Northrop's proposal represented the best value without consideration of the TWPs. This conclusion however, obscures the fact that even if Boeing's higher TEP was reduced to within 103 percent of Northrop's lower TEP, Northrop would remain the best value on the basis of its lower TWP. Further, even if Boeing's TEP were lower than Northrop's TEP, if Northrop's higher TEP remained within 103 percent of Boeing's lower TEP, Northrop would continue to represent the best value on the basis of its lower TWP.

20 The EMD MPC total does not exactly match the previously-discussed evaluated EMD MPCs due to the addition of several fixed-price EMD line items in this calculation, including technical studies, data packages, and AR, Tab 174, Comparative Analysis Report, at 8.

Due to Northrop's $ cost/price advantage in LRIP production price, Northrop's TWP would always remain lower than Boeing's TWP while TEPs are within 103 percent.

Thus, in order to prevail in its protest, Boeing must show that its EMD MPC should Northrop's TEP is not within 103 percent. This reduction would require Boeing to demonstrate an evaluation error of over $

Boeing attempts to demonstrate competitive prejudice in this context by asserting both that Northrop's EMD MPC was understated, and that the Air Force's cost adjustments to its initial cost proposal were warranted. Boeing's efforts in this regard fall short on all counts. First, we find no basis to conclude that the Air Force failed to appropriately consider the technical risks of Northrop's proposal in the cost realism evaluation. Second, Boeing's protest acknowledges that the Air Force's cost by nearly $[DELETED]." Boeing Comments at 76. Since we conclude that the Air Force thoroughly considered [DELETED] cost data and reasonably rejected it as a basis for estimation, it is evident that Boeing cannot demonstrate the necessary competitive prejudice necessary for our Office to sustain the protest.1 Competitive prejudice is an essential element of a viable protest; where the protester fails to demonstrate that, but for the agency's actions, it would have had a substantial chance of receiving the award, there is no basis for finding prejudice, and our Office will not sustain the protest, even if deficiencies in the procurement are found. HP Enter. Servs., LLC, B-411205, B-411205.2, June 16, 2015, 2015 CPD 1 202 at 6; Booz Allen Hamilton Eng'g Servs., LLC, B-411065, May 1, 2015, 2015 CPD 1 138 at 10 n.16.

We have reviewed Boeing's allegation that the Air Force failed to adequately consider multiple high-risk aspects of Northrop's technical approach in the EMD

21 Boeing calculates the exact sum of the upward adjustment stemming from the Air Comments at 75. If only that amount were added to Boeing's final EMD cost proposal as an upward MPC adjustment, Boeing's resulting TEP would be $[DELETED]. As Northrop's TEP of $ would remain within 103 percent of Boeing's lower TEP in this scenario, Northrop's proposal would remain the best value on the basis of its lower TWP. Thus, Boeing cannot demonstrate prejudice where the Air Force properly rejected the Boeing cost proposal estimates substantiated by [DELETED].

protest. First, as addressed in the technical capability analysis above, we have approach, and concluded that they are unsubstantiated, pertain to SRD requirements not identified for evaluation under the RFP criteria, and fail to demonstrate any error in the Air Force's evaluation. Second, we conclude that the risks that were noted in the Air Force's evaluation of Northrop's technical approach were appropriately recognized and captured in the Air Force's EMD cost realism analysis.

Generally, the Air Force prepared a cost/technical crosswalk for each offeror which reconciled the cost and technical evaluations for the purpose of assuring that each offeror's EMD MPC was appropriate, considering technical approach and evaluated risks. In this case, the cost/technical crosswalk for each offeror explained that additional upward adjustments were not required to account for evaluated technical risks, because the likely costs of addressing those risks were subsumed within the upward adjustments made to address the offeror's substantial understatements of concluded that:

Although significant differences existed between the IGE and proposed cost for [EMD], this was less a manifestation of risk associated with the technical plan and more a result of [the offeror's] failure to generate a credible estimate for its technical approach.

Northrop Final Cost/Price Evaluation Summary, at 158. Consistent with this finding, the cost/technical crosswalks concluded that upward adjustments previously made to address each offeror's understatement of costs, also sufficiently addressed the impact of technical risks.

<!-- image -->

<!-- image -->

Boeing next alleges that the Air Force erred in rejecting Boeing cost estimates that were based on Defense Contract Audit Agency-certified cost data from the [DELETED]. Boeing's cost proposal relied on [DELETED] cost data to develop cost estimates concerning multiple cost categories. For the majority of those estimates, the [DELETED] was the exclusive source for the basis of estimate.

Boeing argues that the [DELETED] is [DELETED] to measure Boeing's proposed LRS-B costs in relevant areas. Boeing argues that [DELETED]. Boeing also argues that [DELETED]: In sum, Boeing argues that [DELETED] "provided a trove of actual cost data supporting substantially leaner, more efficient, and less large aircraft programs," and that the Air Force's rejection of this data in favor of analogies from less relevant programs (largely the F-22), was unreasonable. Protest at 66.

We have reviewed the Air Force's evaluation and conclude that Boeing's use of [DELETED] cost data to develop its EMD cost estimates was fairly considered and reasonably rejected. For example, concerning

, under which Boeing's proposed costs were upwardly adjusted by $[DELETED]--the record shows that the Air Force considered Boeing's use of cost estimating relationships based on [DELETED], and rejected them for several reasons.

First, the Air Force concluded that "[r]esearch conducted by the Air Force Life Cycle Management Center (AFLCMC) Cost and Economics Division shows that [DELETED]. AR, Tab 124, Boeing Final Cost/Price Evaluation Summary, at 32-33. Second, the Air Force concluded that Boeing's cost analysis was flawed where it appeared that Boeing had calculated its proposed LRS-B costs using factors from [DELETED] as opposed to [DELETED]" consistent with [DELETED]. Id. at 33. Third, the Air Force concluded that while Boeing had proposed adjustments to the [DELETED] data to account for differences in development scope and schedule, the "adjustments were inadequate for adjusting [DELETED] to a level consistent with full-scale weapon system development." Id. In this regard, the Air Force's evaluation determined that Boeing's overall of $[DELETED] was based on approximately [DELETED] million labor hour estimate for the cost category in the Boeing-specific iGE, which was based on cost data from the F-22 program.22 Id. at 32-33.

During discussions, the Air Force communicated to Boeing the basis for its rejection of [DELETED] cost data, and the IGE of $[DELETED]. Boeing continued to base its proposed costs on [DELETED], did not adjust its costs, and attempted to further substantiate its costs based on a crosscheck of 16 other historical programs. However, the Air Force concluded that many of the programs cited in the crosscheck were older and failed to reflect integration of low observable technologies, and that the analysis "did not enhance the credibility of Boeing's conclusions or the realism of its estimate for AR, Tab 124, Boeing Final Cost/Price Evaluation Summary at 34.

22 The Air Force explains that the Boeing-specific IGE for was based on an analogy to the costs of the F-22 program, adjusted to reflect Boeing's LRS-B EMD approach including "design maturity and the evolution of design tools and processes since the F-22." AR, Tab 124, Boeing Final Cost/Price Evaluation Summary at 34. Generally, the IGE estimate was based on an estimating relationship deriving adjusted labor hours per month from specific phases of the F-22 program, and translating those labor hours to Boeing's unique schedule at 125-127.

Additionally, the Air Force conducted its own cross-check considering total non-recurring development hours on other historical aircraft EMD programs, and ([DELETED] million hours for plus [DELETED] million hours for )was more consistent with derivative aircraft development programs such as the P-8 to adapt a commercial aircraft) and the B-1B to adapt from the B-1A). AR, Tab 124, Boeing Final Cost/Price Evaluation Summary at 34. The Air Force, however, believed that the more apt comparison was to the level of effort for new aircraft development programs such as the C-17 ), F-22

), or B-1A since the LRS-B effort involves new aircraft development as opposed to the adaption of an existing aircraft. Id. In this analysis, the Air Force concluded that Boeing's [DELETED] million non-recurring labor hours was understated in comparison to the IGE, which developed an estimated total of [DELETED] million non-recurring hours. Id. However, based on review of Boeing's response and re-analysis of F-22 program data to better distinguish between engineering and manufacturing labor hours, the Air Force reduced the Boeing IGE costs for to $[DELETED], based on [DELETED] million labor hours. Id.

In subsequent discussions, the Air Force communicated its conclusions to Boeing. AR, Tab 31, Boeing Evaluation Notices, May 19, 2015, at 22-23. In response, Boeing again continued to base its proposed costs on [DELETED], did not adjust its costs, and attempted to further substantiate its costs by providing another crosscheck, which was apparently limited to the [DELETED] programs. AR, Tab 124, Boeing Final Cost/Price Evaluation Summary at 35. The Air Force analyzed this data and concluded that the labor hours projected by the crosscheck credibility of Boeing's estimate. Id. at 35. Further, the Air Force disagreed with adjustments that Boeing made to [DELETED] program data in the crosscheck, which the Air Force concluded "essentially ignored the [DELETED] commonality and understated the effort required for [DELETED] "23 Id. The Air

23 Briefly, Boeing's crosscheck substantially downwardly adjusted the [DELETED] labor hours on the basis of its own analysis of [DELETED]. AR, Tab 81, Boeing Revised Proposal, July 24, 2015, at 22. This analysis led Boeing to conclude that because the [DELETED] program included [DELETED], and because a significant re-design was required following completion of the

, the [DELETED] labor hours actually represented the effort required to develop [DELETED] new aircraft designs. Id. at 22-25. The Air Force reviewed Boeing's analysis and concluded the analysis gave insufficient weight to the commonality of the [DELETED] in adjusting the [DELETED] labor hours, which weakened the credibility of the crosscheck.

Force concluded that "normalization of [DELETED] labor hours strongly influenced the [crosscheck] result and called into question the validity of the crosscheck." Id.\_

Thereafter, the Air Force again communicated its conclusions to Boeing. AR, Tab 37, Boeing Evaluation Notices, July 16, 2015, at 10-11. In response, Boeing again continued to base its proposed costs on [DELETED], did not adjust its costs, and attempted to further substantiate its costs by providing another crosscheck. This crosscheck updated the prior crosscheck to modify adjustments to the [DELETED] cost data. AR, Tab 124, Boeing Final Cost/Price Evaluation Summary at 36. The Air Force again concluded that Boeing's adjustments of the [DELETED] data gave insufficient recognition to [DELETED]. Id. Additionally, the Air Force could not determine what additional programs were used in the updated crosscheck, and noted that Boeing twice explained that the update corrected errors concerning [DELETED] data, which was not a program that had been identified as within the prior crosscheck. Id. The Air Force also noted that labor hours in the with no rationale as to why the crosscheck was valid to support the proposal. Id. at 36.

After three rounds of discussions in this area, the Air Force concluded that Boeing had failed to substantiate the proposed costs estimated from [DELETED] data, which was rejected based both on [DELETED], and because the estimated costs remained uncorroborated by any other analysis performed by Boeing or by the Air Force. In its final evaluation, the Air Force elaborated that:

[DELETED]. [DELETED] is not comparable to the LRS-B development effort. Examples of differences in [DELETED] include [DELETED]; [DELÉTED]. [DELETED]. [DELETED]. [DELETED].

<!-- image -->

not substantiate its proposed cost for ," which "is not Evaluation Summary at 37. The Air Force therefore adjusted Boeing's costs to the Boeing-specific IGE level.

We see no error in the Air Force evaluation of Boeing's cost estimates derived from the [DELETED]. As noted, from the initial evaluation, Boeing was advised that [DELETED], was "not an analogy representative of reasonable economy and efficiency upon which to build a realistic estimate," and that the use of [DELETED] data, versus the [DELETED] data, was a major concern. We have reviewed the explanations for these decisions as recorded in the contemporaneous record and conclude that they are thorough and reasonably based. Concerning additional substantiation, through three rounds of discussions Boeing repeatedly refused to modify its [DELETED] assumptions and initial proposed costs, despite its failure to present any acceptable analysis of other programs corroborating its proposed costs. In this context we can see no basis on which to conclude that the Air Force's rejection of those costs and upward adjustment to the level of its own estimate of the costs of Boeing's approach was improper. Offerors are responsible for submitting a well-written proposal with adequately-detailed information that allows for a meaningful review by the procuring agency. AdvanceMed Corp.: offeror is responsible for affirmatively demonstrating the merits of its proposal and runs the risks associated with its failure to do so. Saco Defense, Inc., B-252066, May 20, 1993, 93-1 CPD 1 395.

To the extent Boeing argues that the Air Force, in the contemporaneous evaluation record or in its report, failed to rebut Boeing's own analysis of the similarities between the [DELETED] and the LRS-B EMD effort, the Air Force is not required to do so. Rather, the RFP emphasized that "[t]he burden of proof for credibility of of the offeror's substantiation need only be reasonable. AR, Tab 4c, RFP Amendment 0003, at 21; Wisconsin Physicians Serv. Ins. Corp., B-401068.14, B-401068.15, Jan. 16, 2013, 2013 CPD 1] 34 at 13 ("While [the protester] effectively asserts that the burden was on the agency to disprove the realism of its projections, the contrary is true. An offeror is obligated to present an adequately written proposal, including sufficient support for its assumptions."). We see no error in the Air Force's evaluation. 25

<!-- image -->

Boeing also alleges that the Air Force unreasonably rejected its proposed materials costs, where those costs were well substantiated and based on fixed-price vendor quotations. The record does not support Boeing's allegations in this case. Rather, the record shows that the majority of Boeing's materials quotations were not fixed-price or otherwise lacked adequate support.

Specifically, the evaluation record shows that the agency reasonably concluded that the majority of Boeing's material quotations were, at best, rough order of magnitude (ROM) estimates that specifically included qualifying language. For example, among Boeing's materials quotations are the following limitations: [DELETED]; [DELETED]; [DELETED]. AR, Tab 74, Boeing Revised Proposal, March 23, 2015, Enclosure 3, at 2218, 2228, 2238.

We see no error in the Air Force's consideration of Boeing's proposed materials costs and conclusion that the materials costs were unsupported. Further, to the costs at a material by material level as set forth in its proposal, as described in scientific certainty, the methodology employed must be reasonably adequate and provide some measure of confidence that the rates proposed are reasonable and time of its evaluation. Metro Mach Corp., supra, at 10-11. Parametric estimation of costs based on relevant historical cost data is a reasonable cost realism approach. See FAR 15.404-1(c)(2)(i)(B) and (C) (establishing that the government may use various cost analysis techniques to verify cost information, including historical cost or pricing data" and "estimates generated by appropriately calibrated and validated parametric models"); Hernandez Eng'g, Inc., B-286336, et al., Jan. 2, 2001, 2001 CPD 1 89; AT&amp;T Corp., Adv. Tech. Systems, B-261154, B-261154.4, Oct. 16, 1995, 96-1 CPD 1 232.

<!-- image -->

improper normalization of costs without respect to each offeror's unique approach. In this regard, Boeing notes that the Air Force rejected the vast majority of each offeror's proposed costs, and thereafter substituted offeror-specific IGE cost the most basic aspects of the individual proposals, such as schedule, weight, and labor rates. Boeing alleges that this "top-down" methodology was fundamentally flawed. Boeing maintains that the Air Force was required to derive the MPC from the bottom up, considering each offeror's cost elements at the lowest level of detail proposed, and upwardly adjusting individual items where the specific items were unsubstantiated or lacked cost realism. Boeing Comments at 67.

We disagree, and conclude that the Air Force conducted a reasonable cost realism evaluation that sufficiently incorporated consideration of each offeror's unique approach. First, we have reviewed the record here and, as discussed above concerning [DELETED] costs, we conclude that the agency thoroughly considered and reasonably rejected the offerors' cost proposals in the majority of cost categories, on the basis that both offerors failed to generate credible estimates of Final Cost/Price Evaluation Summary at 106-107. Accordingly, the Air Force was required to determine some reasonable basis on which to upwardly adjust the offeror's proposed costs to reflect the MPC of a resulting contract. See FAR should realistically expect to pay for the proposed effort").

Second, we disagree that the Air Force considered only superficial aspects of each offeror's unique approach, and are aware of no authority that requires an agency to consider the realism of proposed costs at the lowest possible level of granularity. Rather, agencies have great discretion in selecting an appropriate method for their cost realism analyses. Tetra Tech EC, Inc., B-406975 et. al., Oct. 9, 2012, 2012 CPD 1 286 at 3. In this case, the agency developed an estimate of each offeror's individual cost to perform various cost categories comprising the EMD effort through relevant historical programs. In this regard, for each cost category, for each offeror, the Air Force first identified relevant analogous aircraft EMD programs and considered the availability of reliable and sufficiently granular cost data. Then, in

6 While this analysis did, for most cost categories, result in the use of the same prior program data for each offeror due to the availability of sufficient program cost data between programs, the analysis also selected different programs for each (continued...)

consultation with technical subject matter experts, the Air Force adjusted the chosen program data to account for anomalies in the program or differences with respect to the LRS-B. The cost/price evaluators and technical subject matter experts then made offeror-specific adjustments based on each offeror's unique approach, and incorporated additional data provided by the offerors through discussions. The result of this analysis represented both the offeror-specific IGE, and the Air Force's best estimate of realistic costs of pursuing each offerors' proposed technical approach. COSF at 20-22.

The Air Force compared the offeror-specific IGE with each offeror's proposed costs. If the proposed cost was not reasonably close to the IGE in the view of the subject matter experts, the evaluation considered the substantiation underlying the protester's estimate. Where the estimate was reasonably close to the IGE or was otherwise substantiated, the Air Force accepted the proposed costs.27 Only where the estimate was deemed unsubstantiated and remained unsubstantiated following multiple rounds of discussions, did the Air Force adjust the cost category to the cost level of the offeror-specific IGE for the purposes of the MPC. COSF at 22; AR, Tab 173, Proposal Analysis Report, at 14.

because the Air Force relied on the same basic estimating methodology in establishing each offeror-specific IGE, or because the parametric estimates for many cost categories utilized the same prior programs for comparison. Instead, it is apparent from the record that for each cost category, the Air Force selected the most relevant prior program that captured the required cost data. For example, the Air Force considered information from the F-22 program where it required recent

(...continued) offeror where data was available and unique aspects of the proposals justified the difference. For example, for weapon suspension and release equipment non-recurring, the Air Force estimated Boeing's costs by direct analogy to the [DELETED] program due to similar [DELETED] technology, and estimated Northrop's costs by analogy to the [DELETED] due similar [DELETED] technology. Supplemental AR at 165.

27 For example, concerning Boeing's propulsion approach, Boeing proposed cost for the cost category was approximately $[DELETED]. AR, Tab 124, Boeing Final Cost/Price Evaluation Summary at 56. The initial Boeing-specific IGE estimate for Boeing's propulsion approach was $[DELETED], and was not considered to be reasonably close to Boeing's proposed costs. Id. However, after consideration of Boeing's substantiation and a discussions response, the Air Force concluded that "Boeing's proposed cost for this element was consistent with properly-adjusted representative historical data," and included Boeing's proposed propulsion costs in the MPC. Id. at 57.

data distinguishing the costs of a ground test article versus flight test aircraft, because the F-35 program data did not capture that distinction. AR, Tab 127, Boeing IGE, at 28.

On this record we conclude that the Air Force's cost estimates were prepared at the lowest level of detail consistent with the subject matter experts' degree of confidence in adjusting relevant historical aircraft EMD cost data. This approach far exceeds the minimum threshold necessary for a reasonable cost realism analysis. Our Office has previously considered and accepted the use of parametric estimates based on historical program costs to develop an offeror's most probable model to evaluate offeror's costs where the offeror failed to provide sufficient detail to verify its proposed cost estimate); Newport News Shipbuilding &amp; Drydock Co.. insufficient data were available from other relevant ships); FAR 15.404-1(c)(2)(i)(C). Accordingly, we cannot conclude that the Air Force's methodology was unreasonable, or constituted an improper normalization of costs.

In sum, our review of Boeing's allegations and the evaluation record in this case provides no basis on which to sustain the protest. In our view, the record demonstrates that the Air Force reasonably interpreted the RFP's evaluation criteria including the role of SRD and SOW requirements and definition of acceptable in the technical capability evaluation. The Air Force's evaluation of Northrop's proposal under the technical capability factor was reasonable and consistent with the RFP. Finally, with respect to the cost/price evaluation, we see no support for Boeing's argument that the Air Force failed to reasonably account for Northrop's technical evaluation of Boeing's proposal was flawed. Significant structural advantages in Northrop's proposal--specifically, its labor rate advantage and decision to absorb significant company investment--also strongly impacted the outcome of this essentially low-price, technically acceptable procurement, and Northrop's significantly lower proposed prices for the LRIP phase created a near-insurmountable obstacle to Boeing's proposal achieving best-value, or to Boeing's protest demonstrating prejudice in the cost realism evaluation.

The protest is denied.

Susan A. Poling General Counsel