Trusted Base complains that DOS misevaluated its proposal under every evaluation factor. In reviewing protests of alleged improper evaluations, it is not our role to reevaluate proposals. Rather, our Office will examine the record to determine whether the agency’s judgment was reasonable and in accord with the stated evaluation criteria and applicable procurement laws and regulations. Worldwide Language Res., Inc., B-297210 et al., Nov. 28, 2005, 2005 CPD para. 211 at 3. A protester’s mere disagreement with the agency’s judgment does not establish that an evaluation was unreasonable. UNICCO Gov’t Servs., Inc., B-277658, Nov. 7, 1997, 97-2 CPD para. 134 at 7. We have reviewed all of Trusted Base’s arguments and find that none has merit. We address some of Trusted Base’s arguments below.

Trusted Base asserts that its proposal was impermissibly downgraded under the understanding of the requirements and technical approach evaluation factor. The protester complains that it was penalized for failing to adequately address the approach/knowledge of office administrative support evaluation subfactor, even though this subfactor had been deleted from the evaluation scheme. Protest at 5; Protester’s Comments at 5.

Although DOS agrees that this evaluation subfactor was deleted--and acknowledges that despite its deletion the subfactor remained on the previously-prepared evaluation materials--it explains that its concern was that the protester, unlike AGTech, did not propose any FTEs for the [deleted] position, a staffing requirement set forth in the TOR’s “Service Offering Estimate.” Thus, in the agency’s view, it was not Trusted Base’s failure to discuss its general approach to, or knowledge of, office administrative support that resulted in the assessment of a weakness in Trusted Base’s proposal. Rather, it was the omission of an important position. The agency notes in this regard that this omission raised concerns about the firm’s understanding of the comprehensive TOR staffing requirements.

Trusted Base argues that it was unreasonable for the agency to assume that the firm’s proposed [deleted] FTEs for the base year staff did not include the [deleted] function position. See Protester’s Comments at 5. In fact, Trusted Base points out that its proposal clearly indicated that it would satisfy this requirement by using its [deleted] support to ESOC. Id.

While the record shows that Trusted Base’s proposal explained that its [deleted] support, we do not think it was unreasonable for the agency to be concerned about the proposal’s failure to address the requirement for a [deleted]. In addition, this position was specifically identified as a requirement in the TOR, and in other areas the agency found that other aspects of Trusted Base’s proposed staffing indicated a lack of understanding of the TOR’s comprehensive requirements. To the extent the protester believes that the information in its proposal explaining how it would use its [deleted] (a position not usually associated with [deleted]) should have addressed the agency’s concerns, the protester is simply disagreeing with the agency’s judgment; the protester’s disagreement does not make the agency’s concerns unreasonable. See UNICCO Gov’t Servs., Inc., supra.

In any case, Trusted Base’s central arguments focus on the agency’s judgment that the protester’s proposed staffing for the base and option years was understated and created doubt as to the company’s ability to successfully perform the TOR services. The protester disagrees with this evaluation finding on the basis that the data in the “Service Offering Estimate” was simply an estimate of the staffing hours required. As such, the protester asserts it could propose staffing hours that, in its business judgment, would be sufficient to perform the required services. Protester’s Comments at 2-5.

Again, while Trusted Base is correct that the staffing hours and labor categories in the TOR were estimates, the protester’s argument misses the point. Potential offerors were instructed to use this information--which was based on historical workload data and TOR requirements--in preparing their proposals. Thus, this information provided a level-playing field for reviewing how offerors would perform--and the relative differences in their cost to perform--the time and materials effort at issue here. When the protester elected to generate its own estimates of the future workload, it assumed the risk that its fewer proposed staffing hours would result in an evaluation finding that the firm’s staffing plan was understated. Also again, the protester’s disagreement with the agency’s evaluation judgment does not render it unreasonable. Coastal Drilling, Inc., B-285085.3, July 20, 2000, 2000 CPD para. 130 at 4.   (Trusted Base, LLC, B-401670, October 27, 2009) (pdf)

First, AINS objects to the agency's finding that its product failed "to fully meet" critical requirement [deleted], which requires the vendor to propose [deleted] RFQ, Att. D (Requirements Matrix), at 24. The protester responded to this requirement by noting in its self-assessment that this feature was targeted for release in version 7.0 of its FOIAXpress software, which was due out in July 2009. During discussions, the agency noted that the requirement had to be met by the time of user acceptance testing, and that AINS's proposed date of July 2009 appeared to be well beyond the user acceptance testing date. The agency also noted that the proposed solution would be in an AINS product (version 7.0) that was not yet in existence. Accordingly, the agency asked AINS to confirm whether it could meet the requirement by the time of user acceptance testing and to explain how it would do so and the state of development of its software with regard to meeting the requirement.

AINS responded by stating that it would have critical requirement [deleted] met by the time of user acceptance testing. The protester explained that it would accomplish this by enhancing the latest version of its FOIAXpress software to include the ability to [deleted], and that after this had been accomplished, a user with the appropriate permission would be able to [deleted], which could then be [deleted]. The protester furnished a detailed timeline for completing and testing the enhancement.

While the contemporaneous record lacks detail as to the basis for the evaluators' finding that AINS's response failed to demonstrate full compliance with requirement [deleted], the agency furnished the following explanation in its report: although AINS's revised quotation had indicated that, once enhanced, the protester's software would be able to [deleted], and that these reports could then be exported into various formats, "it [did] not address making the underlying [deleted], which is what Section [deleted] specifically requires." AR at 16. The agency also noted that the protester's response made clear that its software was in a developmental stage.

The protester argued in response that it had addressed the requirement for saving the [deleted] in a format that would allow it to be made available to the public by explaining in its discussion question response that its upgraded software would permit the [deleted]. The protester further argued that the evaluators had treated the vendors unequally by failing to identify Privasoft's response to the requirement as a weakness, even though it also failed to address making the [deleted] available to the public.[8] In its report on the protest, the agency responded to the protester's allegation of unequal treatment by conceding that Privasoft had not directly addressed the requirement pertaining to [deleted]; DOJ maintained that it reasonably had not questioned Privasoft's compliance, however, because Privasoft had "elsewhere indicated unequivocally it could meet the requirement." The agency noted in this connection that Privasoft had represented in its self-assessment that the capabilities required by [deleted] were "currently available" in its software. Supp. AR at 16.

We fail to see a meaningful distinction between the two vendors' responses to the requirement pertaining to [deleted], neither of which specifically addressed the capability to make the data available outside the context of a report.[9] Accordingly, we do not think that the agency has demonstrated a reasonable basis for finding that AINS, but not Privasoft, failed to "fully meet" the above requirement.  (AINS, Inc., B-400760.2; B-400760.3, June 12, 2009) (pdf)
 

SBSS asserts that the technical evaluation of IST’s proposal was inconsistent with the cost/price evaluation, in that the technical proposal was not reevaluated in light of the staffing adjustments made for purposes of the most probable cost adjustment. According to the protester, an offeror “could propose a technical approach which included staffing well in excess of the Government’s optimal staffing levels (and thus receive a higher technical rating) and not be penalized” in the technical evaluation even where the agency assumed a lower staffing level in evaluating cost. SBSS Comments at 2.

Where the cost and technical evaluation of a proposal reach contradictory conclusions, the agency generally is required to reconcile the evaluations. See Serco, Inc., B-298266, Aug. 9, 2006, 2006 CPD para. 120 at 7; Information Ventures, Inc., B-297276.2 et al., Mar. 1, 2006, 2006 CPD para. 45 at 6.

Here, there was an ultimate disparity between the technical proposal staffing and the staffing level used in the cost evaluation--based on its estimate of required FSA staffing, the Air Force assumed lower FSA staffing for IST in the cost evaluation than the level proposed in IST’s technical proposal, resulting in a downward adjustment in IST’s cost/price. While, on its face, this appears to present an evaluation inconsistency, in fact, there is no indication that IST’s proposal received a higher evaluation rating as a result of the higher staffing level in its technical proposal; the record indicates that IST’s proposal in fact received only an acceptable rating, without any evaluated strengths, under the technical approach/support subfactor of the mission capability factor. Since IST received no evaluation benefit from its higher proposed staffing level--there is no argument that IST’s staffing should have been found less than acceptable--there is no basis for us to object to this aspect of the evaluation of IST’s proposal.  (SelectTech Bering Straits Solutions JV; Croop-LaFrance, Inc., B-400964; B-400964.2; B-400964.3,  April 6, 2009) (pdf)

As an initial matter, we are aware of no statute or regulation that requires an agency to create a consensus report in evaluating proposals, nor is there any requirement that every individual evaluator’s scoring sheet track the final evaluation report. See Andrulis Corp., B-281002.2, June 2, 1999, 99-1 CPD para. 105 at 5 (a consensus score need not be the same score as initially scored by the individual evaluators). In short, these matters alone will not lead our office to conclude that there was an impropriety in the agency’s evaluation or a violation of procurement law or regulation.  (Smart Innovative Solutions, B-400323.3, November 19, 2008) (pdf)

As an initial matter, we find the protester’s criticisms of the SEB’s assignment of adjectival ratings and point scores and its corresponding documentation of this process misplaced. In Wackenhut’s view, the number of strengths and weaknesses found for each offeror “has to be the most important basis for an adjectival rating and score,” Protester’s Comments at 16, and, given that the “significant strengths” were the chief discriminator between the proposals, “if [Wackenhut’s] having twice as many significant strengths as Coastal is not a meaningful difference, then nothing in the SEB’s process can be.” Id. at 18. Wackenhut also maintains that in those instances where the SEB in fact gave some significant strengths more value than others, it failed to create a record explaining its rationale in this regard, thereby, in the protester’s view, rendering the SEB’s exercise of its discretion “unreviewable” and “illegal.” Id.

In a hearing conducted by our Office, during which we heard testimony from the SSA and the SEB Chairperson, the SEB Chairperson described the evaluation process and explained that each proposal was evaluated independently against the criteria set forth in the RFP, with the SEB reaching consensus on its findings (the various strengths and weaknesses identified for each proposal). Based upon these findings, the SEB assigned, on a consensus basis, adjectival ratings, and then point scores. In assigning ratings and scores, the SEB Chairperson explained that, with respect to significant strengths, it was the content of the offerors’ proposals, as reflected in the specific findings, which was used to rate and score offerors. The SEB Chairperson stated that the process of assigning ratings and scores was not merely a “numbers game” driven by the mere number of significant strengths identified in an offeror’s proposal. Hearing Transcript (Hr. Tr.) at 82.

We find the agency’s evaluation process unobjectionable. At the core, Wackenhut seeks a mathematical or mechanical consideration of the number of significant strengths in determining the evaluation ratings and attributing points to the offerors’ proposals. Adjectival ratings and point scores, however, are mere tools in the evaluation and selection process and should not be mechanically derived or applied. Rather, it is the agency’s qualitative findings in connection with its evaluation of proposals--in this case the documented written narratives underlying and justifying the SEB’s findings of particular significant strengths--that govern the reasonableness of an agency’s assessment of offerors’ proposals. MCR Federal, Inc., B-280969, Dec. 14, 1998, 99-1 CPD para. 8 at 9.

Here, the offerors’ proposals were evaluated independently against the RFP’s evaluation criteria and the evaluation ratings and scores simply served as an expression of the agency’s exercise of its discretion in making its qualitative findings. As a consequence, there is nothing per se improper with Coastal and Wackenhut receiving similar ratings and scores under the mission suitability factor, and related subfactors, notwithstanding the fact that Wackenhut’s proposal had more significant strengths as compared to the proposal submitted by Coastal. All Star Maintenance, Inc., B-271119, June 17, 1996, 96-1 CPD para. 278 at 4 (holding that agency’s evaluation of two offers as essentially equal was not unreasonable notwithstanding the fact that the protester had five strengths while the awardee had only two strengths). Absent some basis for concluding that the technical findings underlying the ratings and scores were flawed, there is no basis for our Office to conclude that the SEB’s ratings were unreasonable. Moreover, with respect to the documentation issue, the relevant material is NASA’s underlying technical findings, which have been amply documented by the SEB, and it is those findings which are reflected in the SEB’s qualitative findings and which formed the basis of the adjectival ratings and point scores.  (Wackenhut Services, Inc., B-400240; B-400240.2, September 10, 2008) (pdf)

PHASE-IN

ManTech challenges NASA's evaluation of offerors' approaches to phase-in at the start of the contract. In this regard, the RFP, which included a 30-day phase-in period, provided that the government would evaluate under the management plan subfactor (under the mission suitability factor) each offeror’s phase-in approach

for continuity and a smooth transition with the incumbent Contractor during the 30-day phase-in period. The Government will evaluate how clearly the phase-in plan demonstrates an ability to assume full contract responsibility on the effective date of the contract. The Government will evaluate how the phase-in plan specifically address[es] . . . the proposed . . staffing plan. . . .

RFP sect. M.3. ManTech asserts that the agency failed to recognize its advantage as the incumbent contractor.

Analex’s proposal was rated overall excellent under the management plan subfactor. This rating was based in part on two evaluated significant strengths, one for proposing (1) an “extremely well-planned, comprehensive and detailed phase-in plan that provides for a smooth transition and demonstrates a superb ability to assume full contract responsibility,” and the other one for a superior approach to using bonuses tied to the award fees earned by Analex and other bonuses as an incentive to maximize overall employee morale and performance. SSD at 4; Integrated Evaluation Panel Revised Final Report at 41-53. In addition, Analex’s proposal received four strengths under the management plan subfactor, for (1) proposing a good system (its online task order management system discussed above) to process task orders and manage multiple ongoing tasks, (2) a good approach for mitigating performance risks and for interfacing with government personnel, (3) a robust fringe benefits package, which would aid in recruiting and retaining employees, and (4) a thorough, comprehensive Mission Assurance Plan which demonstrated Analex’s knowledge of and compliance with GSFC’s Mission Assurance Guidelines. Analex’s only evaluated weakness under this subfactor was an inadequate location allowance and severance pay policy. Integrated Evaluation Panel Revised Final Report at 41-53.

ManTech’s proposal likewise was rated overall excellent under the management plan subfactor. ManTech’s rating was based in part on a single evaluated significant strength for a superior approach to phase-in, involving no major changeover of personnel, processes or procedures, which would significantly reduce the risk of phase-in. In addition, ManTech’s proposal received six strengths under the management plan subfactor, for proposing (1) policies and incentives that have a high probability of retaining employees, maintaining high morale and increasing productivity in a union environment, (2) a good task order and property management system, (3) a good approach to managing workload variability, (4) desirable benefits for part-time workers, (5) an Integrated Knowledge Environment portal that demonstrated a good approach to managing contract work, and (6) a robust total compensation plan. In addition, ManTech’s proposal received a weakness on the basis that, in contrast to Analex’s thorough, comprehensive mission assurance plan, ManTech’s significantly less detailed mission assurance plan was inadequate, failing to demonstrate compliance with GSFC’s Mission Assurance Guidelines. SSD at 7; Integrated Evaluation Panel Revised Final Report at 104-18.  Thus, the record indicates that NASA in fact recognized, and assigned ManTech’s proposal a significant strength under the management plan subfactor on account of its superior approach to phase-in. NASA specifically recognized that, given ManTech’s policies and incentives, reflected in a 95 percent employee retention rate over the past 5 years, ManTech had a high probability of maintaining its incumbent workforce, thereby significantly reducing transition risk. Integrated Evaluation Panel Revised Final Report at 104-06.  However, NASA also recognized that Analex had proposed an “exceptional strategy,” warranting a significant strength, in support of its proposal to hire [REDACTED] percent of the incumbent employees. In this regard, the agency noted that Analex had proposed a very attractive package for recruiting incumbent employees, including: [REDACTED]. In addition, NASA concluded that Analex’s proposed superior approach to bonuses (for which it earned a second significant strength) would likewise increase the likelihood of recruiting incumbent employees. In this regard, the agency noted that Analex had proposed bonuses [REDACTED], in a total amount of up to approximately [REDACTED] percent of salaries, with the bonuses commencing when Analex earned an award fee of at least [REDACTED] percent. (In contrast, in ManTech’s proposed approach of offering bonuses tied to the earned award fee, Mantech commenced the bonuses only when ManTech earned an award fee above [REDACTED] percent, and it only encouraged, but did not require, participation by its subcontractors.) NASA also determined that Analex’s robust fringe benefits package would aid in recruiting employees. Finally, the agency determined that Analex’s plan for [REDACTED], would greatly improve the likelihood of a successful transition. Integrated Evaluation Panel Revised Final Report at 41-45; Agency Supplemental Report, Mar. 17, 2008, at 4-11; ManTech FPR Mission Suitability Proposal at 113, 118; Analex FPR Mission Suitability Proposal at C-26.

We conclude that NASA reasonably determined that Analex offered an exceptional transition approach, with many incentives for incumbent employee retention, which warranted a significant strength under the management plan subfactor. Furthermore, ManTech has not shown that, given Analex’s overall two significant strengths and four strengths under the management plan subfactor, it was unreasonable for the agency to rate Analex’s proposal as excellent in this area. Certainly the fact that ManTech was the incumbent contractor here did not automatically entitle it to a higher rating under the management plan subfactor. See Karrar Sys. Corp., B‑310661, B-310661.2, Mar. 3, 2008, 2008 CPD para. 51 at 4-5. (ManTech International Corporation, B-311074, April 15, 2008) (pdf)

The evaluation of proposals is a matter within the discretion of the contracting agency, and in reviewing protests against allegedly improper evaluations, it is not our role to reevaluate proposals. Rather, our Office examines the record to determine whether the agency’s judgment was reasonable, in accord with the evaluation factors set forth in the RFP, and whether the agency treated offerors equally in its evaluation of their respective proposals and did not disparately evaluate proposals with respect to the same requirements. Hanford Env’t. Health Found., B-292858.2, B-292858.5, Apr. 7, 2004, 2004 CPD para. 164 at 4; Rockwell Elec. Commerce Corp., B-286201 et al., Dec. 14, 2000, 2001 CPD para. 65 at 5.

As stated above, the contemporaneous record does not evidence any consideration, or even awareness, by the Army of the stated “assumptions” in Fluor’s TEP that there would be a period between TO award and NTP during which it would receive incremental funding and could perform various tasks, such as procuring material and deploying equipment. See Fluor TEP, Scenario Approach, at 3, 33-34. The fact that the agency may have been unaware of the assumptions on which Fluor’s TEP stated it was predicated does not change the fact that Fluor’s assumptions were not consistent with the terms of the scenario that the other offerors, as well as the agency, treated as mandatory. It is a fundamental principle of federal procurement that a contracting agency must treat offerors equally, and the agency’s acceptance of Fluor’s proposal stating that it was predicated on the above-mentioned assumptions was improper and unfair to the other offerors.[9] See Farmland National Beef, B‑286607; B-286607.2, Jan. 24, 2001, 2001 CPD para. 31 at 8 (a proposal that fails to conform to one or more of a solicitation’s material terms is technically unacceptable and cannot form the basis for an award); Loral Terracom; Marconi Italiana, B-224908; B‑224908.2, Feb. 18, 1987, 87-1 CPD para. 182 at 9 (agency’s acceptance of a proposal based upon a methodology different than that set forth in the solicitation was improper where the agency did not inform all offerors that the agency’s requirements were not as rigid as indicated in the solicitation). Moreover, we cannot find the agency’s evaluation of Fluor’s TEP under the technical evaluation subfactors reasonable, given that it was based upon a misreading of Fluor’s proposal with regard to the stated “assumptions.” The record does not show that this is an instance where an agency noted that a proposal was taking exception to a solicitation requirement in some respect, but ultimately determined that the proposed approach would meet the agency’s needs. Rather, the contemporaneous record, as well as the agency’s arguments during the course of these protests, evidence that the Army simply misread or altogether overlooked the stated “assumptions” in Fluor’s proposal regarding the period of time and availability of funding for tasks to be performed between TO award and NTP.  (Contingency Management Group, LLC; IAP Worldwide Services, Inc., B-309752; B-309752.2; B-309752.3; B-309752.4; B-309752.5; B-309752.6; B-309752.7, October 5, 2007) (pdf)

GAP asserts that the agency’s proposal scoring scheme was flawed in that it essentially “negated” the technical distinctions among the proposals. In this respect, the record shows that the agency evaluated proposals by assigning numeric scores between 0 and 5 points for each of the 13 evaluation subfactors, and then multiplied the raw score by the weight assigned to each of the subfactors to arrive at weighted scores. According to the protester, because all of the acceptable proposals (that is, proposals that it describes as likely to have been included in the competitive range) would in practice be assigned raw numeric scores of either 3 or 4, the effect was to artificially narrow the range of possible total scores, such that it would appear from the numeric scores that all of the proposals were technically equal. According to the protester, this effectively left the source selection to be based on low price rather than on technical considerations which were to have received paramount consideration under the terms of the RFP.  This argument is without merit. It is well established that ratings, be they numerical, adjectival, or color, are merely guides for intelligent decision making in the procurement process. Business Consulting Assocs., LLC, B-299758.2, Aug. 1, 2007, 2007 CPD para.134 at 4. Where the evaluators and source selection official reasonably consider the underlying bases for the ratings, including advantages and disadvantages associated with the specific content of competing proposals, in a manner that is fair and equitable and consistent with the solicitation, a protester’s disagreement over the actual adjectival or numeric ratings assigned essentially is inconsequential in that it does not affect the reasonableness of the judgments made in the source selection decision. Id. The record here shows that the agency performed an evaluation of the proposals consistent with the RFP evaluation factors and prepared detailed narrative materials reflecting the evaluators’ findings. AR exhs. D12, D19. The record also shows that those findings were specifically considered in detail by the agency’s source selection official in making the agency’s award decision. Indeed, because the scoring was so close among the four proposals, the source selection official expressly queried the evaluators “to determine if there was a proposal that contained technical aspects that would clearly set it apart from any of the other ones.” AR exh. G7, Source Selection Decision, at 8. The technical panel concluded that no proposal had technical advantages or discrepancies that would set it apart from the other proposals and, as a result, the proposals were determined to be technically equal. GAP does not challenge any of the agency’s underlying substantive findings with respect to the relative merits of the competing proposals. The protester does not, for example, allege that the agency unreasonably failed to identify strengths that were present in its proposal or unreasonably identified weaknesses that were not present; nor does it challenge the agency’s underlying findings with respect to the other proposals in the competitive range, including the awardee’s. Moreover, GAP also has not alleged or demonstrated that it should have been assigned higher numeric scores, or that the awardee should have been assigned lower numeric scores, based upon the strengths or weaknesses identified by the agency in its evaluation. In these circumstances, given the absence of any substantive challenge to the agency’s detailed narrative evaluation findings, we simply have no basis to question the agency’s evaluation results. We therefore deny this aspect of GAP’s protest.  (Gap Solutions, Inc., B-310564, January 4, 2008) (pdf)

To the extent that Sherrick argues that the selection decision did not accurately tally the number of strengths and weakness for each proposal, or that the selection decision focuses on specific discriminators between the proposals instead of their evaluation ratings, these arguments are unavailing. The evaluation of proposals and the assignment of adjectival ratings should generally not be based upon a simple count of strengths and weaknesses, but upon a qualitative assessment of the proposals consistent with the evaluation scheme. See Smiths Detection, Inc., B-298838, B-298838.2, Dec. 22, 2006, 2007 CPD para. 5 at 5-6. Moreover, it is well established that ratings, be they numerical, adjectival, or color, are merely guides for intelligent decision-making in the procurement process. Citywide Managing Servs. of Port Washington, Inc., B-281287.12, B-281287.13, Nov. 15, 2000, 2001 CPD para. 6 at 11. Where an agency reasonably considers the underlying bases for the ratings, including advantages and disadvantages associated with the specific content of competing proposals, in a manner that is fair and equitable and consistent with the terms of the solicitation, the protesters’ disagreement over the actual adjectival or color ratings is essentially inconsequential in that it does not affect the reasonableness of the judgments made in the source selection decision. See id.; National Steel & Shipbuilding Co., B-281142, B-281142.2, Jan. 4, 1999, 99-2 CPD para. 95 at 15. The record here shows that the agency had a rational basis for its evaluations and source selection determination. Although our decision does not address every challenge raised by Sherrick regarding the number of strengths and weaknesses or adjectival ratings assigned to each offeror’s proposal, we have reviewed all of the issues raised and find that none has merit. (Sherrick Aerospace, B-310359.2, January 10, 2008) (pdf)

Pemco first challenges the agency’s assignment of “satisfactory” confidence ratings to both Pemco’s and Boeing’s proposals under the past performance factor. In this regard, Pemco complains that the agency “unreasonably failed to rate Pemco higher than Boeing,” and that the agency “was unreasonable in rating both Boeing and Pemco equally.” Protest at 15, 17. As our Office has frequently stated, adjectival ratings are only guides to assist agencies in evaluating proposals; information regarding particular strengths and weaknesses of proposals is the type of information that source selection officials should consider, in addition to ratings and point scores, to enable them to determine whether and to what extent meaningful differences exist between proposals. See, e.g., TPL, Inc., B-297136.10, B-297136.11, June 29, 2006, CPD para. 104 at 17. In this regard, proposals with the same adjectival ratings are not necessarily of equal quality, and an agency may properly consider specific advantages that make one proposal higher quality than another. See Pueblo Envtl. Solution, LLC, B-291487, B‑291487.2, Dec. 16, 2002, 2003 CPD para. 14 at 10; Oceaneering Int’l, Inc., B-287325, June 5, 2001, 2001 CPD para. 95 at 13. Here, the record clearly establishes that the agency did not evaluate Boeing’s and Pemco’s proposals as being equal with regard to past performance; rather, Pemco’s proposal was found superior to Boeing’s. Specifically, although both proposals received overall ratings of “satisfactory” confidence, the agency’s contemporaneous evaluation documents show that, after considering multiple strengths and weaknesses in both offerors’ past performance record, Pemco’s overall past performance rating was considered to be at “the high end of satisfactory” and Boeing’s overall rating was considered to be [deleted]. PAR at 159; Email from SSA (Riechers) to SSAC Advisors, Aug. 5, 2007. Similarly, the SSA unambiguously stated in his source selection decision: “I consider Pemco to have a better past performance record than Boeing.” SSDD at 22. Accordingly, there is no merit to Pemco’s protest that the two proposals were evaluated “equally” with regard to past performance; to the contrary, the agency considered this aspect of Pemco’s proposal to be better than Boeing’s. (Pemco Aeroplex, Inc., B-310372, December 27, 2007) (pdf)

Bio-Rad finally argues that the agency’s evaluation of IDEXX’s proposal as “satisfactory” under the small disadvantaged business participation factor was unreasonable. The proposal preparation instructions stated with regard to the small disadvantaged business participation factor that offerors “[s]hall identify in this section of the technical proposal specific large business, small business, HUBZone small business, small disadvantaged business, [WOSB], [VOSB], and HBCU/MI and Service Disabled Veteran-owned small business subcontract participation in contract performance.” RFP at 56. The RFP concluded here that “[t]he proposed participation of subcontractors will be evaluated based on the information submitted in the Subcontractor Information Sheet,” a copy of which was provided as an attachment to the solicitation. RFP at 57. The section of IDEXX’s technical proposal addressing the small disadvantaged business participation factor consists of one sentence, which states that “[t]here is no subcontract participation in the contract performance.”[15] AR, Tab 13, IDEXX’s Technical Proposal, at 18. The Subcontractor Information Sheet attached to IDEXX’s proposal contained a single notation of “‑NA‑.” Id. at 19. The agency evaluated IDEXX’s proposal under the small disadvantaged business participation factor as follows:

[IDEXX] does not propose any subcontracting under this contract and did not submit a subcontracting plan. Therefore, the firm is technically neutral in this area and will receive a “Satisfactory” rating.

AR, Tab 6, TEB Consensus Memorandum, at 11; Tab 7, BCM, at 23. The Source Selection Memorandum provides some explanation as to the reasoning behind this aspect of the agency’s evaluation, stating:

The firm did not intend to subcontract any portion of the subject contractor to large or small business entities if it received award. Therefore, in accordance with the [FAR] and the [SSP] I found the firm satisfactory in the area of small business subcontracting.

AR, Tab 8, Source Selection Memorandum, at 4.

In responding to Bio-Rad’s protest, the agency does not defend the propriety of its evaluation of IDEXX’s proposal under the small disadvantaged business participation factor as “satisfactory.” With that said, we note that contrary to the apparent belief of the SSA, there is no provision in the FAR that either requires or allows an agency, under the circumstances here, to evaluate a proposal as “neutral” and thus “satisfactory” under a factor designed to evaluate the participation of small disadvantaged businesses and the other types of small businesses, where the proposal expressly provides that there will be no participation by such businesses if the offeror is awarded the contract. Additionally, and contrary to the view of the SSA, there is nothing in the SSP or RFP providing or otherwise indicating that the evaluation of IDEXX’s proposal as “satisfactory” under the circumstances here was appropriate, particularly given that firm’s large business status.[16] Accordingly, the agency’s evaluation of IDEXX’s proposal under the small disadvantaged business participation factor as “satisfactory” was not reasonably based. (Bio-Rad Laboratories, Inc., B-297553, February 15, 2006) (pdf)

The record shows that, in calculating the number of hours necessary for SCLINS 0002AAA and SCLIN 0003AA, the IGCE standard apparently had included the staffing required for two other SCLINS (SCLINS 004AAA and 0005AAA, the program management functional area SCLINS for Hungary and Macedonia). Amended KTR Program Management Document, May 27, 2005, at 1. The record further shows that there were two variables that were considered by the agency in making these changes, the number of hours per year for FTEs and the number of FTEs overall for each functional area. We find nothing objectionable in the agency’s recalculation of the number of labor hours per FTE; because the IGCE standard had used what was subsequently considered an unreasonably high number of hours per FTE, the agency reasonably averaged the offerors’ number of hours per FTE to arrive at a revised figure for use in the LSA standard. Id. However, the agency’s explanation for arriving at the number of FTEs required to perform these functional areas is problematic. The record shows that the agency used KBR’s proposed staffing approach as the basis for arriving at the distribution of FTEs among the various SCLINS. Specifically, the cognizant agency personnel determined that, because KBR’s staffing approach was similar to the IGCE staffing approach, they would use KBR’s proposed staffing to arrive at the appropriate number of FTEs for the various program management functional area SCLINS; they concluded that this would provide a more realistic FTE and hour count for the government standard. Amended KTR Program Management Document, May 27, 2005, at 1. Both the agency and the awardee maintain that there was nothing improper in the agency’s actions because, by using KBR’s proposed staffing approach as the basis for establishing the agency’s evaluation standard, the agency actually reduced the FTEs under SCLINs 0002AAA and 0003AA more than if ITT’s proposed staffing had been used. The agency’s actions were improper. The agency’s use of an offeror’s staffing approach to arrive at the government’s standard was not a reasonable substitute for an agency-generated estimate based on historical workload data and the PWS requirements. While the agency’s estimate and the staffing approach proposed in the KBR proposal could be similar, in the absence of an analysis from the agency reconciling the two, there was no reasonable basis for the agency to use the KBR staffing approach as its benchmark for evaluation purposes. Using the KBR staffing approach without such an analysis had the effect of possibly giving KBR an improper competitive advantage since, obviously, KBR’s staffing automatically would be deemed acceptable, while ITT’s would not. We note as well that the agency performed this recalculation of the government standard after receiving FPRs. Thus, in addition to the standard being problematic in and of itself, ITT was not afforded the opportunity to align its proposed staffing with the revised LSA standard for program management. (ITT Federal Services International Corporation, B-296783; B-296783.3, October 11, 2005) (pdf)

KBR asserts with regard to the contingency plan factor that the agency overlooked a number of strengths, and assessed a number of weaknesses that were unreasonable. The agency responded in cursory fashion that KBR’s proposal was “more general” and provided “limited details,” and contended that the protester’s arguments reflect only “mere disagreement” with the agency. AR at 17; SAR at 9. Our review of the record shows more than “mere disagreement.” In its protest filing, KBR provided citations to its proposal showing where in KBR’s proposal the firm addressed each of the areas identified in the RFP for this factor, and showing where it addressed items the agency stated were weaknesses. KBR also made detailed arguments why its proposal was deserving of strengths or significant strengths, including pointing out where other offerors received similar strengths. See, e.g., KBR’s Protest at 14-23. The agency has failed to respond to these specific allegations. Moreover, the TEB report contains only one short statement that “KBR’s only support for their [contingency] response plan was to reference past projects and provided a general overview of their plans for contingency response,” and identified the following unelaborated weakness in KBR’s proposal under the contingency plan factor:

Very limited detail provided on existing resources and how these resources will be coordinated. Contractor mainly used past contracts to show responses. Very limited detail provided for subcontractor agreements, pre-positioned materials, equipment or people. AR, Tab 21, TEB Report, at 74-75.

Our review of the record does not show that KBR’s contingency plan is more limited or general than the awardees’ plans. Although URS proposed a plan that was approximately twice as long as the other offerors’, KBR, Fluor, and ACC all provided contingency plans of similar length. From our review, it is not apparent that the level of detail in any of the plans is significantly different. Notwithstanding KBR’s specific protest contentions, the agency has failed to provide any specific examples of where the plans are dissimilar, and none are apparent from the record. All offerors appear to have addressed the requirements of the RFP and all offerors cited past projects as examples to demonstrate how their plan would successfully be implemented. Although KBR perhaps cited a few more examples than the other offerors, the agency has not explained why this is a weakness and not a strength, given that the examples appear to demonstrate that the proposed contingency plan has been implemented successfully. Given that the record does not, on its face, support the agency’s ratings, and the agency has otherwise failed to explain the difference in ratings, we sustain the protest on this ground. (Kellogg Brown & Root Services, Inc., B-298694; B-298694.2; B-298694.3, November 16, 2006) (pdf)

Regarding Honeywell's proposed staffing, the SEB identified nine "strengths" in Honeywell's proposal and one "weakness" in evaluating Honeywell's proposal under the technical performance subfactor paragraph relating to the staffing plans." Among the strengths noted by the SEB were "Honeywell has provided [DELETED]." AR, Tab 14, Honeywell's Proposal's Strengths and Weaknesses, at 7. Additionally, the record reflects that, based upon its review of two exhibits in Honeywell's proposal pertaining to its proposed staffing plan (detailing Honeywell's staffing by, among other things, PWS section, skill category, number of personnel, and minimum qualification standards proposed), the SEB found that one exhibit "demonstrates that Honeywell has thought through the critical skill mix required to operate the [centers]," and that the other exhibit "further demonstrates a strong knowledge of the qualification standards required by each permanent member of the [contract work team]." Id. As mentioned previously, Honeywell's proposal received an overall rating of "excellent" under the technical performance subfactor. AR, Tab 14, SSA Initial Briefing, at 40. However, during its cost realism evaluation of Honeywell's proposal, the same SEB found that the proposal had "limited supporting rationale for being 45 FTE's below the Government estimate from day one." AR, Tab 43, Honeywell Cost Realism Rationale, at 1. The contemporaneous documentation of the agency's cost realism evaluation of Honeywell's proposal with regard to proposed staffing is two pages long, with only one page actually addressing the agency's "rationale" for the adjustments made. On this page, the agency identified each of the specific skill categories where adjustments were found to be necessary in the proposed staffing, typically concluding in each case, with little further elaboration, that Honeywell had not provided "sufficient rationale" for its proposed staffing. This evaluation resulted in a significant increase in the evaluated most probable cost of Honeywell's proposal, as well as a significant deduction from the proposal's mission suitability score, such that Honeywell's proposal score fell from the "excellent" range to the "very good" range. The contemporaneous record, however, does not discuss or attempt to reconcile the SEB's conclusions in the cost evaluation, where Honeywell's staffing plan was deemed "inadequate" by 43 FTEs, with the evaluation of the staffing plan under the technical performance subfactor, where Honeywell's plan was found to contain "strengths" and an "appropriate" number of proposed personnel. Thus, the contemporaneous record provides no basis to conclude whether the SEB's evaluation of Honeywell's proposed staffing under the technical performance subfactor or with regard to cost realism, or either, was reasonable. (Honeywell Technology Solutions, Inc.; Wyle Laboratories, Inc., B-292354; B-292388, September 2, 2003) (pdf)

It is apparent from the foregoing that the TEB here re-rated CMR's performance under the various projects using different questions and a different rating scale than those used by the evaluators who conducted the reference checks. Given the difference between the rating scales used and the questions asked, there is no way that we can be certain that the ratings on the forms completed by the TEB here accurately reflect the opinions expressed by the individuals contacted. For example, there is no evidence that any of the references contacted expressed an opinion as to whether CMR's performance conformed to its quality control plan, yet the TEB rated CMR's performance as having met the standard (but not exceeded it) with regard to all four projects. Further, there is no evidence that it was reasonable for the TEB to translate the references' original ratings (of excellent/good/satisfactory/etc.) into virtually across-the board ratings of "met the standard." In this regard, the agency has furnished us with neither the original rating sheets, nor with an explanation as to how it translated the scores. For example, it has not been explained (and it is unclear to us) whether ratings of "met the standard" reflect original ratings of satisfactory (as would seem appropriate) or original ratings of good (pursuant to the reasoning that "met the standard" was the second highest rating under the original scale, whereas "good" was the second highest under the scale used by the TEB). Without such information, we have no basis upon which to conclude that the agency's evaluation of the protester's past performance was reasonable. Further, as noted above, the record reveals that Pizzarotti's past performance was initially rated using a different, more liberal rating scale than either of those described above. According to the statement of the individual who conducted Pizzarotti's reference checks pursuant to an earlier solicitation, he explained to the references whom he contacted that they were to rate Pizzarotti's performance in accordance with the following scale: met or exceeded the standard, close to the standard, departure from the standard with increased risk, departure from the standard with significant risk, unacceptable departure from the standard. This scale differs from those used to rate CMR's past performance in that it collapses the top two (in the case of the exceeded the standard/met the standard/etc. scale) or possibly three ratings (in the case of the excellent/good/satisfactory/marginal/poor scale) into the single top category of met or exceeded the standard. It would obviously have been easier for Pizzarotti to attain top ratings under this scale than for CMR to attain top ratings using either of the others. In our view, the use of these different rating scales calls into question the even-handedness of the evaluation of CMR and Pizzarotti's past performance. (Cooperativa Muratori Riuniti, B-294980; B-294980.2, January 21, 2005) (pdf)

Since an offeror is not bound to provide goods or perform services that it has not offered to provide or perform, agencies may properly evaluate proposals only on the basis of the information and representations contained therein. See, e.g., Microcosm, Inc., B-277326 et al., Sept. 30, 1997, 97-2 CPD ¶ 133 at 6-7. Here, as shown above, the agency credited ATK’s proposal with meeting the required level of counter‑countermeasure performance--even though ATK’s proposal did not address the [deleted] necessary to achieve the specified level of performance. Rather, the agency’s determination that ATK’s proposal “include[d] a full description of design feature[s] that enable stated performance” was based on an agency advisor’s perception of the knowledge and capabilities of an ATK subcontractor. Since ATK’s proposal did not address the [deleted], it is not clear that all of the costs associated with ATK’s proposed [deleted] were included in its proposal, nor that ATK, or its subcontractor, is bound to provide the [deleted] the agency assumed to be part of ATK’s proposal. Based on this record, the agency’s evaluation of ATK’s proposal lacks a reasonable basis.  (Lockheed Martin Corporation, B-293679; B-293679.2; B-293679.3, May 27, 2004) (pdf)

In sum, because the agency essentially ignored Locus’s revised proposal and failed to consider price in the evaluation, we sustain the protest. (Locus Technology, Inc., B-293012, January 16, 2004)  (pdf)

The task of gathering data regarding the offerors' past performance was assigned to a TEP coordinator . . . . He gathered past performance evaluations from the offerors' contract references and recorded the information onto worksheets . . . . The worksheets identifies six separate aspects of past performance to be evaluated . . . and utilizes six separate adjectival ratings -- Outstanding, Excellent, Good, Fair, Poor and N/A [not applicable]. The [TEP coordinator] recorded the references' evaluations . . . [and] then transcribed this data to a summary matrix, recording a score of “5” for each “Outstanding” rating, a score of “4” for each “Excellent” rating, etc. . . . The rating provided by two of Shaw's references . . . for “Overall Performance” were incorrectly recorded on the summary matrix as [deleted] rather than [deleted]. . . . The [TEP coordinator] then utilized these incorrect scores to arrive at an average score for Shaw's overall performance. 

In addition to the evaluation errors discussed above--correction of which would decrease the margin of Shaw's total point score superiority from [deleted] to [deleted]--ManTech's supplemental protest identifies other apparent errors, including another apparent transcription error relating to Shaw's rating under the most heavily weighted evaluation factor, personnel. Specifically, ManTech points out that the final evaluation record incorporates an evaluator's rating of [deleted] for Shaw's proposed analytical chemist/manager, but the contemporaneous evaluator worksheets reflect a score of either [deleted] or [deleted]. Accordingly, ManTech maintains that Shaw's evaluated advantage with regard to proposed personnel was improperly inflated due to transcription errors in much the same way the agency acknowledges occurred with regard to Shaw's past performance rating. As discussed above, it is clear that, even considering only the past performance evaluation errors, which the agency acknowledges, the initial cost/technical tradeoff, on which the source selection decision was based, reflected consideration of a technical point score superiority for Shaw that was more than double the margin that existed following correction of the agency errors. Correction of these errors, alone, eliminates Shaw's evaluated advantage under the second most important evaluation factor--past performance--which, in part, formed the basis for the agency's conclusion that Shaw's technical superiority outweighed ManTech's [deleted] cost advantage. Further, as discussed above, the contemporaneous evaluation record regarding the most important evaluation factor--personnel--suggests that Shaw's slightly higher point score under that factor may have been, similarly, inaccurately recorded. In short, it appears that the evaluation record supporting Shaw's evaluated technical superiority, including the total point score ratings on which the initial source selection decision appears to significantly rely, was materially flawed. (ManTech Environmental Research Services Corporation, B-292602; B-292602.2, October 21, 2003) (pdf)

As discussed, the record reflects numerous instances where the agency either unreasonably reached conclusions relating to the EDS offer in light of the language included in the proposal (for example, in the areas of single sign-on access and the provision of Oracle database software), or apparently failed to thoroughly evaluate the proposals critically, and in a manner that would have revealed inconsistencies or deficiencies in what was being offered (for example, in the remote access, IMAC and small business areas). We find the agency’s conclusions troubling in light of its evaluation of the LMIS proposal. For example, one of the discriminators relied on by the agency to make award to EDS was its conclusion that the LMIS proposal did not actually provide mainframe and distributed systems operating system or database software. AR, exh. 30, at v, 36; AR, exh. 32, at 2. The agency reached this finding based on the wording of the software support core function narrative of the LMIS PWS, concluding that the proposal was “carefully worded to exclude the actual provision of software.” AR, exh. 30, at v. LMIS vigorously contests the agency’s finding in this regard, maintaining that other sections of its PWS, as well as the list of products contained in its CWBS made clear that the firm had in fact offered the software in question. We observe that, in reading the two proposals, the record shows that the agency seems to have applied a double standard. On the one hand, when reading the EDS proposal, the agency tended to be expansive, resolving doubt in favor of EDS (which, as we have found, led it to reach conclusions not warranted by the actual language of the firm’s offer). On the other hand, when reading the LMIS proposal, the agency applied a more exacting standard, requiring an affirmative representation within the four corners of each section of the PWS before it was prepared to conclude that one or another requirement was being met. We need not resolve which standard should have been applied in the agency’s evaluation but, to the extent that HUD essentially applied a more exacting standard in reviewing one proposal than it did in reviewing the other proposal, this was improper.  (Lockheed Martin Information Systems, B-292836; B-292836.2; B-292836.3; B-292836.4, December 18, 2003) (pdf)

To argue that the agency's failure to meaningfully evaluate a material part of Orenda's proposal is excusable on the basis that the plan's “process itself is negotiable” and will be considered and negotiated after award ignores the contracting by negotiation process set forth in part 15 of the FAR. That process generally provides that in negotiated acquisitions, an agency is to evaluate proposals in accordance with the evaluation factors set forth in the solicitation, document its evaluation of proposals, if appropriate negotiate with offerors through the conduct of discussions and allow offerors to revise their proposals, and select a proposal for award based upon the selection criteria set forth in the solicitation. FAR §§ 15.304-15.308. The negotiated acquisition process as described in the FAR and set forth in this RFP does not contemplate the selection of an inadequately evaluated proposal for award with the intent to negotiate the process by which the offeror will accomplish the agency's requirements at some later point in time. See Global Assocs. Ltd., B-271693, B‑271693.2, Aug. 2, 1996, 96-2 CPD ¶ 100 at 5 (protest sustained where the awardee's proposal was at best unclear as to its compliance with a material requirement of the solicitation, and the agency's post-award communications with the awardee to clarify the awardee's proposal in this regard constituted improper post-best and final offer discussions). Accordingly, we sustain Sabreliner's protest on the basis that the agency's evaluation of Orenda's proposal as acceptable under the transition subfactor to the technical evaluation factor was not reasonably based.  (Sabreliner Corporation, B-290515; B-290515.2; B-290515.3, August 21, 2002)  (txt version)

Failure of individual evaluators to comment upon proposals under each evaluation factor in evaluation worksheets does not render evaluation flawed--there is no general requirement for such all-inclusive comments--particularly where consensus evaluation sheet contained references to proposal advantages and disadvantages, as well as consensus scores.  (North State Resources, Inc., B-282140, June 7, 1999)

Specifically, the protester complains that two original TEB members assigned its proposal three points, the maximum possible score, while, without explanation from the original or reconvened TEB, two others assigned it zero points.  This argument is without merit. First, disparate scoring among evaluators by itself does not suffice to establish an improper evaluation. Unisys Corp., B-232634, Jan. 25, 1989, 89-1 CPD para. 75 at 6.  (General Security Services Corporation, B-280388; B-280388.2, September 25, 1998)