The evaluation of an offeror's past performance, including the agency's determination of the relevance of an offeror's performance history and the weight to be assigned to a subcontractor's past performance, is a matter of agency discretion, which we will not find improper unless it is inconsistent with the solicitation's evaluation criteria. CLS Worldwide Support Servs., LLC, B-405298.2 et al., Sept. 11, 2012, 2012 CPD ¶ 257 at 15. The evaluation of past performance, by its very nature, is subjective and we will not substitute our judgment for reasonably based evaluation ratings; an offeror's disagreement with an agency's evaluation judgments does not demonstrate that those judgments are unreasonable. FN Mfg., LLC, B-402059.4, B-402059.5, Mar. 22, 2010, 2010 CPD ¶ 104 at 7.

Here, we have no basis to question the agency's evaluation of VNT's past performance. VNT identified in its proposal five past performance references--three that stemmed from its own performance and two from its proposed subcontractor. AR, exh. 3, VNT Proposal, at 0073-77. With respect to its three projects, one--involving safety attendants at schools--was reasonably deemed not relevant; the other two projects--involving ambulette services--were considered relevant. AR, exh. 6, SSD, at 0230-31. Only one past performance questionnaire (PPQ) was returned for VNT's projects, which was for its performance on the predecessor contract to transport patients for the SAVAHCS. AR, exh. 3, VNT Proposal, at 0083-85.

In the PPQ, the contracting officer documented numerous concerns with respect to VNT's performance. He highlighted problems with the effectiveness of VNT's management, including "several turnovers at the manager position," and the failure to pay subcontractors on time. Id. at 0084. He described VNT's quality as satisfactory, and noted that an 80-year old patient was injured because VNT had not followed procedures. Id. Other concerns focused on timeliness, quality control, and overall customer satisfaction, which was merely satisfactory. Id. at 0084-85. In addition, the contracting officer pointed to a fine imposed by the Department of Labor due to VNT's failure to pay its employees in accord with the Service Contract Act. Id. at 0085. In the end, he explained that VNT "does not understand working with the federal government and the laws and regulations that go along with that," and has "no concept of how to conform to a contract's terms and conditions." Id. The contracting officer wrote that he would not award the company another contract.

The contacting officer also considered additional information obtained from procurement files and his personal knowledge, specifically regarding VNT's performance on the predecessor contract. The contracting officer discussed in detail VNT's "failure to perform the services under the contract," which resulted in the VA terminating VNT's contract for cause in June 2017. AR, exh. 6, SSD, at 0234. The contracting officer documented numerous examples of problems the agency experienced during VNT's performance, including an injured patient, failure to report an accident, and failure to meet transportation timelines. Id. According to the past performance record, VNT was given multiple letters of concern, a cure notice, and a show cause notice. Id. Ultimately, VNT "abruptly ceased performance," which resulted in "numerous patients . . . stranded at the hospital and up to 100 patients per day . . . in jeopardy of missing their scheduled appointments." Id. Hence, the contract was terminated for default.

The agency also took into account the past performance record of VNT's proposed subcontractor. Id. at 0234-35. While the subcontractor's record showed generally positive past performance, the agency nevertheless assigned little weight to the subcontractor's past performance. See id. at 0235-36. In this regard, the contracting officer explained that VNT failed to describe in its proposal the percentage of services that the subcontractor would provide. Id. at 0236. In addition, the contracting officer noted that VNT previously failed to timely pay its subcontractors, "raising a question of how long the proposed subcontractor might actually perform work on this contract." Id. Based on the totality of the past performance record, including the consideration of somewhat positive PPIRS information, the contracting officer concluded that there was a "low expectation that the offeror will successfully perform the required effort." Id.

Here, in light of the record described above, we have no basis to question the agency's limited confidence in VNT's performance. In addition, contrary to VNT's suggestions otherwise, the agency was under no obligation to afford greater weight to the subcontractor's performance record. In this respect, the significance of, and the weight to be assigned to, a subcontractor's past performance is a matter of contracting agency discretion. See Diversified Tech. & Servs. of Va., Inc., B-412090.2, B-412090.3, Dec. 16, 2015, 2015 CPD ¶ 34 at 6-7. Given that VNT failed to disclose what percentage of the effort its proposed subcontractor was to perform, the agency's consideration of VNT's subcontractor's performance record here is unobjectionable.

Moreover, we see nothing improper with the contracting officer's consideration of VNT's poor performance on the predecessor effort. In this respect, as discussed above, the RFP expressly provided that the agency could consider "other relevant information." RFP at 0148. Indeed, the agency may have had an obligation to take this information into account, given that the poor performance was on a contract for the same services for the same geographic location with the same contracting office. See, e.g., International Bus. Sys., Inc., B-275554, Mar. 3, 1997, 97-1 CPD ¶ 114 at 5; TRW, Inc., B-282162, B-282162.2, June 9, 1999, 99-2 CPD ¶ 12 at 5.

Lastly, and equally unavailing, is VNT's dubious assertion that the firm's past performance rating is the result of agency bias, which, according to VNT, is substantiated by the comprehensiveness of the documentation in the record. See Comments/Supp. Protest at 8. As we recently explained to VNT, government officials are presumed to act in good faith, and a protester's contention that contracting officials are motivated by bias or bad faith must be supported by convincing proof; we will not attribute unfair or prejudicial motives to procurement officials on the basis of inference or supposition. See Veteran Nat'l Transp., LLC, B-415011, Oct. 31, 2017, 2018 CPD ¶ 51 at 6; Career Innovations, LLC, B-404377.4, May 24, 2011, 2011 CPD ¶ 111 at 7-8. Here, the record is devoid of any support for VNT's bias claim. Indeed, rather than agency bias, VNT's poor performance on the incumbent effort is what led to its less than favorable past performance assessment.  (Veteran National Transportation, LLC B-415696.2, B-415696.3: Apr 16, 2018)

The protester first alleges that the agency improperly attempted to contact only one of its eight past performance references--the one that responded positively following the agency’s corrective action. Protester’s Comments at 1-3. In support of this contention, GMCS points to the eight emails it provided with its initial protest that were from the past performance references identified in its proposal. GMCS also provides follow up emails from four of its past performance references in its comments on the agency’s report. In these emails, the references continue to state that they were not contacted by the agency. Protester’s Comments at 16-23.

GMCS also disputes the agency’s initial report of a negative reference from one of the eight references identified in its proposal. GMCS argues that this individual “has maintained both in writing and by phone that he never communicated with the Agency or provided any feedback on GMCS’ performance to the Agency.” Protester’s Comments at 3. As a result, the protester argues that the agency’s personnel have committed “fraud by suggesting that they have contacted any of our past performance references, except [for the one] noted above.” Protester’s Comments at 3.

In response, the agency has provided an affidavit from the agency official who contacted the protester’s references, the official’s notes from her phone calls, and the underlying emails sent from the official to the protester’s references. Declaration of Quality and Safety Manager, OAS, Dec. 4, 2017; AR, Tab 12, Agency Past Performance Notes. With regard to the individual the protester asserts was never contacted--that is, the individual that the agency claimed provided the negative review during the initial evaluation--the agency official explains that during the initial evaluation she emailed this individual, and subsequently spoke by phone with this individual. During this phone call, the individual provided a negative past performance review, which the agency official documented in her contemporaneous notes. AR, Tab 12, Agency Past Performance Notes, at 4. The agency explains that all subsequent attempts (following corrective action) to contact this individual were unsuccessful. While the protester asserts that the agency has misrepresented that it contacted this individual, and points to the email from its initial protest in which this person stated that no one from the agency had contacted him, we find that the agency has sufficiently demonstrated that it did, in fact, contact this individual, and find nothing improper about the agency’s reliance on the negative past performance information it received. With regard to the second individual who the protester asserts was never contacted, the agency official explains that she attempted to email this individual, and never received a response. However, the record shows that the email address incorrectly spelled this individual’s name, and thus it appears that the agency’s attempt to email this individual was unsuccessful. Declaration of Quality and Safety Manager, OAS, Dec. 4, 2017, Email Attachment.

The record here shows that the agency either contacted, or attempted to contact, three of the eight past performance references provided by GMCS in its proposal. In addition, the record shows that the agency reached one of these references prior to the agency’s corrective action. AR, Tab 19, Agency Past Performance Memorandum, at 1‑2. Following the agency’s corrective action, the agency attempted to contact two additional organizations identified by GMCS in its proposal as past performance references. The record shows that the agency reached one of these references and recorded its input. Id. at 2‑3. Given this, we find the record does not support the protester’s contention that the agency failed to contact any of its past performance references GMCS provided. To the extent the crux of the protester’s complaint is that the agency’s attempts to contact its references were inadequate, such that the agency was required to contact the other past performance references the protester had provided, there is no requirement that an agency contact all of an offeror’s references. See OMV Med., Inc.; Saratoga Med. Ctr., Inc., B‑281387 et al., Feb. 3, 1999, 99‑1 CPD ¶ 52 at 4.  (Government and Military Certification Systems, Inc. B-414740.5: Dec 19, 2017)

DynCorp objects to the three marginal ratings, arguing that the ratings are not supported by the record and are inconsistent with the CPARs. Protest at 15-22. DynCorp further contends that its overall satisfactory confidence rating is irrational because it is based on the agency's "selective reliance on one-off comments" made in the narrative portions of the CPARs or during the past performance interviews. Comments/Supp. Protest at 7.

Here, we find reasonable the agency's evaluation of DynCorp's past performance. In this respect, the PPET's [past performance evaluation team] determination that DynCorp's past performance record included three adverse performance issues is unobjectionable. As noted above, the RFP defined adverse as "past performance information that supports a less than satisfactory rating on any evaluation element or any unfavorable comment received from sources without a formal rating system." RFP at 333. In this respect, contrary to DynCorp's arguments, the CPAR ratings alone did not control whether an offeror's past performance record included adverse issues. Instead, the record reflects that the evaluators considered various sources of information and reasonably reached the conclusion that DynCorp had not effectively implemented actions to remedy performance issues described in the CPARs.

(sections deleted)

The evaluation of an offeror's past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. MFM Lamey Group, LLC, B-402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10. Where a solicitation calls for the evaluation of past performance, we will examine the record to ensure that the evaluation was reasonable and consistent with the solicitation's evaluation criteria and procurement statutes and regulations. Divakar Techs., Inc., B-402026, Dec. 2, 2009, 2009 CPD ¶ 247 at 5. In addition, the relative merits of an offeror's past performance information is generally within the broad discretion of the contracting agency. See Paragon Tech. Group, Inc., B-407331, Dec. 18, 2012, 2013 CPD ¶ 11 at 5. A protester's disagreement with the agency's judgment does not establish that an evaluation was unreasonable. FN Mfg., LLC, B-402059.4, B-402059.5, Mar. 22, 2010, 2010 CPD ¶ 104 at 7.

Our review of the record confirms the reasonableness of the PPET's assessment. In this respect, the CPARs demonstrate that while DynCorp was lauded for certain aspects of its performance, supply support-related concerns were documented as well. For instance, in the 2011-2012 CPAR, in addition to the comments cited by the PPET, the CPAR evaluator noted that DynCorp's "weakness lies in the areas of Aircraft Maintenance and Logistical Support." AR, Tab 17, DynCorp PPI for INL/A Contract, at 13. This same weakness regarding logistical support remained in the 2012-2013 CPAR, and the 2013-2014 CPAR highlighted the related concerns cited by PPET, discussed above. Id. at 18, 23. The CPARs did not include other comments relevant to the PPET's assessment under the supply support element or indicate that DynCorp's approach to logistics improved during performance. In fact, the PPET's March 2016 interview with the program COR (contracting officer's representative) discussed DynCorp's "ongoing problem with supply support." Id. at 26. According to the interview record, the COR reported that DynCorp "displayed a negative trend in providing parts and supply for the Beechcraft aircraft that were and are stationed in Afghanistan and Iraq." Id. The COR also relayed that the State Department created a position to specifically provide oversight and directives to DynCorp. Id. Thus, the record reflects that supply support remained an ongoing problem at the time the PPET conducted its evaluation.

On this record, we find unobjectionable the agency's assessment that the supply support element for DynCorp's INL/A contract reflected adverse performance, as defined in the RFP. While DynCorp disagrees with this determination, primarily contending that the firm never received less than satisfactory ratings on the CPARs, such disagreement does not demonstrate an unreasonable evaluation. See CLS Worldwide Support Servs., LLC, B-405298.2 et al., Sept. 11, 2012, 2012 CPD ¶ 257 at 15-16. Indeed, the entirety of the past performance record, including the narrative explanations in the CPARs and the interviews with INL/A officials, supports the reasonableness of the agency's evaluation.  (DynCorp International LLC B-414647.2,B-414647.3: Nov 1, 2017)

Our Office will question an agency’s past performance evaluation where the record indicates that the agency either failed to evaluate, or otherwise unreasonably considered, the relevance of past performance references in accordance with the solicitation’s stated evaluation criteria. Arctic Slope Mission Servs., LLC, B-410992.5, B-410992.6, Jan. 8, 2016, 2016 CPD ¶ 39 at 12. Additionally, in reviewing an agency’s evaluation of proposals, we consider not only whether the agency deviated from the RFP evaluation criteria, but also whether the actual evaluation was unreasonable. American Dev. Corp., B-251876.4, July 12, 1993, 93-2 CPD ¶ 49 at 10 (finding agency evaluation that “divorced the contract performance rating from the relevancy rating” to be unreasonable).

Based on our review of the record, we conclude that the agency’s past technical performance evaluation was unreasonable. The RFP here provided that the agency would “first evaluate the relevancy of the past performance references submitted in the offerors past technical performance proposal as well as obtained through other means (PPIRS, interviews, etc.)” and would “next evaluate the quality of this past performance through a review of questionnaire responses and information available in CPARS.” RFP at 177. The agency would then calculate the overall confidence rating based on “the amount of experience, relevancy of that experience to the current requirement and quality of [the] offeror’s past performance.” Id. Of note, the solicitation defined a substantial confidence rating as “[b]ased on the offeror’s recent/relevant performance record, the Government has a high expectation that the offeror will successfully perform the required effort.” Id. Clearly, the RFP envisioned that the agency would make both relevancy and quality determinations for the past performances references submitted in USMMI’s proposal, and that the agency’s overall confidence assessment rating would be an integrated assessment based on these determinations.

Here, however, the agency failed to make the dual determination envisioned by the RFP. Instead, the agency credited the orphan questionnaire ratings without examining the relevance of such ratings, and conversely credited the orphan relevance determinations without receiving information about the quality of USMMI’s performance. All of these relevancy and quality ratings were then included in a general range of scores, and the agency determined subelement confidence ratings based on whether the range included high relevance or quality scores. The net effect of this was to distort the agency’s evaluation because it added quality ratings of unknown relevance and relevancy determinations of unknown quality. This was inconsistent with the evaluation scheme, which anticipated that the agency would assess both the relevance and the quality of each past performance reference, and that the confidence assessment would examine the offeror’s recent/relevant performance to predict whether the offeror could successfully perform the required effort.

For instance, with regard to the subelement relating to lifecycle management experience providing maintenance and repair,[4] the agency’s evaluation was as follows:

2. Lifecycle Management
a. Experience, in the last five years, in executing lifecycle management programs to ensure:
i. Adequate and timely maintenance and repair.

USMMI provided eight contracts as evidence of its experience for element 2(a)(i). USMMI demonstrated [it] has Relevant and Very Relevant experience for element 2(a)(i). See attachment A for the relevancy evaluation ratings and summary. Questionnaires were received from seven respondents that addressed element 2(a)(i). USMMI received questionnaire ratings ranging from Satisfactory to Very Good.

Based on an evaluation of the offeror’s recent/relevant performance record for this element, USMMI received a rating of SUBSTANTIAL CONFIDENCE for this element as the Government has a high expectation that the offeror will successfully perform the required effort.

AR, Tab 8, Past Technical Performance Report, at AR0235. However, of the eight contracts noted above that were evaluated for relevancy, the agency only received three questionnaire scores. See AR, Tab 10, Past Technical Performance Report, at AR0248. Similarly, of the seven questionnaires received by the agency,[5] the agency only evaluated the relevance of three of the contracts. Compare id. with AR, Tab 9, Past Technical Performance Report, at AR0243. Removing these orphans, as well as the undocumented contract reference noted below, from the agency’s assessment, would result in only two contract references where the agency possessed both relevance and questionnaire ratings. Both of these references were found to be relevant and satisfactory. Accordingly, had the agency’s evaluation considered only these two contracts, rather than the numerous references quoted above, the agency may have reached a different conclusion with regard to its subelement confidence assessment.[6]

TOTE next argues that the agency improperly determined that the only submitted contract on which USMMI was the prime contractor, the [DELETED] contract, was very relevant under five of the past technical performance subelements.[7] In this regard, the solicitation defined “very relevant” under each subelement as having the same experience as that called for under the solicitation’s scope of work. For instance, a determination of very relevant under subelement 2(a)(i) was defined as follows:

RFP, Attach. N. For each of the five subelements in question, USMMI’s proposal narrative submitted the [DELETED] contract for each subelement but did not specifically explain how the contract met the specific relevancy criteria. See AR, Tab 11, USMMI Vol. 2, at AR0264-276. For instance, for subelement 2(a)(i), USMMI’s proposal narrative contained a general discussion of its maintenance experience, but did not contain any explanation of how the [DELETED] contract involved maintenance and repair on systems that were the same as those on the SBX-1. See id. at AR0269.

In response to this argument, the agency concedes that three of the subelement relevancy ratings were incorrect and should have been rated “relevant” rather than “very relevant,” though the agency contends these errors were harmless.[8] The agency further asserts that the five subelement relevancy determinations were made based on the personal knowledge of the agency’s evaluators, who had general and direct knowledge of the [DELETED] contract and were therefore familiar with its systems. Supp. MOL at 9.

While, as a general matter, the evaluation of an offeror’s past performance is within the discretion of the contracting agency, we will question an agency’s evaluation of past performance where it is unreasonable or undocumented. Solers, Inc., B-404032.3, B-404032.4, Apr. 6, 2011, 2011 CPD ¶ 83 at 8. Although an agency is not required to retain every document generated during its evaluation of proposals, the agency’s evaluation must be sufficiently documented to allow our Office to review the merits of a protest. Apptis, Inc., B-299457 et al., May 23, 2007, 2008 CPD ¶ 49 at 10. Where an agency fails to document or retain evaluation materials, it bears the risk that there may not be an adequate supporting rationale in the record for us to conclude that the agency had a reasonable basis for its source selection decision. TriCenturion, Inc.; SafeGuard Services, LLC, B-406032 et al., Jan. 25, 2012, 2012 CPD ¶ 52 at 13.

Here, we find the agency’s evaluation was not sufficiently documented to allow us to determine whether the agency’s conclusions were reasonable. In this regard, while the agency asserted that its evaluators relied on their personal knowledge of the [DELETED] contract, this assertion is not supported in the contemporaneous record and does not explain the agency’s admitted mistake with respect to three of the five subelement relevancy determinations. Moreover, for the five subelements in question, the contemporaneous record does not contain any information regarding the systems in place on the [DELETED] vessels, or how such systems compare to the corresponding system on the SBX-1 vessel.

In addition, the agency’s explanation that it relied on its evaluators’ knowledge is inconsistent with the past technical performance evaluation report. The report points to USMMI’s proposal narrative as the source of the agency’s relevancy determinations; the report does not state that such relevancy determinations were made based on the evaluators’ personal knowledge of the contract. See AR, Tab 9, Past Technical Performance Chart, at AR0242-245. Accordingly, we find that the contemporaneous record does not contain sufficient documentation for us to determine the reasonableness of the agency’s relevancy determinations with respect to the five subelements in question.  (TOTE Services, Inc. B-414295, B-414295.2: Apr 25, 2017)

Past Performance

The protesters challenge the agency’s past performance evaluation in many respects. As set forth below, our Office identified a number of concerns with the agency’s evaluation of proposals under this factor, and therefore sustain several of the protests challenging the agency’s evaluation. Some of the errors appear to have impacted the evaluation of several proposals. In such circumstances, we address below the nature of the concern and highlight representative examples. In other instances, the errors appear to have been unique to individual proposals.

As a general matter, we will review an agency’s past performance evaluation to determine whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation’s evaluation scheme. Logistics Mgmt. Int’l, Inc., et al., B-411015.4 et al., Nov. 20, 2015, 2015 CPD ¶ 356 at 8. We will question an agency’s evaluation conclusions where they are unreasonable or undocumented. OSI Collection Servs., Inc., B-286597, B-286597.2, Jan. 17, 2001, 2001 CPD ¶ 18 at 22. On the other hand, a protester’s disagreement with the agency’s judgment, without more, is insufficient to establish that an evaluation was improper. Beretta USA Corp., B‑406376.2, B-406376.3, July 12, 2013, 2013 CPD ¶ 186 at 10. Additionally, it is fundamental that a contracting agency must treat all offerors equally, and therefore it must evaluate offers evenhandedly against common requirements and evaluation criteria. Logistics Mgmt. Int’l, Inc., et al., supra, at 16.

Evaluation of Past Performance Information

The agency appears in several circumstances to have unreasonably either ignored or discounted relevant information bearing on the quality of offerors’ past performance if it was not included in a CPARS report. For example, DMA identified three student loan collection references, its incumbent contract (for which the agency has a CPAR report) and two contracts with commercial entities. The TEC rated all the references as highly relevant. Under the quality subfactor, however, the agency appears to have only considered the incumbent contract. The TEC noted that DMA had received a satisfactory quality rating on its most recent CPARS report, and “compared favorably” against other incumbents due to its low error rate under the agency’s 2015 focused review for regulatory compliance. Based on this one reference, the TEC concluded that “[t]he satisfactory past performance rating received [on the incumbent contract] demonstrates that [DMA] performed at an acceptable level and that there is high confidence that the offeror will successfully perform the PWS requirements as outlined in the solicitation with minimum risk.” Consolidated AR, Tab 68, DMA TEC Report, at 1-2.

Absent from the TEC’s analysis, however, is the agency’s consideration of the quality of DMA’s performance on its other two highly relevant contracts. The contemporaneous record (and the agency’s response to the protest) is devoid of any indication that the agency considered the detailed information in DMA’s proposal regarding these past performance references, or made any effort to contact DMA’s references to inquire about the quality of the protester’s performance. In this regard, we note that DMA’s proposal includes detailed information supporting the quality of its performance on the contracts. See, e.g., Consolidated AR, Tab 29, DMA Proposal – § B, at 33 (noting the customer’s prior “exceptional” rating on another DOE past performance questionnaire), 37 (including a recommendation letter for the second contract).

As another example, with respect to TGSL, the firm identified, as one of its three past performance references, its performance as a subcontractor in connection with the incumbent requirements. With respect to this reference, the TEC found that “TG[SL] did not provide indicators of performance quality.” TGSL AR, Tab P, TGSL TEC Report, at 2. The record, however, does not support this evaluation finding. Rather, the record reflects that the agency had past performance information from the prime incumbent contractor concerning TGSL’s subcontractor performance. Specifically, the prime contractor advised that: “TG[SL] continues to perform at a high level while maintaining thorough compliance and zero complaints for the pool of accounts they are servicing . . . [we are] extremely happy with TG[SL]’s service, training, and overall collections as a current subcontractor on our Task Order.” TGSL AR, Tab O, Email from TGSL Past Performance Reference, at 1-2. We have previously explained that an agency’s past performance evaluation is unreasonable where the agency fails to give meaningful consideration to available relevant past performance information. See, e.g., Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, JV, B-401679.4 et al., Mar. 10, 2010, 2010 CPD ¶ 77 at 8 (sustaining protests where agency failed to reasonably consider past performance questionnaires); DRS C3 Sys., LLC, B‑310825, B‑310825.2, Feb. 26, 2008, 2008 CPD ¶ 103 at 26 (same regarding a CPARS report).

This pattern by the agency of neglecting to consider, or unreasonably discounting, relevant past performance information, based on what appears to be the agency’s fixation on CPARS reports to the exclusion of other past performance information, is further highlighted by the agency’s evaluation of the past performance of VRH, one of the awardees. VRH submitted three past performance references; one was for performance of the incumbent contract as a subcontractor, and the other two were student loan debt collection contracts with commercial entities. See GRC AR, Tab Q, VRH TEC Report, at 1. Although the TEC evaluated VRH’s two commercial contracts as “highly relevant,” it did not consider the quality of VRH’s performance on these contracts. Instead, the TEC focused on CPARS reports for contract references not identified by VRH. Specifically, it focused on VRH’s performance of a contract with the Department of Energy, which was for professional support services, including financial analysis and the development of policy recommendations, relating to the “evaluation and award of direct loans and loan guarantees to applicants seeking funding for the commercialization of technologies in the automotive, renewable and conventional energy fields”. GRC AR, Tab M, VRH Past Performance Information, at 2. The record, however, lacks any analysis of how this Department of Energy reference was relevant to the student loan debt collection activities contemplated under this RFP, or any explanation for why the agency eschewed consideration of the highly relevant references cited by VRH. Here again, it is apparent that the agency unreasonably relied almost exclusively upon CPARS reports in lieu of reasonably considering available past performance information, which the agency itself identified as being highly relevant.  (General Revenue Corporation; Account Control Technology, Inc.; Williams & Fudge, Inc.; Performant Recovery, Inc.; Collection Technology, Inc.; Alltran Education, Inc.; Texas Guaranteed Student Loan Corp.; Van Ru Credit Corporation; Global Receivables Solutions, Inc.; Progressive Financial Services, Inc.; Automated Collection Services, Inc.; Gatestone & Co. International, Inc.; Sutherland Global Services; Delta Management Associates, Inc.; Allied Interstate LLC; Collecto, Inc. d/b/a EOS CCA, B-414220.2: Mar 27, 2017)

Evaluation of Crowley’s Past Performance

XPO alleges that the agency’s evaluation of Crowley’s past performance was flawed because Crowley’s past efforts were substantially smaller in magnitude than the effort required under the solicitation. Comments at 15-22; Supp. Comments at 27-40. As related to this allegation, the record reflects that the agency evaluated more than 200 of Crowley’s past efforts. AR, Tab 119, Crowley Past Performance Summary Addendum, at 2-34. Of these efforts, 16 were found somewhat relevant. Id. All of the others were found not relevant. Id. After completing its relevancy evaluation, the agency determined that the 16 somewhat relevant efforts, combined, supported an overall past performance confidence assessment rating of satisfactory confidence. Id. at 35.

XPO argues that the 16 efforts assessed as somewhat relevant were “too miniscule compared to the current procurement to merit such a rating.” Comments at 19. To illustrate, XPO analyzed the relative magnitude of 15 past efforts deemed somewhat relevant by calculating each effort’s value as a percentage of the $3 billion maximum value of the contract awarded under the solicitation. Id. at 20‑21. The calculations show that the value of one Crowley effort was approximately 2 percent of this procurement’s $3 billion maximum value, one effort was approximately 1 percent of the maximum value, and all 13 of the other efforts had values of less than 1 percent of this procurement’s maximum value.[10] Comments at 20-21. Based on this disparity in values, XPO argues that the agency’s assessment of the efforts as somewhat relevant was unreasonable. Id. at 21. XPO further argues that because the efforts should not have been found somewhat relevant, the satisfactory confidence rating assigned to Crowley’s proposal was improper. Id.

In response, the agency argues that XPO’s comparison of the value of Crowley’s efforts to the total value of this procurement is flawed. Supp. AR at 15-16. The proper comparison, according to the agency, begins with a calculation of a “value per year equivalency” for each of Crowley’s past efforts. Id. at 15. This amounts to an average annual value of each effort, calculated by dividing the effort’s total value by the total months of performance, then multiplying that number by 12. Id. at 16. After presenting these calculations, the agency argues that its evaluation was reasonable because the aggregate of the average annual value of 12 of the Crowley efforts exceeds $[DELETED]. Id. The agency chooses $[DELETED] as its benchmark because this figure is the maximum value for the transportation CLIN for the 2-year base period of the contract awarded to Crowley under the solicitation. Id. (citing AR, Tab 122, Crowley Contract, at 3). The agency justifies its choice of this value--rather than the significantly higher total contract value, or any of the significantly higher-valued option periods--on the basis that “the option years only reflect ‘potential’ contract value” if additional military services use the contract. Id. at 17.

As explained in detail below, the agency’s response is problematic for a number of reasons. First, the analysis presented by the agency is not reflected in the contemporaneous record. In addition, the agency’s selection of only the relatively low‑value base period of the contract as awarded to Crowley unreasonably distorts the comparison of the magnitude of Crowley’s past efforts to the magnitude of the solicitation.

Before proceeding further, we observe that as a general matter, the evaluation of an offeror’s past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. See Al Raha Grp. for Tech. Servs., Inc.; Logistics Mgmt. Int’l, Inc., B-411015.2, B‑411015.3, Apr. 22, 2015, 2015 CPD ¶ 134 at 5. However, we will question an agency’s evaluation conclusions where they are unreasonable or undocumented. See id. The critical question is whether the evaluation was conducted fairly,

reasonably, and in accordance with the solicitation’s evaluation scheme. See id. Finally, we have found the assessment of a somewhat relevant rating to an offeror’s past effort to be unreasonable where the solicitation requires the agency to consider the magnitude of the past effort compared to the solicited requirement, and the agency fails to reasonably explain why a past effort involving a minimal amount of the requirement justifies such a rating. See id. at 7-8; Health Net Fed. Servs., LLC, B-401652.3, B-401652.5, Nov. 4, 2009, 2009 CPD ¶ 220 at 17; see also Si-Nor, Inc., B-292748.2 et al., Jan. 7, 2004, 2004 CPD ¶ 10 at 17 (low dollar value of awardee’s past effort showed agency’s determination that the effort was similar to the solicited requirement to be unreasonable); Cont’l RPVs, B‑292768.2, B‑292768.3, Dec. 11, 2003, 2004 CPD ¶ 56 at 12 (agency’s conclusion that the awardee’s past effort was similar to the solicited requirement was unreasonable because the past effort involved less than three percent of the solicited requirement).

Returning to the merits of the protest here, we begin with the solicitation’s evaluation criteria for the past performance factor. As stated at the outset, the solicitation provided that the agency would evaluate “the relevancy of each past performance effort” submitted by an offeror. RFP at 53 (emphasis in original). The solicitation defined the relevancy ratings to be assigned to “each past performance effort” as follows:

Id. These definitions show that to warrant a rating of somewhat relevant, an effort must involve “some” of the magnitude of effort required under the solicitation. The definitions further show that if an effort involved “little or none” of the solicitation’s magnitude, a rating of not relevant was appropriate. At issue here is whether the relatively low value of Crowley’s past efforts renders unreasonable the agency’s conclusion that Crowley’s efforts involved “some” of the solicitation’s magnitude.

We begin our analysis by noting that it is not clear from the contemporaneous record how the agency determined that the magnitude of Crowley’s past efforts supported the ratings of somewhat relevant. In this regard, the past performance evaluation report for Crowley shows the following information for most of the past efforts at issue: the number of monthly shipping transactions, the period of performance, and an approximate total value. AR, Tab 119, Crowley Past Performance Summary Addendum, 2-29. However, there is no evidence that the agency calculated an average annual value for the efforts in order to make an “apples-to-apples” comparison of the efforts’ magnitude relative to the magnitude of the solicitation; i.e., there is no evidence that the agency employed the methodology that it has argued to be the appropriate methodology in response to XPO’s protest. There also is nothing in Crowley’s past performance evaluation report to reflect what benchmarks might have been used to assess whether Crowley’s past efforts met the criteria for the various relevancy ratings defined in the solicitation. While such benchmarks might have included the total or average annual dollar value of an effort (or the number of total or monthly shipment transactions under the effort[11]), it is not evident if or how the agency took this information into consideration.

We turn now to the agency’s argument that the evaluation was reasonable because the sum of the average annual value of 12 Crowley past efforts exceeds the $[DELETED] maximum value of the transportation CLIN for the 2‑year base period of the contract awarded to Crowley. The agency’s argument is based on the general proposition that where the period of performance of an offeror’s past effort materially differs from the period of performance in a solicitation, some method of making an “apples-to-apples” comparison of the magnitude of the two efforts is necessary. We agree with the agency that to make such an “apples-to-apples” comparison, it may be appropriate to consider the average annual value of an offeror’s past efforts. Under a solicitation with numerous option years with significantly differing values, such as the one here, the question becomes, what annual value under the solicitation should be used as the basis of comparison--the smaller value at the start of performance, or the significantly higher values anticipated in the later years? Here, the value the agency has chosen is the base period of Crowley’s contract, as awarded. As explained below, given the circumstances of this procurement, we find the agency’s choice unreasonable.

The record reflects that the agency prepared an IGCE that included projections of the value of each period of performance under the solicitation. AR, Tab 3, ICGE, at 4‑7; AR, Tab 4, DFTS Acquisition Strategy Panel Slides, at 4. The table below shows these values, as well as the maximum values for each period of performance in Crowley’s contract, as awarded.

See AR, Tab 4, DFTS Acquisition Strategy Panel Slides, at 4; AR, Tab 122, Crowley Contract, at 3-9. The table shows that [DELETED]. It also shows that the value of each option year is significantly higher than the value of the base period. The IGCE explained that this increase was due to the “potential addition” of military users of the contract. AR, Tab 3, IGCE, at 2. The IGCE explained the basis for including the additional military users as follows:

There is currently a concerted effort for the [Department of Defense] to move away from awarding non-FAR based tenders for freight transportation. As there are no known feasible acquisition alternatives, it seems reasonable that the military services might opt to be included as customers under [the] DFTS [contract].

Id. Thus, the record reflects that the agency anticipated the value of the option years to be significantly higher than the value of the base period--both at the time it prepared the IGCE and when it awarded Crowley’s contract. The record also reflects that the agency included option year pricing in its calculation of the offerors’ total evaluated prices and that the SSA considered the offerors’ total evaluated price in his tradeoff decision. RFP at 54, 653; AR, Tab 120, SSDD, at 5‑6. For all of these reasons, we find the agency’s selection of the significantly lower-valued transportation CLIN for the base period of Crowley’s contract to be an unreasonable benchmark for assessing the relevance of the magnitude of Crowley’s past efforts.

The question of which annual value is selected as a comparative benchmark has a material effect on whether Crowley’s past efforts reasonably can be assessed as somewhat relevant versus not relevant. For instance, if the annual value of the base period of Crowley’s contract is used (as the agency has proposed), the value of 2 of the 12 Crowley past efforts at issue are in the range of 60 to 70 percent of the solicited requirement, 2 are in the range of 25 to 35 percent, 6 hover in the range of 10 to 15 percent, while the final 2 are approximately 5 and 0.5 percent. See Supp. Comments at 37-38. Some of these figures could support assessments of somewhat relevant. As discussed above, however, we conclude that using only the base period of Crowley’s contract is not reasonable under the circumstances here.

One reasonable benchmark under the circumstances here could be the average value of each year of performance (including options) of the contract. When the average value of each year of performance is used as the benchmark, 2 of Crowley’s past efforts are approximately 10 percent of the solicited effort, 2 are in the range of 4 to 6 percent, while the others are approximately 2 percent or less. See Supp. Comments at 35-36. As discussed above, the solicitation’s distinction between the ratings of somewhat relevant and not relevant was whether the effort involved “some” versus “little or none” of the solicitation’s effort. While the 2 efforts involving 10 percent of the requirement’s average annual effort might reasonably be assessed as involving “some” of the effort, the vast majority of Crowley’s efforts involve 6 percent or less of the effort. Under the ratings definitions established by the solicitation, this level of effort appears to be more closely aligned with an assessment of “little or none” of the requirement, which would result in ratings of not relevant.

As established above, it is not clear from the contemporaneous record how the agency concluded that Crowley’s past efforts supported ratings of somewhat relevant. Additionally, the evaluation methodology that the agency has advanced in response to XPO’s protest uses only the value of the base period as a comparative benchmark, despite the agency’s estimates--both at the time of preparing the IGCE and at the time of awarding Crowley’s contract--that the option years would be of significantly higher values. In addition, this approach is inconsistent with the agency’s decision to include option year pricing in the offerors’ total evaluated prices, and to consider the offerors’ total evaluated prices in the tradeoff decision. In sum, the value of Crowley’s past efforts are extremely small relative to the value of the requirement, indicating that the agency’s assessment of ratings of somewhat relevant to these efforts was misplaced. For these reasons, we sustain XPO’s allegation regarding this aspect of the evaluation.

Before concluding, we note XPO also argues that the agency’s evaluation of several of Crowley’s past efforts was unreasonable because these efforts involved very low numbers of shipments per month or, in the case of an effort that was evaluated for the use of special equipment, because the record lacks specific information regarding Crowley’s experience. Comments at 22; Supp. Comments at 39-40. We have considered the agency’s responses to these allegations and we find, based on the record, that these allegations also have merit. However, we do not discuss these allegations in detail because the issues raised necessarily would be addressed through the implementation of our recommendation--discussed below--that the agency reevaluate Crowley’s past performance in a manner that is reasonable and consistent with both the solicitation and this decision.

Finally, we note that XPO argues that it was improper for the agency to aggregate Crowley’s past performance efforts in its determination of Crowley’s overall confidence assessment rating. Comments at 17-19; Supp. Comments at 27-30. We do not reach this issue because the record does not support the agency’s findings that Crowley’s past performance efforts were somewhat relevant, which should be a prerequisite to aggregating the results.  (XPO Logistics Worldwide Government Services, LLC, B-412628.6, B-412628.7: Mar 14, 2017)

CSR protests the agency’s evaluation of its past performance. Specifically, the protester contends that the agency unreasonably failed to consider all available assessment reports when assessing the quality of its past performance. CSR also alleges that the agency’s evaluation of its past performance was disparate from how the agency treated BAH. CSR argues that had the agency performed a proper evaluation, it would have received an “exceptional” past performance rating, equal to that of BAH (on which the agency relied when making its best-value decision). As detailed below, we find the agency’s evaluation of CSR’s past performance was unreasonable because it was unequal with how the agency treated BAH.

In reviewing a protester’s challenge to an agency’s evaluation of vendors’ past performance, our Office does not independently evaluate quotations; rather, we review the agency’s evaluation to ensure that it is reasonable and consistent with the terms of the solicitation and applicable statutes and regulations. Castro & Co., LLC, B-412398, Jan. 29, 2016, 2016 CPD ¶ 52 at 8; SOS Int’l, Ltd., B-402558.3, B-402558.9, June 3, 2010, 2010 CPD ¶ 131 at 2. An agency’s determination of the relevance or merit of a vendor’s performance history is a matter within the discretion of the contracting agency, which we will not disturb unless the agency’s assessments are unreasonable or inconsistent with the solicitation criteria. See Rotech Healthcare, Inc., B-413024 et al., Aug. 17, 2016, 2016 CPD ¶ 225 at 3; Logistics Mgmt. Int’l, Inc., et al., B-411015.4 et al., Nov. 20, 2015, 2015 CPD ¶ 356 at 8. The critical question is whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation’s evaluation scheme. Halbert Constr. Co., Inc., B-413213, Sept. 8, 2016, 2016 CPD ¶ 254 at 8; Al Raha Grp. for Tech. Servs., Inc.; Logistics Mgmt. Int'l, Inc., B-411015.2, B-411015.3, Apr. 22, 2015, 2015 CPD ¶ 134 at 5. An agency’s past performance evaluation is unreasonable where the agency engaged in disparate treatment. Halbert Constr. Co., Inc., supra, at 11.

The RFQ permitted vendors to submit up to nine past performance references, but also limited the length of the past performance submissions as follows: a table not exceeding three pages of required information regarding the vendor’s references; and up to four additional pages detailing the scope and relevancy of the references listed. RFQ at 31-32. The solicitation advised vendors that “the Government may use all data provided by the Offeror in this quote and data obtained from other sources, including, but not limited to, other DOJ and OJP contracts and information from Government repositories . . . .” Id. at 32. The RFQ also established that the agency would assess both the relevance and quality of each vendor’s past performance in order to assess the probability of successful performance. Id.

CSR submitted six past performance references--four for itself and two for proposed subcontractor [DELETED]--using the RFQ-allotted pages. AR, Tab 7, CSR Quotation, Vol. I, Technical, at 373-79. Three of CSR’s references concerned task orders involving PMT services previously performed for DOJ’s OJP. Id. at 373-74.

The TEB, when evaluating CSR’s past performance, assessed the relevance of the vendor’s prior work based on the information provided by CSR in its quotation. AR, Tab 14, TEB Report, at 641. When assessing the quality of CSR’s prior work, the evaluators took into account three Contractor Performance Assessment Reports (CPAR) regarding the OJP task orders that CSR had referenced in its quotation. AR, Tab 9, CSR CPARs, at 573-80. Overall, the TEB found all of CSR’s references to be relevant and the quality of the vendor’s prior work to be mixed, and rated the protester’s past performance as acceptable. AR, Tab 14, TEB Report, at 641.

CSR argues that the agency unreasonably failed to consider other relevant CPARs regarding its past performance, and all involving DOJ. The protester points to two earlier CPARs involving its first referenced OJP task order, as well as a CPARs for another relevant project (i.e., research, evaluation, and performance measurement services) not listed as a reference in its quotation (the three additional CPARs identified by CSR were uniformly of high quality). Protest, exh. 4, CSR CPARs, at 1-11. The protester contends that this additional past performance information was relevant, too “close at hand” for the agency to ignore as part of its evaluation, and would have affected the assigned past performance rating. Supp. Protest, at 9.

The agency asserts that, when retrieving CPARs, it limited its search to only those specific projects that were identified by the vendors in their quotations, and only the most recent CPARs that were available for those projects. COS, Dec. 22, 2016, at 2. As CSR had identified three specific OJP task orders in its quotation, the agency states, the contract specialist pulled only the most recent CPARs for each of those referenced task orders. Id. The agency also states that it did not consider the three other CPARs which CSR points to in its protest because two were older reports for a referenced project, and the other was not considered because it was not among the past performance projects that CSR identified in its quotation. Id.

The record reflects, however, that in conducting its past performance evaluation of BAH, the agency considered CPARs for past performance projects that were not identified in the awardee’s quotation. BAH submitted six past performance references for itself and its proposed subcontractors in its quotation. AR, Tab 4, BAH Quotation, Vol. I, Technical, at 189-95. Relevant to the protest here, one of BAH’s references was for its law enforcement systems analysis (LESA) project for the Department of Homeland Security, Immigration and Customs Enforcement (ICE). Id. at 190. BAH’s quotation also detailed the relevance of its LESA project to the PMT work to be performed here. Id. at 193-94.

The agency considered three CPARs when evaluating BAH’s past performance--two of which concerned projects referenced in the vendor’s quotation. AR, Tab 6, BAH CPARS, at 300-312. The third CPARs, however, involved BAH providing support services to ICE at its Pacific Enforcement Response Center (PERC). Id. at 304-06. The record indicates that BAH’s LESA project and its PERC support services effort were different task orders issued under the same contract with ICE: in addition to involving different types of work, the projects also had different award dates, performance periods, dollar values, performance locations, and contracting officers. Compare AR, Tab 4, BAH Quotation, Vol. I, Technical, at 190, with AR, Tab 6, BAH CPARs, at 304.

The TEB thereafter found all of the past performance references in BAH’s quotation to be relevant, and the CPARs to be uniformly of high quality, and rated the awardee as exceptional. AR, Tab 14, TEB Report, at 635. The record does not indicate that the TEB members, when evaluating BAH’s past performance, were aware that one of the CPAR reports on which they based their evaluation of BAH’s performance quality was for a project other than the identified references on which they had assessed BAH’s past performance relevance. See id.

We have previously found nothing improper in an agency’s decision to limit its review of past performance information in various ways. In this regard, an agency has the discretion to determine the scope of the past performance history to be considered, provided all quotations are evaluated on the same basis and the evaluation is consistent with the terms of the solicitation. See Hygeia Solutions Partners, LLC; STG, Inc., B-411459 et al., July 30, 2015, 2015 CPD ¶ 244 at 13; Weidlinger Assocs., Inc., B-299433, B-299433.2, May 7, 2007, 2007 CPD ¶ 91 at 8. Here, however, the record reflects that vendors’ past performance history was not evaluated on the same basis, and that the agency engaged in disparate treatment of CSR.

As set forth above, the agency’s evaluation of CSR’s past performance was based on only the most recent CPARs for those specific projects identified by the vendor in its quotation. However, when evaluating BAH’s past performance, the agency considered CPARs for other than the specific projects that BAH had identified in its quotation. Specifically, the PERC support services CPARs at issue here did not involve one of the projects that BAH referenced in its quotation. It was an entirely different project than the LESA effort that BAH had performed for ICE. Quite simply, to the extent that the agency’s past performance evaluation of BAH considered CPARs for other than the projects specifically referenced by the awardee in its quotation, the agency was required to do the same when evaluating CSR’s past performance. As the agency was required to treat vendors equally and evaluate past performance evenhandedly, and failed to do so here, the agency’s actions were disparate and unreasonable.

The agency does not dispute that BAH’s referenced LESA effort and the evaluated PERC effort were different projects, or that its evaluation took into account the CPARs for BAH’s PERC project. The agency nonetheless argues that its actions were not unfair or disparate because the referenced and evaluated projects “bore the same contract number,” and because “the agency record . . . do[es] not support a conclusion that [the] assessment of work under the same contract number was wholly irrelevant” to the evaluation of BAH’s past performance. Memorandum of Law, Jan. 5, 2017, at 2. We find these contentions unconvincing. The fact that the LESA and PERC efforts may have been performed under the same overarching contract vehicle does not alter the fact that they were different projects, and the agency had previously decided that only the CPARs for “the specific projects that were identified by the offerors” in their quotations would be considered. COS, Dec. 22, 2016, at 2. Furthermore, the agency’s assertion that BAH’s PERC project was not wholly irrelevant misses the point: if the agency wanted to consider relevant CPARs for other than for the projects referenced for BAH, it was then required to do the same for CSR (or any other vendor). Because the agency failed to consider all available, relevant CPARs when evaluating CSR, it failed to take into account information that may have affected the agency’s evaluation rating.  (CSR, Inc. B-413973, B-413973.2: Jan 13, 2017)

The only remaining question is whether the 6th Floor Project was complete at the time of the evaluation. The CPARS evaluation indicates that the project was only 45 percent complete during the period assessed (November 2012 through April 2014), AR, Tab 13, CPARS, at 1, and there is no evidence in the record that the Navy obtained any additional information indicating that the project had been completed prior to the Navy’s evaluation here. Moreover, in its protest, Halbert asserted that the project was not complete, Protest at 8, and the agency never disputed that assertion in its responses, see generally, COS/MOL at 15-16; Supp. MOL at 3. Thus, the record establishes that the project was not 100 percent complete as defined by the solicitation here.

The Navy, however, contends that this relevancy criterion did not apply to information obtained by the agency, but rather, applied only to information submitted by the offeror. Supp. MOL at 3 (explaining that “the Solicitation required offerors to submit projects that were 100% complete[,]” but that “this requirement for ‘completeness’ did not apply to information found from other sources such as the PPIRS[.]”). The Navy further contends that it “was entitled to obtain information for use in the evaluation of past performance ‘from any and all sources[.]’” Id. (citing RFP at 36). Accordingly, the Navy argues that it was appropriate for it to obtain and consider interim CPARS evaluations regarding incomplete projects. Id. at 4.

As a general matter, an agency is not precluded from considering any relevant information, regardless of its source, and is not limited to considering only the information provided within the “four corners” of an offeror’s proposal when evaluating past performance. Al Raha Grp. for Tech. Servs., Inc., supra at 10 (citing FAR § 15.306(a)(2)(ii)); Paragon Sys., Inc., B‑299548.2, Sept. 10, 2007, 2007 CPD ¶ 178 at 8. Here, the solicitation also notified offerors that the past performance evaluation would not be limited to a review of the information provided in the offerors’ proposals, but that the agency could also obtain and review past performance information from other sources. RFP at 36. Accordingly, we agree with the Navy that it was permitted to obtain and consider information from other sources.

Our Office will question an agency’s past performance evaluation where the record indicates that the agency either failed to evaluate, or otherwise unreasonably considered, the relevance of past performance references in accordance with the solicitation’s stated evaluation criteria. Rotech Healthcare, Inc., supra, at 5‑6; Artic Slope Mission Servs., LLC, B‑410992.5, B‑410992.6, Jan. 8, 2016, 2016 CPD ¶ 39 at 12; Al Raha Grp. for Tech. Servs., Inc. supra, at 6. This is true whether the information is furnished by the offeror or obtained independently by the agency. Al Raha Grp. for Tech. Servs., Inc., supra, at 8-12 (sustaining protest because information independently obtained by the agency failed to satisfy the RFP’s recency and relevance criteria); Logistics Mgmt. Int’l, Inc., supra, at 14-16.

Here, even if we accept, for the sake of argument, the Navy’s contention that the RFP’s relevancy criteria did not apply to information independently obtained by the agency, we find that the Navy did not treat offerors equally in this regard. The record reflects that, in evaluating the past performance of Offeror 7, the TET found that one of the 23 evaluations in the PPIRS database for Offeror 7 demonstrated “relevant adverse past performance.” AR, Tab 7, TET Report, at 22. Despite this conclusion, however, the TET did not assign Offeror 7’s proposal a weakness for this project. Id. at 23. Likewise, the project is not mentioned in the Navy’s tradeoff analysis. AR, Tab 9, SSAC Report, at 5. The contemporaneous record suggests that the project was not considered by the Navy.

After Halbert challenged this aspect of the Navy’s evaluation, Comments at 5‑6, the Navy responded that the TET’s statement was a typographical error and that, indeed, Offeror 7’s project was not considered by the agency in its past performance evaluation because it did not meet the solicitation’s criteria for relevancy. Supp. MOL at 2 n.1. Specifically, the Navy alleges the project is not relevant because the dollar value fell below the solicitation’s $3 million minimum construction cost requirement. Id.

Thus, the record reflects that the agency did not consider Offeror 7’s adverse performance on a project because the agency concluded that it did not satisfy the RFP’s relevancy criteria, while simultaneously considering Halbert’s adverse performance on a project that did not satisfy the RFP’s relevancy criteria. It is fundamental that a contracting agency treat all offerors equally, and therefore, it must evaluate offers evenhandedly against common requirements and evaluation criteria. Logistics Mgmt. Int’l, Inc., supra, at 16. On this record, we find the Navy’s evaluation of the past performance of Halbert to be unreasonable because the Navy engaged in disparate treatment of Halbert.  (Halbert Construction Company, Inc. B-413213: Sep 8, 2016)

Past Performance

Rotech challenges the agency’s assignment of a rating of good to Lincare’s proposal under the past performance evaluation factor, arguing that the record fails to establish that the rating was based on Lincare’s performance on contracts similar in size, scope, and complexity to the effort here, as required by the RFP. As discussed below, we agree with the protester.

The RFP instructed offerors to provide past performance information on contracts of similar size, scope, and complexity that were ongoing or completed within the last three years. RFP at 97. The solicitation explained that past performance would be evaluated as follows:

Past performance will be evaluated based on the level of risk of poor performance as indicated by an offeror’s past performance references of similar size, scope, and complexity that are ongoing or have been completed within the last three years. A Past Performance Survey template will be included as an attachment to the solicitation, to include questions concerning timeliness of performance, cost control, effective management, customer satisfaction, quality awards, and the technical success of the project. All of the [P]ast Performance Survey questions are of equal importance relative to each other. Offerors shall submit those surveys to their previous contract references, who then shall complete the survey and submit directly to the contracting officer for evaluation.

RFP at 101.

In its proposal, Lincare identified eleven recent/ongoing contracts on which it was performing (or had performed), the number of oxygen patients under the contract, and the annual contract value. Two of the contracts were with the VA. Nine of the contracts served patient populations of approximately 1,000 or less; the remaining two served populations of 2,612 and 7,383 patients, respectively. Only the final contract had an annual value of more than $5 million.

The agency received completed past performance surveys from references on only two of the contracts identified by Lincare. One of the surveys pertained to a contract involving 962 oxygen patients (the annual value of which was $3.5 million) on which Lincare’s performance was generally rated as excellent. The other pertained to Lincare’s performance on a contract involving 218 patients (with an annual value of $2.5 million) on which Lincare’s performance was generally rated as satisfactory; in response to the survey question as to whether the reference would award another contract to Lincare, this reference indicated “No.” In evaluating Lincare’s past performance, the evaluation team also took into consideration the personal experience of one of its members dealing with Lincare in a private industry setting. Based on her experience, this individual rated Lincare’s past performance as excellent; the evaluation record contains no information as to the similarity of the private sector contract to the work solicited here, however.

The evaluation team summarized its basis for assigning Lincare’s proposal a past performance rating of good as follows:

The offeror received a long list of strengths identified by the evaluators, as evidenced by the many past performance references provided. The offeror received good comments on their references, which include current contracts with entities within the Dept. of VA. The offeror has over forty years of experience, already has a large patient population including VA, military, and private insurance patients. Nothing in their past performance references indicated poor performance, and one evaluator had personal experience dealing with the offeror in the private industry setting-with excellent reviews. They appear to be experienced, provide good customer service, and excel at solving communication issues that arise. Some weaknesses were identified, however, including the fact the offeror is not currently contracted with any entire VISN in VA, a few mixed reviews on their past performance questionnaire, including Walla Walla, WA VA rating the offeror satisfactory but not likely to renew their contract due to some problems with rural service.

Lincare’s past performance consensus rating resulted in “Good” despite weaknesses identified, because the overall strengths more than offset the weaknesses. Some evaluators did give Lincare a “Satisfactory” overall rating, and one an “Excellent” rating, but after discussion the consensus was agreed upon to rate Lincare’s past performance as “Good” due to the overwhelming strong responses.

SSD at 13.

As noted above, the protester contends that the agency’s evaluation was flawed in that it failed to consider the similarity and relevance of Lincare’s contract references to the effort solicited here.

An agency is required to consider, determine, and document the similarity and relevance of an offeror’s past performance information as part of its past performance evaluation. Phillips Healthcare Informatics, B-405382.2 et al., May 14, 2012, 2012 CPD ¶ 220 at 9; see Federal Acquisition Regulation (FAR) §15.305(a)(2). As a general matter, the evaluation of an offeror’s past performance, including the agency’s determination of the relevance and scope of an offeror’s performance history to be considered, is a matter within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. The Emergence Group, B-404844.5, B-404844.6, Sept. 26, 2011, 2011 CPD ¶ 132 at 5. We will question an agency’s evaluation conclusions where they are unreasonable or undocumented, however. Clean Harbors Envtl. Servs., Inc., B-296176.2, Dec. 9, 2005, 2005 CPD ¶ 222 at 3. The critical question is whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation’s evaluation scheme, and whether it was based on relevant information sufficient to make a reasonable determination of the offeror’s past performance. DRS C3 Sys., LLC , B‑310825, B-310825.2, Feb. 26, 2008, 2008 CPD ¶ 103 at 22.

Here, the record simply does not demonstrate that the agency had a reasonable basis for assigning Lincare’s proposal a past performance rating of good based on Lincare’s performance on contracts similar in size to the solicited effort. There is no evidence in the contemporaneous record that the evaluators considered whether Lincare’s contracts were similar in size to the contract to be awarded here; moreover, the agency received completed past performance surveys on only two of Lincare’s contracts, neither of which appears to have been comparable in size (in terms of number of patients or annual contract value) to the contract here. Furthermore, there is no information in the record establishing that the contract on which one evaluator had personal experience was comparable in size, scope, and/or complexity to the solicited effort. Under these circumstances, we are unable to conclude that the agency had a reasonable basis for its rating of Lincare’s past performance. Accordingly, we sustain the protest on this basis.  (Rotech Healthcare, Inc.
B-413024, B-413024.2, B-413024.3: Aug 17, 2016)

In one respect, however, NASA’s evaluation of CGJV’s past performance was unreasonable and contrary to the terms of the RFP. ABJV challenges NASA’s consideration of past performance by CTR Management regarding a contract with HHS because there was no record of performance for that contract. Protester’s Comments at 3-7; Protester’s Supp. Comments at 3-4. ABJV argues that since the RFP expressly provided that the past performance factor would assess the “offeror’s performance record,” and that NASA “w[ould] not consider past performance on any contract that has no documented performance history,” it was improper for NASA to consider CTR Management’s past performance under this contract. Id. at 4. ABJV argues that given CTR Management’s roles as manager of the joint venture, and as the firm primarily responsible for performance of [DELETED] SOW sections, NASA’s failure to identify the lack of past performance information for CTR Management was unreasonable. Protester’s Comments at 5-6.

Although NASA concedes that the record contains no documentation of the performance by CTR Management under the HHS contract, the agency nevertheless argues that it was proper to consider the contract in the past performance evaluation. Specifically, even though there was no record of CTR Management’s performance between the award of that contract in August and the submission of proposals under this RFP in November,[9] NASA maintains that it still was proper to evaluate the HHS contract as both recent and relevant. NASA argues that recency and relevance were separate elements of the past performance evaluation, and therefore, the agency could evaluate past performance for CTR Management despite having no record of the firm’s performance. Supp. AR at 3-4.

In our view, the terms of the RFP precluded consideration of past performance for which there was no record of performance, as was indisputably the case for CGJV’s HHS contract. Further, we do not see how the fact that one joint venturer had recently been awarded a contract for similar services would have any reasonable relationship to the agency’s past performance evaluation absent a record of the quality of that performance. In sum, the record provides no rational basis for NASA to consider the HHS contract in the past performance evaluation of CGJV.

We also conclude that ABJV was prejudiced by the misevaluation of CGJV’s past performance. The record does not reflect precisely how significant the consideration of the HHS contract was to the agency’s judgment about CGJV’s past performance. The record does show, however, that the HHS contract was the only past performance identified by CTR Management; that CTR Management was to have overall management of the JV; and that CTR Management was to have primary responsibility for performance of [DELETED] of the seven main SOW areas. AR, Tab 11, CGJV Proposal, vol. II, at 3 (organizational chart and explanation of roles). Given the discussion of CTR Management’s HHS contract positively in both the evaluation report and the source selection decision, the record shows that the misevaluation influenced the agency’s tradeoff judgment. See AR, Tab 13, Evaluation of CGJV, at 18-20; AR, Tab 15, Source Selection Decision, at 12. In such circumstances, we resolve doubts regarding prejudice in favor of a protester; a reasonable possibility of prejudice is sufficient to sustain a protest. Crowley Logistics, Inc., B-412628.2 et al., Apr. 19, 2016, 2016 CPD ¶ 120 at 8. Accordingly we sustain this aspect of ABJV’s challenge to the past performance evaluation.  (Alutiiq-Banner Joint Venture B-412952, B-412952.2, B-412952.3, B-412952.4: Jul 15, 2016)

Past Performance

Delfasco challenges the agency’s evaluation of the awardee’s past performance as “relevant.” According to the protester, given the several types of work required under the contemplated contract for which GTI lacks relevant experience, no more than a “somewhat relevant” past performance rating was warranted.

An agency’s evaluation of past performance, including its consideration of the relevance, scope, and significance of an offeror’s performance history, is a matter of agency discretion which we will not disturb unless the agency’s assessments are unreasonable or inconsistent with the solicitation criteria. SIMMEC Training Solutions, B-406819, Aug. 20, 2012, 2012 CPD ¶ 238 at 4. Where a protester challenges an agency’s past performance evaluation, we will review the evaluation to determine if it was reasonable and consistent with the solicitation’s evaluation criteria and with the procurement statutes and regulations, and to ensure that it is adequately documented. Falcon Envtl. Servs., Inc., B-402670, B-402670.2, July 6, 2010, 2010 CPD ¶ 160 at 7.

Here, the solicitation provided that:

The Past Performance information will be evaluated to determine the relevancy of the like or similar items in accordance with the descriptions below:

* * * * *

Relevant - Present/past performance effort involved similar scope and magnitude of effort and complexities this solicitation requires.

Somewhat Relevant - Present/past performance effort involved some of the scope and magnitude of effort and complexities this solicitation requires.

RFP § M.3.3. The solicitation further defined relevant past performance as follows:

Relevant is defined as having previously produced like or similar items. Like or similar items are defined as items that have been produced using similar manufacturing processes, including experience with casting, machining, forging, metal forming, welding, essential skills and unique techniques required to produce the MK-76 with MK‑14, BDU-33 and the 25lb Suspension Lug.

RFP § L.3(f).

As indicated in its proposal, Delfasco has been a previous producer of the BDU-33 Practice Bomb, MK-76 Practice Bomb, and 25-pound suspension lug under both single and multi-year contacts. AR, Tab 20-02, Delfasco Proposal, Technical Subfactor 1, at 1; Tab 21-03; Delfasco Proposal, Past Performance Subfactor 2.2, at 10-11. Delfasco proposed using existing practices, technology, personnel, and equipment, all of which it has utilized to produce “millions of BDU-33 and MK-76 Practice bombs as well as thousands of the 25lb Suspension Lug.” AR, Tab 20-02, Delfasco Proposal, Technical Subfactor 1, at 1, 9. Based on its extensive prior experience, Delfasco’s proposal as reevaluated by the agency during the protest received a “relevant” rating under both past performance subfactors and also the overall past performance factor.

In contrast with Delfasco’s extensive experience producing BDU-33 Practice Bombs, MK-76 Practice Bombs, and 25-pound suspension lugs, the agency’s evaluation indicated that GTI had much more limited experience relevant to these items. As discussed below, the agency found the awardee lacked relevant past performance with respect to two necessary skills identified in the RFP, and only somewhat relevant experience with respect to another skill. Thus, the agency evaluated the relevance of GTI’s experience under the quality program problems subfactor of the past performance factor as follows:

The offeror submitted three parts for their relevant items, the Leaf Sight Rail Grabber Assembly, M385A1, and the BDU-48. The 76/33 Bomb Bodies are cast and the respective Lugs are forged. The offeror has not demonstrated they have any relevance [sic] with castings and forgings requirements including inspection and machining of castings and forgings. They did provide a matrix which demonstrates some of the scope of work relevant to the 76/33. This includes plating, MPI, hardness testing, and weld testing. They demonstrated somewhat relevant requirements to machining threads but not specific to class three threads which have a tighter tolerance. There are forming processes for the BDU‑48 practice bomb fin which are relevant and welding which is very relevant to the 76/33. There are machining processes for turning which are relevant to the 76/33. Though the offeror does not appear to have relevant experience i[n] all aspects that will be required on this solicitation, their past performance does involve a similar scope and magnitude of effort and complexities this solicitation requires giving the offeror an overall relevancy rating of “Relevant[.]”

AR, Tab 53-01, GTI Past Performance Evaluation, Subfactor 1, at 5.

Delfasco asserts that given the evaluated limits on GTI’s experience as indicated in the agency’s evaluation, it was unreasonable for the agency to assign the same “relevant” rating assigned Delfasco, which had extensive experience producing the BDU-33 Practice Bomb, MK-76 Practice Bomb, and 25-pound suspension lug being procured here. We agree.

Again, the solicitation defined “like or similar items,” for purposes of establishing that experience as relevant, as “including experience with casting, machining, forging, metal forming, welding, essential skills and unique techniques required to produce the MK-76 with MK-14, BDU-33 and the 25lb Suspension Lug.” RFP § L.3(f). Of those necessary skills, the agency noted that GTI had not demonstrated “any” relevant experience in two (casting and forging) and only “somewhat relevant” experience in the third (machining), such that GTI had only demonstrated “some” of the skills necessary to produce the bomb bodies. AR, Tab 53‑01, GTI Past Performance Evaluation, Subfactor 1, at 5. In our view, the agency has not adequately explained, nor is it otherwise apparent, why GTI’s limited relevant experience warranted a “relevant” rating (“similar scope and magnitude of effort and complexities this solicitation requires”) rather than a lower rating. For these reasons, we sustain the protest.  (Delfasco, LLC B-409514.3: Mar 2, 2015)

Past Performance

The solicitation instructed offerors to provide summaries of up to 5 contracts that were performed within 36 months of the proposal submission deadline and that were similar in size, scope, and complexity to the current requirement. RFP, § L.3.3. The offerors also were required to provide a questionnaire to a reference for each past performance contract listed. Id. The solicitation provided that contracts would be assessed for relevancy and quality of performance, and then assigned an overall confidence rating. RFP, § M at 2.4.

Patricio submitted five past performance references and GID submitted three past performance references. The offerors’ past performance was evaluated as follows:

AR, Tab 15, Past Performance Report, at 6, 7; Agency Response (Mar. 24, 2016) at 6; Supp. AR (April 1, 2016) at 3.

As set forth above, GID received a past performance rating of substantial confidence and Patricio received a past performance rating of satisfactory confidential. In explaining the basis for these ratings, the SSA, who was also the Contracting Officer stated the following:

The PPET [Past Performance Evaluation Team] decided that a SUBSTANTIAL performance confidence assessment would be warranted when ALL relevant past performance was determined to be VERY RELEVANT with EXCEPTIONAL quality ratings. Any other combination of relevancy and quality would result in a SATISFACTORY (or lower) performance confidence assessment. This methodology was consistently employed by the PPET for every offeror.

AR, Tab 25, Declaration of SSA, at 2-3 (emphasis in original).

As a general matter, the evaluation of an offeror’s past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, Joint Venture, B‑401679.8 et al, Sept. 8, 2010, 2010 CPD ¶ 211 at 7; MFM Lamey Grp., LLC, B‑402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10. Where a protester challenges the past performance evaluation and source selection, we will review the evaluation and award decision to determine if they were reasonable and consistent with the solicitation’s evaluation criteria and procurement statutes and regulations, and to ensure that the agency’s rationale is adequately documented. Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, Joint Venture, supra.

For example, in Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, Joint Venture, we sustained a challenge to the agency’s past performance evaluation methodology, finding it unreasonable because it effectively penalized offerors for submitting less relevant contract references, where, had the less relevant contracts not been submitted, the offerors’ past performance rating scores would have been higher. Id. at 8. In that case, we agreed with the protester’s argument that any reasonable consideration of its less relevant contracts could only have increased its score since it received all superior ratings for those contracts. Id.; see also Olympus Bldg. Servs., Inc., B-285351, B-285351.2, Aug. 17, 2000, 2000 CPD ¶ 178 (sustaining protest where agency’s mechanical formula for scoring experience unreasonably penalized an offeror for including extra, less relevant references in addition to relevant ones).

Here, Patricio asserts that the evaluation methodology used by the agency was unreasonable. We agree. Specifically, Patricio, like GID, had two very relevant past performance contracts that were rated exceptional for quality. Patricio also had two additional references that were found to be relevant and rated exceptional for quality, and one additional reference that was very relevant and rated very good for quality. In our view, the agency’s mechanical evaluation of past performance was unreasonable where the result was that additional relevant past performance references with exceptional and very good quality resulted in a downgraded past performance rating. In sum, the agency’s past performance evaluation methodology was not rational and resulted in an unreasonable evaluation of the offerors’ past performance. Therefore, we sustain the protest on this basis.  (Patricio Enterprises Inc.
B-412740, B-412740.3, B-412740.4: May 26, 2016)  (pdf)

SupplyCore’s ID/IQ Contracts

In large part, this protest turns on the parties’ competing interpretations of the following RFP provision regarding what types of past performance would be considered by the Air Force:

If the contract you are submitting is an ordering type contractual vehicle (e.g., an Indefinite Delivery “D” type contract per [Federal Acquisition Regulation (FAR)] 16.5), only after issuance of a delivery/task order does performance occur. Given this, an individual order (or orders) under the basic ordering contract should be submitted in lieu of just the basic ordering contract itself. Each submitted order shall be considered a separate effort unless information is provided indicating follow-on or series of orders for continued performance of the same scope and such can be verified by the user/customer. In this event, such submitted multiple orders shall be considered as a single effort for evaluation purposes.

RFP at 131.

SupplyCore initially submitted past performance information concerning four ID/IQ contracts, but the Air Force stated during discussions that, in accordance with the RFP, only order-related information would be evaluated and that ID/IQ contract-related information would be assessed as “not relevant.” AR, Tab 8, SupplyCore Evaluation Notices and Responses, at 1. In response, SupplyCore amended its past performance proposal to include information about a representative task order for each of the ID/IQ contracts. Id. In its response to the initial protest, however, the agency contended that it had reasonably considered the overall ID/IQ contracts in evaluating SupplyCore’s past performance confidence assessment. See, e.g., Supp. COSF in B-411015.2 and B-411015.3 (Mar. 2, 2015) at 11-12. We sustained LMI’s and RGTS’s protests challenging this aspect of the agency’s evaluation for three independent reasons: (1) there was no contemporaneous evidence that the agency had in fact considered the ID/IQ contracts generally, or specifically in terms of the RFP’s relevance criteria; (2) the agency’s reliance on the ID/IQ contracts was inconsistent with the RFP’s instruction that offerors submit only order-related past performance; and (3) the agency’s reliance on the ID/IQ contracts was inconsistent with its contemporaneous interpretation of the applicable RFP provision during discussions. Al Raha Grp. for Tech. Servs., Inc.; Logistics Mgmt. Int’l, Inc., supra, at 8-10.

During its corrective action reassessment, the Air Force investigated the underlying ID/IQ contracts against which the four orders submitted by SupplyCore were issued.[5] Specifically, the agency contacted the Defense Logistics Agency (DLA), the agency that awarded the ID/IQ contracts against which SupplyCore’s orders were issued, to obtain additional information regarding the nature of the contracts and the resulting orders. AR, Tab 47, Email from Air Force Contracting Officer to DLA (Apr. 29, 2015), at 2. DLA provided the Air Force with copies of the applicable ID/IQ contracts. AR, Tab 67, SupplyCore ID/IQ Contracts. Additionally, two DLA contracting officers spoke to the Air Force by phone. The Air Force summarized one DLA contracting officer’s discussion of two of SupplyCore’s ID/IQ contracts as follows:

[The DLA Contracting Officer] confirmed the [evaluators’] evaluation that DLA uses unique ordering procedures for its IDIQ contracts. She stated that DLA drops delivery orders constantly based on their customers’ mission. Orders can range anywhere from a few dollars to over a million dollars. The scope and magnitude of the orders is based on the customers’ various “projects.” Some orders simply meet an immediate need for material, while others have associated services. The vendor works directly with the customer to support their needs.

AR, Tab 45, Notes from Apr. 30, 2015, Phone Call (May 11, 2015), at 1; see also Tab 46, Notes from May 5, 2015, Phone Call (May 11, 2015), at 1 (noting the second DLA contracting officer stated that “[o]rders can range anywhere from a few cents to thousands of dollars depending on the need”).

The Air Force also reviewed the associated ordering histories for the DLA ID/IQ contracts that were available through the Department of Defense’s Electronic Document Access (EDA) system. The EDA system included only the following information for each order: (1) the order number; (2) the number of modifications, if any; (3) the obligated amount; and (4) the effective date. AR, Tab 47, EDA Data, at 6-2337. Based on the EDA data (and other supplemental data from DLA), the Air Force calculated: (1) the duration of the ordering periods falling within the RFP’s 5‑year recency period; and (2) the number and total value of orders placed against each of the four ID/IQ contracts during the “recent” periods of performance, as follows:

AR, Tab 48, Past Performance Report (July 29, 2015), at 113, 115-16, 118, 120-21.

Next, the Air Force considered the general scopes of these contracts, and whether they satisfied the 7 specific experience areas identified in the RFP. The agency’s evaluation considered the information provided by SupplyCore in its proposal, which consisted of general information regarding the ID/IQ contracts and specific information regarding four individual orders. See AR, Tab 8, SupplyCore Evaluation Notices and Responses. Based on this analysis, the Air Force concluded that SupplyCore’s ID/IQ contracts met the RFP’s exception as a series of orders for continued performance of the same scope of work. The Air Force evaluated the relevancy of the four ID/IQ contracts as follows:

Based on these findings, the SSA determined that the aggregate value of SupplyCore’s past performance work was $502,708,775.94 under the four ID/IQ contracts, and that the prior work was “very relevant” to the RFP’s requirements and warranted a rating of “substantial confidence.” AR, Tab 50, Source Selection Decision (Aug. 5, 2015), at 83.

The protesters first challenge the agency’s determination that SupplyCore’s ID/IQ contracts satisfied the RFP’s exception to the requirement that only orders issued against ID/IQ contracts would be evaluated. The protesters contend that the agency’s determination is inconsistent with the RFP’s terms because: (1) only four representative orders, one for each ID/IQ contract, were submitted in the awardee’s proposal for the agency’s review; (2) the agency unreasonably assumed, based on its review of a single order issued against each of SupplyCore’s ID/IQ contracts, that the other orders issued against the contracts were for follow-on or continuous performance of the same scope; (3) other information available to the agency demonstrates that the tens of thousands of orders issued against the contracts were not for follow-on or continuous performance; and (4) the information was not reasonably verified by DLA or the end user/customer of the orders. The protesters also argue that the Air Force failed to reasonably consider whether the “series of orders” had similar logistical and programmatic considerations as those required by the RFP or satisfied the seven specific experience areas.

The Air Force responds that it reasonably evaluated the tens of thousands of orders issued against SupplyCore’s ID/IQ contracts as constituting a “series of orders” for continuing performance of the same scope. COSF (Sept. 15, 2015) at 12‑15. The agency argues that DLA uses “unique” ordering procedures on its ID/IQ contracts, whereby DLA constantly issues many small value orders that, when aggregated, demonstrate a similar scope and magnitude of effort and complexity as compared to the RFP’s requirements. Id. at 13-14. The Air Force contends that DLA’s placement of many thousands of orders under the overall ID/IQ contracts’ broad statements of work demonstrates that the orders were for the continuing performance of the same scope. Id.

We find unreasonable the Air Force’s conclusion that the many thousands of orders issued against SupplyCore’s four individual ID/IQ contracts constituted, collectively, four discrete references. First, it appears that the agency evaluated SupplyCore’s ID/IQ contracts, as opposed to the individual orders issued against the contracts, in violation of the RFP’s instruction that only orders would be considered. We also find that the agency’s reliance on the overall, general scopes of work for the ID/IQ contracts was insufficient to satisfy the RFP’s requirements that, to be considered as a single effort, multiple orders had to be for follow-on performance or continued performance of the same scope. Where a protester and agency disagree over the meaning of solicitation language, we will resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all of its provisions; to be reasonable, and therefore valid, an interpretation must be consistent with the solicitation when read as a whole and in a reasonable manner. TransAtlantic Lines, LLC, B‑411242, B-411242.2, June 23, 2015, 2015 CPD ¶ 204 at 16.

Here, we conclude that the protesters reasonably interpreted the “series of orders” provision to be an exception to the general requirement that only order-related information would be considered in the past performance evaluation. The protesters argue that the text of this provision applies only where: (1) orders were submitted or provided; (2) for follow-on or continuous performance of the same scope; and (3) such information could be verified by the user/customer. RFP at 131. We agree. In this regard, the agency’s interpretation of the provision that all orders issued under the awardee’s ID/IQ contract references are necessarily for follow-on or continuous performance, would effectively make every ID/IQ contract a “series of orders” eligible for consideration, which in turn would render superfluous the requirement that only order-related information be submitted and evaluated. Additionally, the Air Force’s evaluation of only a single representative task order, and each ID/IQ contract’s general scope of work, ignores the RFP’s requirement that the agency evaluate only order-related information, as opposed to the ID/IQ contracts against which the orders were issued. The agency failed to meaningfully evaluate whether the orders demonstrated follow-on or continuing performance because it did not seek any information regarding the scopes of the tens of thousands of orders upon which its evaluation conclusion relies.

Even the limited information in the record collected by the Air Force about the orders does not support the Air Force’s argument that the orders demonstrated a pattern of continued performance of the same scope. The Air Force’s notes of its conversations with DLA reflect that the DLA contracting officers represented that the scope and magnitude of effort and complexity of the orders varied based on the unique project or customer requirements resulting in the issuance of each order. See AR, Tab 45, Notes from Apr. 30, 2015, Phone Call (May 11, 2015), at 1; Tab 46, Notes from May 5, 2015, Phone Call (May 11, 2015), at 1. Similarly, the record shows that the goods or services performed under the “series of orders,” while tangentially related under the umbrella ID/IQ contracts, varied significantly in terms of dollar value, duration, scope, and magnitude of effort and complexity. For example, based on the EDA data available for contract No. SPM-500-05-D-BP04, SupplyCore had 6 orders, all of which had an effective date of April 11, 2012 and ranged in value from $130.20 to $19,714.08. See AR, Tab 47, EDA Data, at 1929. While no additional information about the scope or programmatic considerations pertinent to the orders was provided, the fact that the orders: (1) all had the same effective date; and (2) were for divergent amounts, suggests that they were not for follow-on or continuous work of the same scope.

Additionally, under contract No. SPM500-05-D-BP06, SupplyCore’s proposal reflected seemingly distinct projects that were not for follow-on work or continuous performance of the same scope. First, under order No. NMM6, SupplyCore discussed a $143,461.67 project for the replacement and installation of three doors at the Misawa Air Base Hospital in Misawa, Japan. AR, Tab 8, SupplyCore Evaluation Notices and Responses, at 21-25. The proposal, however, also discussed SupplyCore’s other work under the contract, including the installation of a large solar project (exceeding $1 million) in Okinawa, Japan, and other facility and maintenance projects, including: (a) heating, ventilation, and air conditioning; (b) plumbing; (c) electrical; (d) roofing and flooring; (e) procurement of construction supplies; and (f) perimeter security. Id. at 24, 27. Thus, based on the awardee’s own description of the work under the contract, it appears that the orders issued against the contract were for discrete projects located at U.S. government sites throughout Japan. On this record, we conclude that there is insufficient evidence to demonstrate that the various orders issued against SupplyCore’s ID/IQ contracts were for follow-on work or a series of orders for the continuous performance of the same scope of work. We therefore find that the Air Force’s evaluation of the tens of thousands of orders under SupplyCore’s ID/IQ contracts as constituting a series of orders for continuing performance of the same scope of work, and thus warranting treatment as a single reference, was inconsistent with the RFP’s express terms, unreasonable, and not adequately documented.  (Logistics Management International, Inc.; Al Raha Group for Technical Services, Inc.; Dalma Tech² Company B-411015.4, B-411015.5, B-411015.6: Nov 20, 2015)  (pdf)

The RFP provided that the agency would assess the relevancy of the past performance examples based on the extent to which the example showed that it involved the same magnitude of effort and complexities involved in performing the solicited requirement. RFP at 72. As noted above, the contract awarded to IGH is for a total value of approximately $49 million, or more than $16 million per year. It contemplates performance on a nationwide basis in all 50 states, with a requirement for an estimated 170 full time equivalent personnel, exclusive of management personnel. RFP at 26-27.

The record shows that IGH included three past performance examples in its proposal. The first of these examples was for performance of a community service relocation readiness program and lending closet program at a single location. AR, exh. 8, IGH Technical Proposal, Vol. II, at 29-30. The period of performance for this contract was 2 years, and the total dollar value of the contract was $58,176. The record shows that the agency assigned this past performance example a rating of relevant, stating without elaboration: “The past performance is within scope and the contract dollar value is not similar to the IGCE.” AR, exh. 14, PPEB Report, at 22.

IGH’s second past performance example was a contract to perform grant management support services for the Department of Housing and Urban Development. AR, exh. 8, IGH Technical Proposal, Vol. II, at 30-32. This contract was for a 3-year period of performance and had a total dollar value of $481,308. IGH’s description of the contract states that the firm was a subcontractor assisting in the provision of grant management services. The agency assigned this contract a rating of relevant, stating without elaboration: “The scope of the past performance is marginally within the scope of the solicited work. The contract dollar value is significantly lower that the IGCE.” AR, exh. 14, PPEB Report, at 24.

IGH’s third past performance example was for a contract to provide program management support services for a pilot program run by the Veterans Health Administration. That contract was for a 1-year period of performance and was for a total dollar value of $25,477. AR, exh. 8, IGH Technical Proposal, Vol. II, at 32-34. The record shows that the agency assigned this past performance example a rating of somewhat relevant, stating without elaboration: “The scope of the past performance is marginally within the scope of the solicited work. The contract dollar value is significantly lower that the IGCE.” AR, exh. 14, PPEB Report, at 24.

In contrast to these comparatively favorable ratings, the record shows that the agency assigned measurably less favorable relevancy ratings to the other offerors’ past performance examples, and in some instances, simply failed to consider some of the apparently relevant examples submitted.

For example, in evaluating the past performance of TASA, the record shows that the firm had a total of four past performance examples, two of which appear to be relevant to the solicited requirement because they were for performance of predecessor contracts in connection with two of the programs being supported under the solicited requirement, the Yellow Ribbon Reintegration Program (YRRP) and the Employment Initiative Program (EIP). See RFP at 7.

The record shows that the first of these contracts was for providing services in connection with the YRRP and EIP case management system; this contract was for an 11-month period of performance, and had a total dollar value of $476,570. TASA AR, exh. 11, TASA Technical Proposal, Vol. II, at 33. The agency assigned this past performance example a somewhat relevant rating, finding as follows: “Although past performance involves some experience within the requirements, the contract dollar value does not support the magnitude of effort this solicitation requires.” AR, exh. 14, PPEB, at 29. This finding appears directly inconsistent with the agency’s assignment of relevant ratings to the first two of IGH’s past performance examples, which had values of, respectively, $58,176 and $481,308. There is no explanation in the record for this apparently disparate evaluation of the two firms’ past performance examples.

The second apparently relevant past performance example for TASA was a contract to develop a web platform for the YRRP and EIP programs. This contract was for a 2-year period of performance and had a total dollar value of approximately $3.9 million. TASA AR, exh. 11, TASA Technical Proposal, Vol. II, at 32. The record shows that the agency simply did not consider this past performance example at all during its evaluation. There is no explanation in either the contemporaneous record or the agency’s submissions during the protest for the agency’s actions, notwithstanding that this contract appears relevant to the solicited requirement, and is of a much larger dollar value than any of the past performance examples proffered by IGH.

As a final example, the record shows that TEK Source included a past performance example for a contract valued at $10.7 million. TEK Source AR, exh. 11, TEK Source Technical Proposal, Vol. II, at 28. In evaluating this example, the agency assigned it a rating of only somewhat relevant. Of significance, the agency observed, in assigning that rating, that the contract dollar value for this contract was “significantly lower than the IGCE [independent government cost estimate].” AR, exh. 14, PPEB Report, at 39.

In sum, the record reflects a lack of consistency in the agency’s assignment of relevancy ratings to the offerors’ past performance examples. IGH’s past performance examples were rated comparatively favorably, despite the fact that the value of those contracts was only a small fraction of the value of the solicited requirement, and despite the fact that, for two of those examples, the agency found the scope of the contract to be only marginally related to the scope of the solicited requirement. While the agency may have had some reasonable basis for assigning the relevancy ratings it did assign to the IGH past performance examples, there is nothing in the record that memorializes the agency’s conclusions.

In contrast, the record shows that, in evaluating the other offerors’ past performance examples, the agency either applied what appear to be more stringent relevancy criteria (for example, finding apparently directly relevant contracts only somewhat relevant, or finding contracts with much larger dollar values only somewhat relevant), or simply failed to evaluate the past performance examples proffered. As with the evaluation of IGH’s past performance examples, while there may be some reasonable basis for the agency’s conclusions, there simply is no explanation in the record to support the agency’s apparently inconsistent evaluation conclusions described above. We conclude that the agency did not have a reasonable basis for assigning relevancy ratings to the past performance examples.  (Metis Solutions, LLC; TASA Information Technology Group, Inc.; TENICA and Associates, LLC; Dynamic Systems Technology, Inc.; Brandan Enterprises, Inc., TEK Source USA, Inc. B-411173.2, B-411173.4, B-411173.5, B-411173.6, B-411173.7, B-411173.8, B-411173.9: Jul 20, 2015)  (pdf)

RGTS and LMI contend that the Air Force’s evaluation of SupplyCore’s past performance as warranting a “substantial confidence” assessment was fundamentally flawed because the agency failed to evaluate SupplyCore’s past performance in accordance with the RFP’s relevancy criteria. Specifically, the protesters argue that the agency failed to meaningfully consider the limited scope and magnitude of effort and complexities of SupplyCore’s past performance references when it assigned the awardee the highest possible past performance confidence assessment. See, e.g., RGTS Comments at 4-12; LMI Comments at 10-14, 16-18; RGTS Supp. Comments (Mar. 5, 2015) at 3-10. As discussed below, we find that the agency’s evaluation of SupplyCore’s past performance was inconsistent with the terms of the RFP and not adequately documented. Therefore, we sustain the protests on these grounds.

As discussed above, our Office will question an agency’s past performance evaluation where the record indicates that the agency either failed to evaluate, or otherwise unreasonably considered, the relevance of past performance references in accordance with the solicitation’s stated evaluation criteria. As relevant here, an agency’s evaluation of an offeror’s past performance is unreasonable where the solicitation requires the agency to consider the value of the offerors’ references as compared to the value of the solicited requirement, and the agency fails to reasonably explain why comparatively small-value references provide a basis to justify a high past performance rating, or in this case the highest possible rating. E.g., Health Net Fed. Servs., LLC, B‑401652.3, B‑401652.5, Nov. 4, 2009, 2009 CPD ¶ 220 at 16 (sustaining a protest where an agency assigned the highest possible past performance rating based on three contracts covering less than 3 percent, and one contract covering 11 percent, of the requirements contemplated by the solicitation); Continental RPVs, B‑292768.2, B‑292768.3, Dec. 11, 2003, 2004 CPD ¶ 56 at 8 (finding prior contracts no larger than 4 percent of the solicitation requirements were not similar or relevant). Additionally, where an agency fails to document or retain evaluation materials, it bears the risk that there may not be an adequate supporting rationale in the record for us to conclude that the agency had a reasonable basis for its source selection decision. Navistar Def., LLC; BAE Sys., Tactical Vehicle Sys. LP, B-401865 et al., Dec. 14, 2009, 2009 CPD ¶ 258 at 13.

SupplyCore submitted four task order references, which was consistent with the RFP’s instruction that orders, not the ID/IQ contracts against which the orders were placed, were relevant for the past performance evaluation. See RFP at 131. Based on the RFP’s stated relevancy criteria regarding scope and magnitude of effort and complexities, we agree with the protesters that the four past performance references cited by SupplyCore appear to have little, if any, relevance to the effort required under the RFP. The effort contemplated by the RFP will require the contractor to provision vehicles and related materials in support of the RSAF’s F-15 operations, and has an estimated value of approximately $110 million and a 5‑year period of performance. The references identified by the awardee, and the agency’s evaluation of relevancy, were as follows:

AR, Tab 6, Final Past Performance Report, at 87-94; SupplyCore Updated FACTS Sheets.

As addressed above, the RFP here required the Air Force to evaluate the relevancy of an offeror’s past performance based on both the scope and the magnitude of effort and complexities of the cited references relative to those required by the RFP, and whether the references demonstrated experience with the RFP’s seven enumerated areas of experience. RFP at 145-46. With regard to the first component of the relevancy analysis, the agency was to consider logistical and programmatic considerations, including, for example, the dollar values of the submitted references. Id. at 146.

We conclude that, on their face, SupplyCore’s references are not comparable, as their combined value of $152,036.53 is only approximately 0.14 percent of the estimated value of the effort required by the RFP.[5] Although the Air Force rated all four references as only “somewhat relevant,” meaning that the references included “some” of the scope and magnitude of effort and complexities as required by the RFP, the record includes no basis for the determination that the cited efforts could reasonably be said to include even “some” of the effort required by the RFP given the extremely small value and short duration of the references. Indeed, the contemporaneous record demonstrates that the SSA specifically expressed concerns regarding the relevance of the past performance references identified by SupplyCore in light of “the magnitude and complexity of some of their efforts.” AR, Tab 4, Source Selection Decision, at 113. Based on the cited references including only a mere fraction of the effort required by the RFP, the references appear more consistent with the RFP’s relevance assessment of “not relevant,” meaning little or none of the effort required by the RFP.

Notwithstanding the small dollar value and short duration of the four references cited by SupplyCore, the Air Force argues that it nonetheless reasonably assigned the awardee a substantial confidence rating for its past performance. The Air Force argues that although the relevance assessment for an offeror’s past performance references was “a consideration in the overall confidence rating,” the relevance ratings did not “dictate a particular confidence rating.” Supp. COSF at 2. Although the Air Force is correct that the RFP’s past performance evaluation criteria provided for consideration of other factors for the overall past performance rating, such as the quality and recency of the past performance, we do not find reasonable the agency’s position that the small-value performance references identified by the awardee could support the highest possible confidence assessment. Relevance was an essential component of the overall confidence assessment, as demonstrated by the fact that the definitions for the past performance confidence assessments state that the assessment will be based on “the offeror’s recent/relevant performance record.” RFP at 147-48 (emphasis added). The agency’s contemporaneous evaluation and post-protest arguments here are very similar to the circumstances in Health Net Federal Services, LLC, where we sustained a protest challenging an agency’s past performance evaluation:

While we recognize that the past performance evaluation was not to be based on size alone, [the agency’s] assertion that its integrated assessment of [the awardee’s] past performance information justified giving [the awardee] the highest past performance rating is unpersuasive. Not one of [the awardee’s] contracts was evaluated as “relevant”; rather, they were all considered to be only “somewhat relevant.” Whether it was reasonable to consider some of the contracts even “somewhat” relevant given that their beneficiary populations were a small fraction of the size of the beneficiary population covered by the [contract to be awarded under the solicitation] is itself questionable. At a minimum, absent some further support in the record, it was not reasonable to give [the awardee] the highest past performance rating in reliance on the “exceptional” performance ratings associated with the prior contracts of such smaller size. On the contrary, the value of the “exceptional” ratings as predictors of [the awardee’s] success on the [to be awarded] contract is inherently diminished by their lack of relevance due to their relatively small size.

Health Net Fed. Servs., LLC, supra, at 17.

The Air Force’s evaluation here, which relied on unsupportable past performance relevance ratings to support the highest possible confidence assessment, is similarly flawed.  (Al Raha Group for Technical Services, Inc.; Logistics Management International, Inc. B-411015.2, B-411015.3: Apr 22, 2015)  (pdf)

Regarding past performance, Diamond argues that it was unreasonable for the agency to consider Diamond’s performance under the MPERS contract in the context of this procurement because the MPERS contract was too different in terms of size, scope, and IT requirements. Protest at 19-20. In this regard, Diamond asserts that the MPERS contract required the development of a non-financial, internal SBA system to be used by a small set of SBA personnel, for only one year of limited support, valued at approximately $400,000. In contrast, Diamond notes that the current requirement is for the development and full support of loan and bond systems for use across the SBA, with an awarded price of nearly $28 million.

An agency is generally not precluded from considering any relevant past performance information, regardless of its source. Weidlinger Assocs., Inc., B‑299433, B-299433.2, May 7, 2007, 2007 CPD ¶ 91 at 8. Regarding the relative merits of vendors’ past performance information, this matter is generally within the broad discretion of the contracting agency, and our Office will not substitute our judgment for that of the agency. Paragon Tech. Group, Inc., B-407331, Dec. 18, 2012, 2013 CPD ¶ 11 at 5.

We do not find the protester’s argument persuasive. First, while a contractor’s successful performance of a small contract might not provide a reasonable basis for predicting success in performing a much bigger requirement, an agency might reasonably to conclude that a firm’s marginal performance of a much smaller effort may suggest a level of risk in that firm’s performance of a much bigger and more complex project. Moreover, as noted above, the RFQ disclosed the agency’s intention to evaluate relevant past performance information from known government sources and did not limit relevant past performance based on a contract’s specific size or scope. Further, Diamond has not shown that the agency unreasonably concluded that the MPERS contract was relevant, based on the similar requirement to use SQL skills and design a relational database. Considering that the MPERS contract was performed for the same SBA office and consisted of IT requirements that were viewed as similar to those here, we find the agency’s consideration of this contract to evaluate Diamond’s past performance unobjectionable.  (Diamond Information Systems, LLC B-410372.2, B-410372.3: Mar 27, 2015)  (pdf)

TMSI asserts that the agency’s evaluation of Genco’s past performance was flawed in that it credited Genco with performance of contracts that did not qualify for evaluation under the past performance factor. TMSI asserts that none of Genco’s prior contracts were sufficiently similar in size or complexity to be evaluated under the past performance factor, yet the agency still considered Genco’s performance of those contracts in its source selection process. TMSI Comments On Agency Report, Nov. 20, 2014, at 1-5. More specifically, TMSI refers to the SSA’s source selection decision memorandum, quoted above, in which the SSA referenced Genco’s experience as part of the rationale for her source selection decision. AR, exh. 16, Source Selection Decision Memorandum, at 6. Accordingly, TMSI complains that the award to Genco was improper “[b]ecause the SSA credited Genco for past performance that was not relevant.” Id. at 4. We disagree.

First, as noted above, the agency assigned Genco a neutral past performance rating. In this regard, TMSI’s excellent past performance rating reflects the agency’s clear recognition that TMSI’s past performance was superior to Genco’s. Further, the solicitation specifically provided that, in evaluating proposals under the most important evaluation factor, technical excellence, the agency would assess various aspects regarding a vendor’s ability to perform the solicitation requirements. Among other things, section 1.6.13 of the performance work statement (PWS), titled “Transition Period,” requires that the successful offeror “shall complete all transition efforts in accordance with their transition plan . . . includ[ing] . . . establishing processing facilities; hiring and training staff; submitting all required forms sufficiently in advance to ensure obtaining clearances for staff; and obtaining data and account information.” AR, exh. 4B, RFQ amend. 7, at 2.

As noted above, in making her source selection decision, the SSA specifically referenced the risk associated with transitioning to a new contractor, stating: “I feel that Genco’s experience and approach (although not previously at the scale of that required of the CMOP program) minimizes that risk and is sufficient to support a successful transition and satisfactory performance.”

Our Office has repeatedly concluded that an agency’s evaluation of risk associated with an offeror’s proposal is appropriate, whether or not risk is specifically stated as an evaluation factor. That is, consideration of risk is inherent in an agency’s evaluation of technical proposals. See, e.g., Avaya Government Solutions, Inc., B‑410387, Dec. 15, 2014, 2014 CPD ¶ 372 at 5; TPMC-Energy Solutions Environmental Services 2009, LLC, B-408343.2 et al., Aug. 23, 2013, 2013 CPD ¶ 215 at 11; Government Acquisitions, Inc., B-401048 et al., May 4, 2009, 2009 CPD ¶ 137 at 6 n.14; Communications Int’l, Inc., B-246076, Feb. 18, 1992, 92-1 CPD ¶ 194 at 6.

Here, we view the SSA’s consideration of Genco’s past experience in performing activities that were not sufficiently similar in size and complexity to be evaluated under the past performance factor to constitute part of the SSA’s risk assessment regarding Genco’s ability to perform the PWS requirements, specifically including the transition requirements. As noted above, the SSA’s consideration of Genco’s experience was expressly tied to her concern regarding transition and performance risk and, as we have held, consideration of risk is always appropriate in a technical evaluation. On this record, we decline to sustain TMSI’s protest based on its assertion that the SSA was precluded from giving any consideration to Genco’s demonstrated capabilities under smaller, less complex contracts. TMSI’s protest in this regard is denied.  (Tri-Starr Management Services, Inc. B-408827.2, B-408827.4: Jan 15, 2015)  (pdf)

Next, FitNet challenges the agency’s substantive analysis of its past performance. For the reasons discussed below, we conclude that the agency’s past performance evaluation was unreasonable and constituted disparate treatment.

As a general matter, the evaluation of an offeror’s past performance is within the agency’s discretion. We will question the evaluation conclusions where they are unreasonable or undocumented. Clean Harbors Envtl. Servs, Inc., B-296176.2, Dec. 9, 2005, 2005 CPD ¶ 222 at 3. The critical questions are whether the evaluation was conducted fairly, reasonably, and in accordance with the stated evaluation terms, and whether it was based on relevant information sufficient to make a reasonable determination of the firm’s overall past performance. Id. It is fundamental that the contracting agency must treat all offerors equally, and therefore it must evaluate offers evenhandedly against common requirements and evaluation criteria. Tidewater Homes Realty, Inc., B-274689, Dec. 26, 1996, 96-2 CPD ¶ 241 at 3.

Here, with regard to past performance, the solicitation stated only that “award will be made to the responsible Seller whose offer conforming to the solicitation will be most advantageous to the Buyer on the basis of price, technical capacity, delivery, and past performance.” Solicitation at 2. As stated above, in evaluating FitNet’s past performance, the contracting officer relied upon a FedBid ActivityCard alert for FitNet, which listed “Late Delivery” and “Unresponsive to the Buyer Request,” as well as the contracting officer’s personal experience with FitNet from 2012. AR, Tab 6, Award Decision; Tab 3, FitNet’s Activity Card Report, at 2. With regard to FedBid’s ActivityCards, FedBid’s website explains that an alert is “a flag placed by a Buyer on a Seller’s account,” and that “[a] Buyer can place these alerts if they are dissatisfied with an aspect of the Seller’s performance on a particular buy on FedBid.” FedBid website, available at: http://fedbid.custhelp.com/app/answers/detail/a_id/42. In addition, it states that the alerts “are active on the Seller’s account for 18 months, unless removed by the Buyer.” Id.

Based on our review of the record, we conclude that the agency’s reliance on the ActivityCard alert was unreasonable. FitNet’s ActivityCard showed a summary of 66 FedBid awards for a total value of approximately $1.5 million, one of which was marked with an alert. AR, Tab 3, FitNet Activity Card, at 1-2. The alert stated only: “Alert Type: DLT, UBR.” Id. at 2. The contracting officer acknowledges that she did not investigate the basis for the alert, but rather, simply relied upon the fact that there was an alert, stating: “I relied upon the alerts and did not investigate further.” Supp. AR (Oct. 3, 2014), CO Statement at 4. The agency also acknowledges that the alerts “do not include subjective determinations.” Id. Accordingly, although the contracting officer may have known the broad category of the alert code (e.g., unresponsive to the buyer), the contracting officer clearly did not know the basis for the alert.

Moreover, the contracting officer does not state, nor does the record reflect, that she considered any of the 65 other awards cited for FitNet; instead, the agency simply found the alert identified “late delivery and unresponsiveness issues.” AR, Tab 6, Award Decision, at 1. On this record, we find the contracting officer’s reliance upon the alert to be unreasonable.

With regard to the contracting officer’s personal experience with FitNet from 2012, FitNet contends that the delay referenced by the contracting officer was caused by the government end‑user, not FitNet, and that based on this, the DOI’s Office of Small and Disadvantaged Business Utilization required that the contracting officer reinstate FitNet’s contract. Protester’s Comments (Oct. 7, 2014), at 4. The agency did not dispute these statements, and in fact, the contracting officer’s summary of the incident in the evaluation states that, after the contracting officer terminated the contract, FitNet responded that it “was unaware [the item] was not delivered [by the manufacturer],” and that the contracting officer thereafter offered to rescind her decision to terminate the contract on the condition that FitNet “provided a delivery date [for] the [item].” AR, Tab 6, Award Decision. The contracting officer also stated that ultimately, delivery was confirmed, and “[t]he [item was] delivered to Riverside Indian School on 10/1/12.” Id. On the record before us, it is not clear that FitNet was at fault for the late delivery in the manner discussed by the contracting officer in the award decision. As such, the contracting officer’s reliance on this incident was not reasonable.

Finally, as previously discussed, there is no indication in the record that the agency conducted the same evaluation of the awardee’s past performance as it did of FitNet’s past performance. It is fundamental, however, that the contracting agency must treat all offerors equally, and therefore, must evaluate offers evenhandedly against common requirements and evaluation criteria. Tidewater Homes Realty, Inc., supra, at 3. The agency’s failure to do so here constituted disparate treatment. We sustain the protest on this basis.  (FitNet Purchasing Alliance, B-410263: Nov 26, 2014)  (pdf)

Here, the record reflects that APT submitted information about three prior contracts/task orders to demonstrate its past performance. The first task order was the incumbent task order with [Missile Defense Agency] MDA for mission assurance; the second task order was awarded by MDA for system safety and occupational health; and the final contract was awarded by the National Aeronautics and Space Administration for safety, reliability, and quality. AR, Tab 9, Technical Evaluation, at 23. Based on the agency’s review of past performance information sheets (PPIs), CPARs (contractor performance assessment reports), MARs (monthly assessment reports), and agency interviews, MDA assigned the incumbent task order a very relevant relevancy rating and a satisfactory quality rating. Id. at 25. The other two efforts received relevant relevancy ratings and very good quality ratings. Id. The agency assigned APT’s past performance an overall quality rating of satisfactory because the agency concluded that the two very good ratings on the relevant efforts did not overcome the effect of the satisfactory rating that APT received on the very relevant incumbent task order. Id. In this regard, MDA believed that APT’s performance on the incumbent task order would be the best indicator of how APT would perform since the incumbent task order was the only effort submitted by APT that encompassed the same magnitude, scope, and complexity as the RTOP. Id.

APT argues that the agency’s satisfactory quality rating for its performance on the incumbent task order was arbitrary and unreasonable. APT asserts that its history of exceptional and very good performance, as documented in its CPAR and MAR ratings, demonstrate the unreasonableness of the agency’s satisfactory quality assessment. In this regard, APT contends that it received MAR ratings of very good or better in 30 of the 38 months of incumbent performance; exceptional ratings in 13 of the MAR reports; never received a MAR rating of less than satisfactory; and exceeded the satisfactory rating in each monthly assessment subsequent to and including July 2011. Protest at 7.

An agency’s evaluation of past performance, including its consideration of the relevance, scope, and significance of an offeror’s performance history, is a matter of discretion which we will not disturb unless the agency’s assessments are unreasonable or inconsistent with the solicitation criteria. SIMMEC Training Solutions, B-406819, Aug. 20, 2012, 2012 CPD ¶ 238 at 4. Since the agency is responsible for defining its needs and the best method for accommodating them, we will not substitute our judgment for reasonably-based past performance ratings. See MFM Lamey Group, LLC, B‑402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10.

As noted, MDA’s quality rating of APT’s performance on the incumbent task order was based on the agency’s assessment of APT’s PPIs, APT’s CPAR ratings, APT’s MAR ratings, and interviews with agency personnel. AR, Tab 9, MDA Technical Evaluation, at 23. The record reflects that the agency found APT’s CPAR and MAR to have quality ratings of exceptional, very good, and satisfactory.[8] Based upon the CPAR and MAR, the agency identified two issues of concern: (1) difficulties with planning and managing costs, and (2) a five month delay in filling a vacant position for a MDA program. Id. at 24. These concerns led the agency to conduct interviews with MDA representatives.

The evaluators interviewed five MDA representatives to gain further insight into the severity and causes of the identified issues. Id. Based on these interviews, the agency concluded that although APT eventually met all technical requirements, it had difficulty planning and managing resources on several occasions that led to missing or delayed coverage of critical mission assurance activities, and that the nature and scope of the problems, and the length of time it took APT to resolve these issues, was a relevant part of its past performance record. Id. at 25. For this reason, the MDA found APT’s performance to be consistent with the solicitation’s definition of a satisfactory quality rating.[9] Id.

APT does not dispute the accuracy of the information that the agency used to determine the quality rating of APT’s performance of the incumbent task order. See Protest Comments at 3-7. Instead, APT argues that it was unreasonable for the agency to assign its proposal a satisfactory quality rating, given that the overall ratings it achieved on the various CPAR and MAR were exceptional or very good. APT asserts that its CPAR and MAR ratings should have resulted in at least a very good past performance quality rating.[10] Id. at 5‑7. Based upon our review of the past performance information and the agency’s evaluation, we find no basis to object to MDA’s evaluation of APT’s past performance.

First, the solicitation’s evaluation criteria for the past performance factor did not obligate the agency to assign the same quality rating that was reflected in an offeror’s CPARs and MARs. Rather, the RTOP permitted the agency to consider the offeror’s CPARs and MARs along with other data in arriving at the task order quality rating. Moreover, the solicitation stated that the agency’s quality rating would be based upon the PPQ definitions, not the definitions in the CPAR or MAR. See RTOP at 13-14; RTOP, attach. 13, PPQ, at 2. Second, the record here reflects that the evaluators found that--despite APT’s record of very good and exceptional performance--the quality of its performance was more consistent with the PPQ rating of satisfactory because of the nature and scope of the problems in its past performance record and the length of time it took APT to resolve the issues. See AR, Tab 9, MDA Technical Evaluation, at 24. We find nothing unreasonable about this conclusion. We conclude that APT has only raised arguments that reflect its disagreement with the agency’s evaluation. Such disagreement is insufficient to render the agency’s evaluation unreasonable. See Glenn Def. Marine-Asia PTE, Ltd., B-402687.6, B‑402687.7, Oct. 13, 2011, 2012 CPD ¶ 3 at 7. For these reasons, we conclude that the record supports the reasonableness of the agency’s evaluation.  (APT Research, Inc., B-409780: Aug 8, 2014)  (pdf)

VSE argues that GSA mistakenly determined that the two Navy [follow-on technical support] FOTS projects identified by VSE were collections of task orders. Protest at 4. The protester contends that each project is a “stand-alone,” single-award contract that provides for “case assignments” (issued as delivery orders) for foreign military sales; VSE argues that the RFP did not address or otherwise restrict the submission of delivery orders. See id. at 3-5; VSE Comments at 4-6.

GSA contends that each Navy FOTS project identified by VSE is not a single contract; rather, because each project identifies an IDIQ contract, as well as multiple task orders issued under the contract, each project is considered a collection of task orders under the terms of the RFP. See AR at 14. The agency points out in this regard that VSE could not have properly claimed many of the points that it claimed for each project, including for the volume of services actually provided, without also counting the underlying orders. Id. at 13. GSA also states that, although the orders under the FOTS contracts were termed delivery orders by the Navy, the FOTS contracts and the orders issued under them to VSE provide for the performance of services such as ship repair, engineering, and warehousing. GSA notes that, as defined by the FAR, orders for services under an IDIQ contract are placed through the issuance of a task order, while supplies are ordered through the issuance of a delivery order. Id. at 13-14; see FAR § 2.101 (delivery order and task order defined).

In reviewing protests challenging the evaluation of proposals, we do not conduct a new evaluation or substitute our judgment for that of the agency but examine the record to determine whether the agency’s judgment was reasonable and in accord with the RFP evaluation criteria. Abt Assocs. Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD ¶ 223 at 4. It is an offeror’s responsibility to submit a well-written proposal, with adequately detailed information which clearly demonstrates compliance with the solicitation requirements and allows a meaningful review by the procuring agency. See, e.g., International Med. Corps, B-403688, Dec. 6, 2010, 2010 CPD ¶ 292 at 7.

We agree with GSA that the two Navy FOTS contracts identified by VSE for its relevant experience are not “single contracts” as that term is used in the solicitation. Both contracts are IDIQ contracts that provided for the issuance of orders for services. In this regard, the contracts include the standard FAR clause set forth at section 52.216-22, Indefinite Quantity, which states, in relevant part, that the contract is an indefinite-quantity contract for supplies or services and that delivery or performance shall be made only as authorized by orders issued in accordance with the contract’s ordering clause. AR, Tab 5-1, VSE Relevant Experience Project No. 2, at 91; Tab 5-2, VSE Relevant Experience Project No. 4, at 77. Additionally, both contracts include the standard FAR clause set forth at section 52.216-18, Ordering, which states that any supplies and services to be furnished under this contract shall be ordered by issuance of delivery orders or task orders. AR, Tab 5-1, VSE Relevant Experience Project No. 2, at 90; Tab 5-2, VSE Relevant Experience Project No. 4, at 76.

Moreover, the record shows that VSE relied upon multiple orders issued under each of the FOTS contracts to substantiate the various points claimed by VSE for its relevant experience. For example, as discussed above, VSE’s proposal highlighted each order in relevant part to substantiate the firm’s performance of work at locations OCONUS. AR, Tab 5-1, VSE Relevant Experience Project No. 2, at 184, 199, 217, 239, 264; Tab 5-2, VSE Relevant Experience Project No. 4, at 126, 135, 141, 151, 166. We also agree with GSA that, notwithstanding that the orders submitted by VSE are entitled “delivery orders,” all of these orders were for services, and not supplies. In other words, these orders are more properly considered task orders, and not delivery orders.

To the extent that VSE did not understand the RFP’s instructions for submitting relevant experience projects, or now believes that GSA should have permitted offerors to submit more than one collection of delivery orders in that regard, the protester had ample opportunity to question and/or protest the solicitation, and VSE’s disagreement with its terms at this point is untimely. 4 C.F.R. § 21.2(a)(1) (2014). The RFP specifically instructed offerors to read the entire solicitation, including all attachments, prior to submitting questions and preparing their offer, and the solicitation was amended 10 times, largely to answer hundreds of questions from offerors concerning the proposal instructions. See RFP at 87, 254-363. Moreover, GSA states that it issued two draft solicitations, about which it answered over 2,000 questions. See Contracting Officer’s Statement at 1.

In any event, even were we to accept VSE’s assertion that it could not reasonably have known that the RFP did not permit offerors to identify more than one collection of “delivery orders” under a single IDIQ contract to establish its relevant experience, VSE has not shown any reasonable possibility of prejudice. VSE has not shown or argued that it would have identified other relevant experience had it been aware of how the agency understood the RFP’s relevant experience requirements with regard to collections of “delivery orders.” Competitive prejudice is an essential element of every viable protest. See, e.g., Geo-Seis Helicopters, Inc., B-294543, Nov. 22, 2004, 2004 CPD ¶ 237 at 4 (protester not competitively prejudiced where record does not suggest, and protester has not argued, that it would have submitted different proposal that would have had a reasonable possibility of award if protester was aware that agency would waive solicitation requirement).

In sum, because the RFP limited the number of collections of task orders that offerors could submit as relevant experience projects, we find that GSA reasonably determined that VSE failed to identify five projects in its proposals, as required by the solicitation, and the protester’s disagreement with the agency’s judgment in that regard does not establish that the agency acted unreasonably. See Citywide Managing Servs. of Port Washington, Inc., B-281287.12, B-281287.13, Nov. 15, 2000, 2001 CPD ¶ 6 at 10-11.  (VSE Corporation, B-408936.5: Aug 25, 2014)  (pdf)

  WSS also challenges the evaluation of its own past performance. Specifically, WSS contends that the agency acted unreasonably when it disregarded two Contractor Performance Assessment Reporting System (CPARS) reports that reflected positive past performance of the incumbent contract, and relied instead on a fee determination for the same period that contained negative information.

In assessing WSS’s past performance, the SEB considered Stoller’s performance under the incumbent contract to be highly relevant. SEB Report at 99. In the SEB’s evaluation, it noted that it had obtained two CPARS reports regarding Stoller’s performance on the incumbent contract. These reports contained exceptional ratings for all rated areas. Id. at 98 n.14. However, the agency found that not only were the reports “erroneously written and approved by a government employee without the authority or knowledge of the contracting officer and contracting officer representative,” they were prepared by a person unfamiliar with the contractor’s performance. Id. Specifically, an individual in DOE’s procurement services office realized that the Stoller contract was missing CPARS reports, so she completed two reports (one for February 2012 to January 2013 and another for January 2013 to April 2013) without having actual knowledge of Stoller’s performance on the contract. As a result, the SEB determined that the CPARS reports on Stoller’s incumbent performance “were not considered valid for purposes of the WSS team’s Past Performance evaluation.” Id.

Instead, the SEB relied on Stoller’s award fee determination for the period of October 2012 to April 2013. SEB Report at 98. In the award fee determination, Stoller received an overall rating of 3.4 out of 4 and a rating of 2.6 out of 4 in the area of management performance. Id.; AR, Tab E.2.E, Stoller Award Fee Determination, at 1, 3. As noted by the agency, the level of detail in this 32-page award fee determination reflected an in-depth knowledge of Stoller’s performance on the contract, with numerous specific examples of problems that had been encountered under the Stoller contract. Thus, the award fee official noted that, despite funding increases, Stoller did not provide the technical support requested by the agency. For example, one program was not able to integrate geochemistry with groundwater flow and transport. Id. at 29. As another example, the award fee determination noted that Stoller’s staff failed to follow the Office of Legacy Management’s policies for review and approval of materials prior to presentations. This resulted in several occasions where agency personnel had to intervene to ensure that presentations received the appropriate level of policy and management review. Id. The award fee determination also stated that many of these issues were raised with Stoller’s management on several occasions without satisfactory resolution. Id.

The award fee determination also found that Stoller’s support for the Church Rock site had not met expectations with regard to providing timely and complete reviews of documents, recommendations on technical issues, insight and foresight into problems, and preparation of correspondence. Stoller Award Fee Determination at 30. As another example, the determination noted security violations, including open windows, open gates, open vehicle doors, and unsecured building doors and doors propped open, most of which occurred after normal working hours. Id. As a result of this information, the SEB assigned WSS a weakness for Stoller’s performance of the incumbent contract. SEB Report at 98. Overall, however, WSS’s past performance was rated good and was assessed one significant strength for highly favorable past performance on other contracts, a weakness related to the award fee determination, as well as one other weakness. Id. at 97.

We find the agency’s evaluation of WSS’s past performance to be reasonable. The record reflects that the previous contracting officer for Stoller’s incumbent contract was a member of the SEB and informed the SEB that the two CPARS reports referenced above were prepared without authorization and without knowledge of Stoller’s performance of the contract. Therefore, the SEB reasonably disregarded those reports and relied instead on the detailed award fee determination which reflected extensive familiarity with Stoller’s performance and was supported by numerous examples. Further, the record indicates that the agency reasonably considered and downgraded WSS with regard to Stoller’s numerous documented performance problems on the incumbent contract. Accordingly, we find no basis on which to question WSS’s overall past performance rating of good.  (CPARS rating by unauthorized person.  (WAI-Stoller Services, LCC; Navarro Research and Engineering, Inc., B-408248.6, B-408248.7, B-408248.8, B-408248.9, B-408248.10, B-408248.11, B-408248.12: May 22, 2014)  (pdf)

The solicitation, a small business set-aside, sought proposals for the award of multiple indefinite-delivery/indefinite-quantity (IDIQ) contracts for new construction, repair, alteration, and related demolition of existing infrastructure. The total maximum value of the acquisition is $240 million for all contracts, with task order minimums and maximums of $3 million and $30 million respectively.

(Sections deleted)

A&D asserts that the agency unreasonably evaluated as not relevant its approximately $8.1 million repair/alteration contract; according to the protester, “[n]owhere in the SOLICITATION does the evaluation criteria state that a project submitted below the threshold of $10M would be rejected as not relevant.” Protest at 3.

Our Office examines an agency’s evaluation of experience to ensure that it was reasonable and consistent with the solicitation’s stated evaluation criteria and applicable statutes and regulations. Herve Cody Contractor, Inc., B-404336, Jan. 26, 2011, 2011 CPD ¶ 27 at 3. An agency has broad discretion, when evaluating offerors’ experience, to determine whether a particular contract is relevant to an evaluation of experience. See All Phase Envtl., Inc., B-292919 et al., Feb. 4, 2004, 2004 CPD ¶ 62 at 3. A protester’s disagreement with the agency’s evaluation judgments, or with the agency’s determination as to the relative merits of competing proposals, does not establish that the evaluation or the source selection decision was unreasonable. Smiths Detection, Inc.; Am. Sci. & Eng’g, Inc., B‑402168.4 et al., Feb. 9, 2011, 2011 CPD ¶ 39 at 6-7.

Here, the chairperson of the agency’s technical evaluation team (TET) states that the TET determined that any project more than 10% less than the target of approximately $10 million would be considered not relevant. Agency Response to Comments, Encl. 1, Declaration of TET Chair, ¶¶ 8‑9. Thus, the protester’s offered contract, valued at nearly 20 percent less than $10 million, was evaluated as not relevant. A&D Proposal Evaluation at 3. The protester asserts that nowhere in the solicitation does the agency announce that a deviation of more than 10 percent below the required contract value will result in an evaluation rating of not relevant. Protester’s Response to Agency’s Response to Comments at 1-2.

The evaluation was reasonable. As an initial matter, in our view, the solicitation requirement that “each project shall be approximately $10,000,000.00 or more in dollar value,” RFP, Evaluation Factors for Award, at 6, clearly indicates that projects not meeting this standard will not be considered. Moreover, we note that even in the absence of such clear language, an agency properly may take into account specific, albeit not expressly identified, matters that are logically encompassed by, or related to, the stated evaluation criteria. Independence Constr., Inc., B-292052, May 19, 2003, 2003 CPD ¶ 105 at 4. Where, as here, a solicitation announces that the evaluation of the relevance of a prior contract will consider its size, we see nothing inherently unreasonable in the agency’s establishment of a minimum relevance value; establishing a threshold value is sufficiently related to the relevance criteria. See AMI-ACEPEX, Joint Venture, B‑401560, Sept. 30, 2009, 2009 CPD ¶ 197 at 4. Nor do we see anything unreasonable in the agency’s determination to define the solicitation standard‑‑“approximately $10,000,000.00”‑‑as encompassing no more than a 10% deviation. In any case, the value of the repair/alteration contract experience cited in A&D’s proposal ($8,066,021) was only 80.66% of the $10 million target established by the solicitation; there simply is no basis for finding unreasonable the agency’s conclusion that the value of the contract identified by the protester was outside the range encompassed by the “approximately” limitation. See Lexis-Nexis, B-260023, May 22, 1995, 95-2 CPD ¶ 14 at 7 (the propriety of an evaluation turns not on internal evaluation standards, but on whether the evaluation ultimately is carried out in accordance with the stated evaluation criteria). We conclude that the agency reasonably elected not to include A&D’s proposal in the second phase of the competition.  (A&D General Contracting, Inc., B-409429: Apr 17, 2014)  (pdf)

A&D protests that the requirement that an offeror show four relevant projects to receive a substantial confidence assessment was an unstated evaluation criterion. Protest at 2; Comments at 3. The protester complains that the RFP only stated a maximum number of projects that an offeror could provide, and did not identify a minimum number of projects that must be provided. See Comments at 2-4. A&D contends that its proposal should have received a substantial confidence rating based upon its three very relevant rated projects. See id. at 8-9.

The Corps disagrees that considering the number of relevant projects provided by an offeror reflects the use of an unstated evaluation criterion. See Supp. AR at 1. The Corps also contends that the SSEB and contracting officer reasonably concluded from its review of A&D’s three relevant projects that the protester had demonstrated only a reasonable expectation that the protester would successfully perform, which warranted a satisfactory confidence rating. See AR at 7.

As a general matter, the evaluation of an offeror’s past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. MFM Lamey Group, LLC, B-402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10. Where a protester challenges an agency’s past performance evaluation and source selection, we will review the evaluation and award decision to determine if they were reasonable and consistent with the solicitation’s evaluation criteria and procurement statutes and regulations, and to ensure that the agency’s rationale is adequately documented. Falcon Envtl. Servs., Inc., B-402670, B-402670.2, July 6, 2010, 2010 CPD ¶ 160 at 7. A protester’s disagreement with the agency’s judgment concerning the merits of the protester’s past performance does not establish that the evaluation was unreasonable. Sam Facility Mgmt., Inc., B–292237, July 22, 2003, 2003 CPD ¶ 147 at 3.

Here, the record supports the reasonableness of the agency’s assignment of a satisfactory confidence rating to A&D’s past performance. As noted above, A&D had only three relevant projects supporting its past performance rating, where other offerors with higher past performance confidence ratings identified a greater number of relevant projects. Although A&D may disagree with the agency’s judgment as to the merits of its past performance, this does not show that the Corps acted unreasonably. In this regard, we do not agree with A&D that reserving the substantial confidence past performance rating for offers that identified four or more relevant projects reflected the use of an unstated evaluation criterion. Offerors were specifically informed that the agency’s past performance assessment would include consideration of the relevance of the firms’ identified projects. Moreover, the RFP explicitly stated that more relevant, recent projects would be considered a strong predictor of potential successful contract performance. RFP amend. 1 § 00 11 00 ¶ 8.2.1. Consideration of how many relevant projects were identified is logically encompassed within this stated review. See Commissioning Solutions Global, LLC, B-403542, Nov. 5, 2010, 2010 CPD ¶ 272 at 4.

The protest is denied.  (A&D General Contracting, Inc., B-409296: Feb 24, 2014)  (pdf)

Insect Shield challenges the agency’s past performance evaluation. The protester contends that UNICOR applied unstated evaluation criteria by improperly considering the size of the past performance contracts and considering whether Insect Shield had performed as a prime contractor, among other things.

Where, as here, a solicitation contemplates the evaluation of vendors’ past performance, the contracting agency has the discretion to determine the relevance and scope of the performance history to be considered, and our Office will not question the agency’s judgment unless it is unreasonable or inconsistent with the terms of the solicitation or applicable procurement statutes and regulations. National Beef Packing Co., B-296534, Sept. 1, 2005, 2005 CPD ¶ 168 at 4; Sam Facility Mgmt., Inc., B-292237, July 22, 2003, 2003 CPD ¶ 147 at 3. A protester’s disagreement with the agency’s judgment, without more, does not establish that an evaluation was unreasonable. Sam Facility Mgmt., Inc., supra, at 3.

We do not agree that UNICOR applied unstated evaluation criteria when it considered the size of the protester’s past performance contracts and whether the protester had performed as the prime contractor. An agency properly may take into account specific matters that are logically encompassed by, or related to, the stated evaluation criteria, even when they are not expressly identified as evaluation criteria. MINACT, Inc., B-400951, Mar. 27, 2009, 2009 CPD ¶ 76 at 3. Here, the RFP specifically stated that offerors were to submit relevant contracts for the agency’s past performance evaluation, and requested information (on the business management questionnaires) about the dollar value of the contracts the offeror had listed as references, and the percentage of the work the offeror had performed. RFP at 14-15; AR, Tab 4, Protester’s Proposal at 18. In this regard, we think that it is both illogical and unreasonable to presume that an agency will pay no attention to the size and similarity of past contracts in its evaluation, since such factors are germane to the relevance of the past performance information. J. A. Jones Grupo de Servicios, SA, B-283234, Oct. 25, 1999, 99-2 CPD ¶ 80 at 7. Given this, we find the agency’s consideration of the dollar value of the contracts and the offeror’s role (as prime or subcontractor) in performing prior contracts consistent with the past performance evaluation.

Insect Shield nevertheless argues that the agency’s past performance evaluation should have taken into account the value of the contracts on a per-year basis, and not merely focused on the total value of the contracts. In this regard, Insect Shield notes that Pine Belt’s largest contract (for $8.5 million) only amounted to a per-year value of $1.2 million, while one of Insect Shield’s contracts, which had a value of approximately $1 million, was performed in only 6 months.

While the protester clearly believes its proposed method of comparison to be superior, it has not shown the agency’s approach was unreasonable. The protester’s contention that the agency should have considered the value of the contracts on a per-year basis constitutes nothing more than disagreement with the agency’s judgment. In this regard, we find reasonable the agency’s assessment of some risk associated with an award to Insect Shield, based on the low value of Insect Shield’s contracts (none of which was more than $1.7 million), relative to value of the contract here (approximately $10 million).

Finally, we find no merit to the protester’s argument that it was improper for the agency to consider whether Insect Shield had performed as a prime contractor with the government, or as a subcontractor. In this regard, the protester notes that the nature of the services being procured here (a finishing treatment to a garment) means that offerors “will for the most part be subcontractors to private industry primes.” Protester’s Comments at 4. We do not disagree with this general observation, but we note that the record shows that both of these offerors actually have past performance as prime contractors. The question for our Office is whether the contracting officer acted unreasonably in comparing Insect Shield’s higher price with Pine Belt’s lower price, and concluding that--based on evaluated differences in their generally excellent past performance (i.e., that Pine Belt has more experience as a prime contractor than Insect Shield)--there is no reason to pay Insect Shield’s
higher price. While Insect Shield may disagree, its disagreement does not mean the agency’s exercise of its judgment in this area was unreasonable.  (Insect Shield Manufacturing, LLC B-408067.3, Aug 8, 2013)  (pdf)

TIMCO argues that the Coast Guard’s evaluation of its past performance was unreasonable because the agency failed to consider the protester’s performance as one of the two incumbent contractors for [progressive structural inspections] PSI services. TIMCO contends that the agency ignored or failed to reasonably consider past performance about which the agency had firsthand knowledge. We agree.

As a general matter, the evaluation of an offeror’s past performance is within the agency’s discretion. We will question the evaluation conclusions where they are unreasonable or undocumented. Clean Harbors Envtl. Servs, Inc., B-296176.2, Dec. 9, 2005, 2005 CPD ¶ 222 at 3. The critical questions are whether the evaluation was conducted fairly, reasonably, and in accordance with the stated evaluation terms, and whether it was based on relevant information sufficient to make a reasonable determination of the firm’s overall past performance. Id. An agency’s past performance evaluation is unreasonable where the agency fails to give meaningful consideration to all the relevant past performance information it possesses. DRS C3 Sys., LLC, B-310825, B-310825.2, Feb. 26, 2008, 2008 CPD ¶ 103 at 22.

As discussed above, the RFP here sought to combine the agency’s requirements for PSIs for its HC-130H and HC-130J aircraft, which had been performed under separate contracts by TIMCO and DRS, respectively. TIMCO submitted information about eight of its prior contracts, including the incumbent contract for HC-130H requirements. Four of these contracts were performed by TIMCO, and the other four were performed by TIMCO’s teaming partners, [deleted]. AR, Tab E, TIMCO Proposal, Vol. II, at II-vii. In its proposal, TIMCO stated that it had sent the required questionnaires for each contract to each of its customers. See id. at II-7. Regarding the incumbent contract for the HC-130H requirements, TIMCO provided an “in-progress” contractor performance report for the period from September 2005 to July 2006. The protester explained that “[f]or the past two [Coast Guard PSI] contracts, TIMCO has received one evaluation from the [Coast Guard].” Id. at II-11. The progress report reflects that for the criteria being evaluated, TIMCO received an excellent rating for quality of product or service, an excellent rating for cost control, an outstanding rating for timeliness of performance, and an outstanding rating for business relations.

As noted, the Coast Guard assigned TIMCO’s proposal a green/acceptable rating and a moderate risk rating for past performance. This evaluation was based on two returned questionnaires. One was for a TIMCO contract with Northrop Grumman and the United States Air Force for KC-10 aircraft, and the other was for a [deleted] contract with the Coast Guard for propeller overhauls. AR, Tab N, PEG Report, at 33.

The agency evaluators noted that although TIMCO’s proposal cited other examples of past performance, “[i]t was difficult to judge the reliability of TIMCO’s past performance assertions without further customer feedback.” AR, Tab N, PEG Report, at 33. Specifically, with regard to the TIMCO’s performance for the HC-130H PSI services, the agency stated as follows:

In an “in-progress” Contractor Performance Report from July 20, 2006, TIMCO received excellent ratings from the Government; however, some of the comments caused concerns and raised potential risks. For example, the Government stated, “Labor costs are controlled well but materials management requires inordinate amounts of costly Government intervention and assistance.” This statement raises concerns due to the amount of material management duties required in the SOW. They also received positive statements such as, “From four vendors over four years and 25 aircraft, this is the first vendor to receive incentive payments for early delivery.” It appears this Contractor Performance Report was never officially submitted.

Id.

With regard to the risk evaluation, the evaluators concluded that “[t]here is moderate risk that TIMCO can perform the proposed effort since there was minimal customer feedback to substantiate their past performance and the fact that no information was provided to substantiate relevant past performance for the facility in Lake City, FL.” Id.

We find that the Coast Guard’s evaluation of TIMCO’s past performance was unreasonable. In this regard, the record shows that the Coast Guard’s evaluation of TIMCO’s performance of the incumbent contract (which was also for the Coast Guard) relied solely upon the 2006 “in-progress” report, rather than any recent performance information. Id. In addition, the agency raised concerns and assessed a moderate risk rating due to the “lack of customer feedback” and the absence of “substantiat[ion]” regarding TIMCO’s performance. Id. In our view, the Coast Guard’s decision not to substantiate TIMCO’s performance on a contract for essentially the same service as here (i.e., HC-130H PSI work also performed for the Coast Guard) was improper.

In answer to this contention, the Coast Guard argues that the evaluators fully considered TIMCO’s past performance on the incumbent contract, despite not having a questionnaire, and despite not addressing the issue, but nonetheless rated the proposal acceptable. Contracting Officer Statement at 4-5. The agency, however, does not explain how the plain statements in the evaluation record support its contention. As noted above, the evaluators found that questionnaires were returned for only two of TIMCO’s past performance references, neither of which was for its incumbent work on the Coast Guard’s HC-130H aircraft. AR, Tab N, PEG Report, at 33. To the extent the agency considered TIMCO’s past performance regarding HC-130H PSIs, the evaluation was limited to the 2006 “in-progress” report. Id. The agency also stated that the lack of questionnaires made it “difficult to judge the reliability of TIMCO’s past performance assertions without further customer feedback.” Id. at 33. In addition, although the Coast Guard’s contemporaneous evaluation record acknowledges that “this Contractor Performance Report was never officially submitted,” id. at 33, it offers no explanation about why the Coast Guard reference did not respond, or why Coast Guard evaluators could not have provided an updated assessment of TIMCO’s performance on the HC-130H PSI effort.

While there is no legal requirement that an agency consider all past performance references, some information is simply “too close at hand” to require offerors to shoulder the inequities that spring from an agency’s failure to obtain and consider information. Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard Recovery Assistance, Joint Venture, B-401679.4 et al., Mar. 10, 2010, 2010 CPD ¶ 77 at 8; Int’l Bus. Sys., Inc., B-275554, Mar. 3, 1997, 97-1 CPD ¶ 114 at 5. For example, our Office has held that an agency may not ignore contract performance by an offeror involving the same agency, the same services, and the same contracting officer, simply because an agency official fails to complete the necessary assessments or paperwork. Int’l Bus. Sys., Inc., supra at 4-5. For these reasons, we think the record shows that the agency either ignored or did not fully consider TIMCO’s performance of the incumbent requirements for the Coast Guard regarding the HC-130H aircraft.[9] We sustain this basis of protest.  (Triad International Maintenance Corporation, B-408374, Sep 5, 2013)  (pdf)

Advanced Computer challenges the agency’s evaluation of its past performance, complaining that GSA failed to take into consideration the circumstances surrounding the termination of the firm’s four orders with the Air Force. Protest at 2. Specifically, the protester states that its failure to deliver was because its supplier of the items was “punishing” Advanced Computer for offering the items at prices below the manufacturer’s GSA schedule prices. Id. The protester contends that its failure to perform these four orders cannot be considered by GSA because Advanced Computer acted properly in refusing “to collude to fix prices.” Advanced Computer also contends that the manufacturer and its supplier “ultimately paid GSA $48 million as a settlement for such practices.” Protest at 2.

The evaluation of past performance is a matter of agency discretion, and we will review the evaluation only to ensure that it was reasonable and consistent with the solicitation’s stated evaluation criteria and applicable statutes and regulations. Guam Shipyard, B-311321, B-311321.2, June 9, 2008, 2008 CPD ¶ 124 at 3. The evaluation by its very nature is subjective; an offeror’s disagreement with the agency’s evaluation judgments does not demonstrate that those judgments are unreasonable. SDV Telecomms., B-279919, July 29, 1998, 98-2 CPD ¶ 34 at 2.

Based on our review of the record, we see nothing improper about GSA’s evaluation of Advanced Computer’s past performance. The record shows that the contracting officer reviewed available information in the PPIRS for both vendors, consistent with the terms of the RFQ. The contracting officer also contacted the Air Force and confirmed the termination of Advanced Computer’s orders for cause. See Contracting Officer’s Statement at 5. Although the protester argues that it should not have been held responsible for its failure to deliver, Advanced Computer’s quotation provided no explanation of the surrounding circumstances for this negative past performance information, nor has the protester explained why the agency should have been aware of this information such that GSA was required to consider it as part of the evaluation.

In summary, while we acknowledge that the protester views GSA’s assessment to be unfair, and while we acknowledge that the Air Force appears to hold the protester blameless for these past performance issues, we see no basis for concluding that GSA acted improperly in reaching its assessment. As a result, we deny Advanced Computer’s challenge to the reasonableness of GSA’s evaluation.  (Advanced Computer Concepts, B-408084, May 30, 2013)  (pdf)

An agency is required to consider, determine, and document the similarity and relevance of an offeror’s past performance information as part of its past performance evaluation. The Emergence Group, B-404844.5, B-404844.6, Sept. 26, 2011, 2012 CPD ¶ 132 at 6; see Federal Acquisition Regulation §15.305(a)(2). As a general matter, since an agency is responsible for defining its needs and the best method for accommodating them, the evaluation of an offeror’s past performance, including the agency’s determination of the relevance and scope of an offeror’s performance history to be considered, is a matter within the discretion of the contracting agency. MFM Lamey Group, LLC, B-402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10; Yang Enters., Inc., Santa Barbara Applied Research, Inc, B-294605.4 et al., April 1, 2005, 2005 CPD ¶ 65 at 5.

On this record, there is nothing to indicate that the agency abused its discretion. In this regard, the agency explains that its technical experts used their knowledge of the agency’s requirements to establish minimums by region for each of the relevance criteria of scope, magnitude of effort, and complexity. AR at 4. Specifically, it calculated the requirements necessary to provide service to a deploying Army brigade (the typical size of deploying units), considering the needs for accountability and maintenance, as well as the need to support multiple locations. Id.; Contracting Officer’s Statement at 2. Based on these calculations, the agency arrived at the minimum relevant experience requirements for the agency to be confident that an offeror meeting them would be capable of meeting the ILSS-2 requirements.

For example, based on the agency’s experience, with regard to the magnitude of effort for the 401st Region (Afghanistan), the $20 million per year minimum represents the average 1-year cost of service contracts in Southwest Asia (Iraq, Kuwait, and Afghanistan); the 200 hand receipts represent the minimum required for two to three brigades worth of equipment; and 100 employees is the average of five to seven property book teams. AR, Tab 112-1 (Rationales).

Likewise, with regard to the complexity of the 406th Region’s work, the number of UICs is based on a typical infantry brigade combat team (the most commonly deployed element); the minimum number of sites was based on the agency’s confidence that the ability to handle two or more locations indicated the capability to support multiple locations; and the minimum of experience with three types of equipment indicated an offeror’s versatility and capability to handle the multiple types of ILSS-2 equipment. Id. Further, while the magnitude and complexity of HTSI’s incumbent contracts for the 401st and 406th Regions are greater than those in the amended RFP’s criteria, the agency explains that its requirements will continue to decrease throughout the life of the ILSS-2 contract, especially given the current federal budget constraints. Contracting Officer’s Statement at 4.

Under these circumstances, HTSI’s protest furnishes no basis for finding that the agency acted unreasonably in exercising its discretion in setting minimums for determining the relevance of past performance. As we have previously stated, there simply is no requirement that an agency’s determination of relevance be based on the same levels for scope, magnitude, and complexity as those under the incumbent contracts or the work anticipated by the solicitation. See e.g., AMI-ACEPEX, Joint Venture, B-401560, Sept. 30, 2009, 2009 CPD ¶ 197 at 4 (nothing inherently unreasonable in agency’s determination that contract valued at approximately half the size of first option year was relevant); KIC Dev., LLC, B-309869, Sept. 26, 2007, 2007 CPD ¶ 184 at 3 (agency reasonably found offeror’s past performance relevant where smaller prior contracts, though not equivalent in magnitude and scope, were sufficiently similar to current requirements).  (Honeywell Technology Solutions, Inc., B-407159.4, May 3, 2012)  (pdf)

With respect to American’s challenges to the agency’s evaluation of Bluewater’s past performance, there is no dispute that all of the alleged deficiencies cited by American occurred subsequent to the closing time of the RFP. Thus, these issues were outside the timeframe of the past performance evaluation established by the solicitation, which provided for examining past performance “[f]or the period two years prior to the solicitation closing date.” RFP at 184. Nonetheless, American asserts that the agency was required to consider this information in assigning Bluewater’s past performance rating where the record demonstrates that the agency was aware of Bluewater’s more recent performance issues. American argues that the knowledge of Bluewater’s recent performance deficiencies was therefore “too close at hand” for the agency to ignore in the past performance evaluation. We disagree.

In reviewing an agency's evaluation of proposals, our Office will question the agency's evaluation only where it violates a procurement statute or regulation, lacks a reasonable basis, or is inconsistent with the stated evaluation criteria for award. Birdwell Bros. Painting and Refinishing, B-285035, July 5, 2000, 2000 CPD ¶ 129 at 5. Here, we find nothing in the agency’s evaluation unreasonable or inconsistent with the terms of the solicitation.

The RFP in this case provided that offerors’ past performance would be assessed “[f]or the period two years prior to the solicitation closing date,” which in this case covered the time period between September 23, 2008, and the closing date of September 22, 2010. RFP at 184. Where the Bluewater performance deficiencies identified by American fall outside of that time period, there was nothing improper about the agency’s decision not to consider these issues. See FR Countermeasures, Inc., B-295375, Feb. 10, 2005, 2005 CPD ¶ 52 (agency not required to consider past performance information outside of the time period set forth in the solicitation, even where solicitation reserved the agency’s right to do so).  (American Apparel, Inc., B-407399.2, Apr 30, 2013)  (pdf)

FNM challenges the agency’s evaluation of its past performance, asserting that the Army unreasonably failed to ascertain delivery information for the additional delivery orders--the DLA orders for replacement M16 barrels--identified by the contract specialist but not referenced in FNM’s proposal, and that the SSA unreasonably ignored information regarding the protester’s past performance.

Our Office has recognized that, in certain limited circumstances, an agency evaluating an offeror’s proposal has an obligation (as opposed to the discretion) to consider “outside information” bearing on the offeror's past performance when it is “too close at hand” to require offerors to shoulder the inequities that spring from an agency’s failure to obtain and consider the information. See, e.g., International Bus. Sys., Inc., B-275554, Mar. 3, 1997, 97-1 CPD ¶ 114 at 5. This doctrine, however, is not intended to remedy an offeror’s failure to include information in its proposal. Paragon Technology Group, Inc., B-407331, Dec. 18, 2012, 2013 CPD ¶ 11 at 6 n.8. Where an offeror is in control of the past performance information contained in its proposal--and not reliant on third parties to submit that information--it exercises its own judgment as to the information that the agency should consider. Under those circumstances, there is “no inequity” in an agency’s decision to base its evaluation on an offeror’s proposal as written, instead of supplementing the proposal with the agency’s understanding of the offeror’s performance under other contracts not cited by the offeror. See L-3 Servs., Inc., B-406292, Apr. 2, 2012, 2012 CPD ¶ 170 at 12 n.10.

Here, the RFP required offerors to furnish up to three past performance references of the offeror’s choosing “based on identification of your most recent and relevant contracts/delivery orders.” RFP § L.5.1.3. As noted above, the agency, on its own initiative, undertook a review of other delivery orders and contracts performed by FNM. However, the contracting specialist was unsuccessful in his efforts to ascertain whether other very relevant delivery orders had actually been performed, and as a result, those delivery orders were not considered in the agency’s past performance evaluation.

FNM asserts that, “[o]nce the Army evaluator identified [very] relevant past performance information and concluded the verification of the information was appropriate, it was unreasonable for the evaluator to fail to contact the very person in the best position to verify the information.” Protester’s Comments, Mar. 19, 2013 at 1. We disagree. As noted by the protester, our review looks to see whether an agency proceeded in a reasonable and prudent manner when reviewing the manner and conduct of an agency in contacting or choosing not to contact references listed in offerors’ proposals. Acepex Mgmt. Corp., B-279173.5, July 22, 1998, 98-2 CPD ¶ 128 at 8. Here, however, the unavailable delivery information concerned delivery orders that were not identified by FNM in its proposal. We see no basis for concluding that the contracting activity here nevertheless was required to take additional steps, such as contacting officials charged with administering the additional delivery orders identified by the contract specialist, to determine the relevance of such delivery orders that were not included in the protester’s proposal.

FNM also asserts that the SSA unreasonably failed to consider knowledge of the protester’s past performance on a prior, very relevant contract. In this regard, FNM points to debriefing slides issued by the Army Contracting Command with respect to the competition under solicitation W56HZV-10-R-0593, for M4 carbines, which resulted in an April 20, 2012 award. In that competition, the agency rated the protester’s past performance as “very relevant” on contract No. W52H09-08-D-0121, under which FNM was to deliver a minimum of 26,275 M16A3 and M16A4 rifles, with an average monthly shipment of 5,764 rifles. Protest, Exh. A, at 30. The source selection evaluation board (SSEB) chairman for solicitation W56HZV-10-R-0593 was also the source selection authority for this procurement.

Once again, however, FNM asks that we shift responsibility to the agency for finding information about another contract that FNM could have identified in its proposal, but did not. For the reasons above, we see no basis for concluding that the agency was required to search for information that FNM did not refer to in its proposal. Moreover, the facts of this case do not suggest that the SSA ignored “personal knowledge” of the contract referenced in the 2012 debriefing slides. Here, the SSA’s role in the prior procurement--that of chairman of the SSEB--ended approximately six months before the challenged evaluation. In that intervening six months, the SSA transferred out of the group which conducted the prior procurement, and following that transfer he was involved primarily in vehicle, not small arms, procurement. Declaration of SSA at ¶ 3. The SSA states, and FNM has not refuted, that “I had no specific knowledge or recollection of FNM’s past performance under any other contract.” Id. at ¶¶ 4, 7.

In summary, the record here shows that Colt’s proposal cited a 3-year contract under which Colt had produced 227,134 M4/M4A1 carbines, with deliveries averaging 6,309 carbines per month and with peak months as high as 12,000 carbines delivered. FNM has failed to cite any relevant experience with such sustained, high delivery rates. In our view, the agency reasonably evaluated Colt’s proposed past performance as superior when the agency viewed Colt’s proposed past performance references as “more relevant with regard to the magnitude of deliveries” and thus providing slightly greater confidence of successful contract performance. Source Selection Decision at 7.  (FN Manufacturing LLC, B-407936, B-407936.2, B-407936.3, Apr 19, 2013)  (pdf)

NSR objects to the agency’s evaluation of its past performance, arguing that it should have received higher than a “limited confidence” rating. In this regard, NSR contends that the Air Force’s reliance on negative information concerning the joint venture’s contract at Lackland AFB was unreasonable, where the joint venture was found to have satisfactory performance overall. NSR also complains that the Air Force failed to consider positive past performance information contained in the CPARS for NSR and/or INC, and improperly considered negative performance information related to contracts not relevant to the work here.

The evaluation of an offeror’s past performance is a matter of agency discretion, which we will not find improper unless unreasonable or inconsistent with the solicitation’s evaluation criteria, National Beef Packing Co., B-296534, Sept. 1, 2005, 2005 CPD ¶ 168 at 4; Command Enters., Inc., B-293754, June 7, 2004, 2004 CPD ¶ 166 at 4, nor will we substitute our judgment for reasonably based evaluation ratings. MFM Lamey Group, LLC, B-402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10. An offeror’s mere disagreement with an agency’s evaluation judgments does not demonstrate that those judgments are unreasonable. FN Mfg., LLC, B-402059.4, B-402059.5, Mar. 22, 2010, 2010 CPD ¶ 104 at 7.

Here, the record shows that the Air Force’s evaluation of NSR’s past performance was reasonable. With respect to the NSR/INC joint venture’s performance at Lackland AFB, the Air Force received information from questionnaire responses and Lackland AFB personnel that indicated that the joint venture had marginal or unacceptable quality control for half of the contract term. See AR, Tab 15, NSR Past Performance Evaluation, at 23. Although the second questionnaire rated the joint venture’s quality control performance as satisfactory overall, that questionnaire continued to indicate the joint venture’s performance problems in specific areas. Moreover, the personnel at Lackland AFB informed the agency, after receipt of the second questionnaire response, that the joint venture had performance problems on this contract and required significant government supervision. Although NSR disagrees that it had performance problems, it does not show that any of this information is in error.

NSR also complains that the agency considered negative past performance information the agency found in the CPARS for two contracts that INC performed, where the agency itself found that these contracts were “not relevant” under at least one of the considerations established by the RFP for determining relevancy. Specifically, NSR notes that Contract No. -4250 (INC’s contract with the Navy) was found to be not relevant under the magnitude/complexity and contract value considerations, and that Contract No. -0002 (INC’s contract with GSA) was found to be not relevant under the magnitude/complexity consideration. AR, Tab 20, Past Performance Recency and Relevancy Worksheet, at 9, 10.

We do not agree that the Air Force could not consider these contracts in its past performance evaluation. In this regard, the RFP did not state that the agency would only consider contracts to be relevant if they were deemed to be at least “somewhat relevant” under every relevancy consideration. Rather, the RFP provided for an integrated assessment of a contract’s relevancy, and stated an order of importance for these considerations. Here, the PCAG found the contracts relevant considering all of the stated considerations, and in particular considering the most important “scope of work” consideration. Although NSR disagrees with this judgment, it does not show it to be unreasonable.

NSR also complains that the Air Force in its review of the CPARS failed to credit NSR for other, positive, CPARS evaluations. In this regard, NSR identifies in its protest CPARS evaluations for three other contracts that NSR believes the agency should have considered. The contracting officer explains that he searched for the CPARS for relevant contracts for NSR, INC, and NSR/INC joint venture by cage codes and contractor names, and states that he did not find any other relevant CPARS evaluations. See Contracting Officer’s Statement at 18. The record provides no basis to question the contracting officer’s statement in this regard. In any event, we find no merit to this complaint, given the past performance concerns that were otherwise identified for NSR.  (NSR Solutions, Inc., B-406894, Sep 20, 2012)  (pdf)

Supreme contends that the agency improperly aggregated the value of prior contracts listed in ANHAM’s proposal when it evaluated ANHAM’s experience/past performance under the size and complexity element. 2d Supp. Protest at 65-78, 85-86, 95-97; Supp. Comments at 65-79. Supreme further contends that none of the contracts listed in ANHAM’s proposal individually met an 85 percent threshold for contracts of “similar size” that was specified in the solicitation’s size and complexity element evaluation criteria. 2d Supp. Protest at 67-69; Supp. Comments at 76-78. Based on these contentions, Supreme argues that the agency’s determination to assign ANHAM’s proposal the highest-possible evaluation rating under the size and complexity element was unreasonable. 2d Supp. Protest at 68-69; Supp. Comments at 66, 70-75. We agree.

Our Office examines an agency’s evaluation of experience and past performance to ensure that it was reasonable and consistent with the solicitation’s stated evaluation criteria and applicable statutes and regulations. Herve Cody Contractor, Inc., B-404336, Jan. 26, 2011, 2011 CPD ¶ 27 at 3; JVSCC, B-311303.2, May 13, 2009, 2009 CPD ¶ 138 at 5.

As described above, factor 1 (experience/past performance) was the most important technical evaluation factor and consisted of four subfactors, one of which was subfactor A, experience. RFP 262. Subfactor A, which was one of two equally-most-important subfactors, included two elements, the most important of which was element 1, size and complexity. Id. The solicitation stated that with respect to this element:

The Government will evaluate the offeror’s experience in fulfilling similar requirements of similar size (85%-100%), and complexity for customers in a prime vendor/regular dealer capacity on an individual contract basis only for its most relevant (in terms of size and complexity) provided contracts.

Id. at 264 (emphasis added). Related to this solicitation provision, the agency, in its evaluation of proposals under the size and complexity element, documented that “[t]he evaluation defined relevant in terms of size and complexity to be 85%-100% of the estimated value or $1.55 [billion] annually (85% of $1.82 [billion]).” AR, Vol. III, Tab 2, ANHAM Final Technical Report, at 7; AR, Vol. III, Tab 1, Supreme Final Technical Report, at 7 (emphasis added).

The documentation of the agency’s evaluation of ANHAM’s proposal under the size and complexity element included a table showing the contracts that ANHAM listed in its proposal for evaluation under factor 1. AR, Vol. III, Tab 2, ANHAM Final Technical Report, at 8-9. The table also showed the number of delivery points for each contract and the following annual contract values: [DELETED]. Id. at 8. From these values, the agency calculated an aggregated contract value for ANHAM of [DELETED] billion. Id. at 7. The agency found that this aggregate value “exceeds the solicitation requirement of $1.82 [billion] annually.” AR, Vol. III, Tab 2, ANHAM Final Technical Report, at 7. The agency then assigned ANHAM’s proposal the highest available rating (outstanding) under the size and complexity element, finding that “[ANHAM,] when combined with a teaming approach[,] presents evidence of providing full line food service as a prime vendor for customers whose size and complexity exceeds the contract dollar value and number of delivery stops required by this solicitation.” AR, Vol. III, Tab 2, ANHAM Final Technical Report, at 7. Thus, the record reflects that the agency based ANHAM’s outstanding rating under the size and complexity element on the aggregated value of the contracts under evaluation.

In response to Supreme’s claim that the aggregation of contract values was improper, the agency asserts that based on an integrated assessment of ANHAM’s proposal, the evaluation was reasonable. See Supp. AR at 23-27. The agency also asserts that the solicitation’s evaluation criteria for the size and complexity element should be read to mean that the agency “would consider only the most relevant contracts on an individual basis” and did “not prohibit aggregating contracts during the evaluation.” Id. at 25-26.

Here, the agency acknowledges that the solicitation contemplated the evaluation of the most relevant contracts on an individual basis. The contemporaneous record, however, does not reflect any consideration by the agency that none of the contracts listed in ANHAM’s proposal met the 85 percent threshold on an individual basis. Rather, the record shows that the predicate for ANHAM’s proposal rating under this evaluation element was an aggregation of the contracts listed in ANHAM’s proposal. In the absence of documentation to show that the agency recognized that none of the contracts individually met the threshold, we find the agency’s determination to assign ANHAM’s proposal the highest available rating under this evaluation element to be unreasonable. Accordingly, we sustain this basis of protest.  (Supreme Foodservice GmbH, B-405400.3, B-405400.4, B-405400.5, Oct 11, 2012)  (pdf)

BSI contends that the agency did not evaluate its past performance in accordance with the terms of the solicitation. Specifically, BSI argues that, with respect to the contract that the agency found was not relevant, the agency ignored the provision in the RFP providing that “[i]n determining relevancy for individual contracts, consideration will be given to the effort, or portion of the effort, being proposed by the offeror, teaming partner, or subcontractor whose contract is being reviewed and evaluated.” See RFP amend. 2, at 15. BSI states that it proposed to perform contract management functions (among other responsibilities), while Accent was proposed to perform quality control and training (and other responsibilities). BSI contends that, in determining the relevance of its construction services contract, the Air Force should have considered BSI’s performance of contract management functions.

Where a dispute exists as to the meaning of a particular solicitation provision, our Office will resolve the matter by reading the solicitation as a whole and in a manner that gives effect to all of its provisions; to be reasonable, an interpretation must be consistent with such a reading. ArmorWorks Enter. LLC, B-405450, Oct. 28, 2011, 2011 CPD ¶ 242 at 3.

Here, we find BSI’s reading of this solicitation provision to be unreasonable, either read alone or in considering the solicitation as a whole. This provision does not state, as BSI maintains, that the agency was required to find a contract for unrelated services (here construction services) to be relevant as compared to the scope of the work being solicited here (custodial services). Moreover, such a reading ignores the remainder of the RFP that plainly provides that the agency would consider as the most important relevance consideration the type of services provided. See RFP amend 2, at 12. Given this, we find nothing unreasonable with the agency’s determination that BSI’s construction contract was not relevant to the custodial services being procured here.  (Building Solutions, Inc., B-406894.2, Sep 24, 2012)  (pdf)

GDIT argues that the Army failed to properly assess relevance when evaluating past performance, and this failure undermined the source selection decision. GDIT also argues that the SSA unreasonably relied only on the offerors’ adjectival performance confidence assessment ratings, which led her to conclude that SAIC’s past performance was equal to GDIT’s past performance. According to GDIT, the evaluation record shows that GDIT’s past contracts are more relevant and therefore deserved greater consideration.

An agency’s evaluation of past performance, including its consideration of the relevance, scope, and significance of an offeror’s performance history, is a matter of discretion which we will not disturb unless the agency’s assessments are unreasonable, inconsistent with the solicitation criteria, or undocumented. L-3 Sys. Co., B-404671.2, B-404671.4, Apr. 8, 2011, 2011 CPD ¶ 93 at 4; Family Entm’t Servs., Inc., d/b/a IMC, B-291997.4, June 10, 2004, 2004 CPD ¶ 128 at 5. A protester’s mere disagreement with such judgment does not provide a basis to sustain a protest. Birdwell Bros. Painting & Refinishing, B-285035, July 5, 2000, 2000 CPD ¶ 129 at 5. Our review of the record shows that GDIT’s allegations have no merit.

Offerors could submit up to five recent and relevant contract references for evaluation. RFP § L.3.3. The agency’s evaluation of proposals, and assignment of performance confidence assessment ratings, was to focus on performance that was relevant to the contract requirements. RFP § M.6.3.

Both SAIC and GDIT submitted five references. Consistent with the RFP’s requirements, the past performance evaluation team first evaluated the information to determine the relevance of the past efforts. After this determination, the team reviewed the past performance information it could gather to determine its quality and usefulness in assigning the performance confidence assessment ratings. The agency evaluated the information in the proposals, made follow-on telephone calls to references, and gathered available Contractor Performance Assessment Reporting System (CPARS) and Past Performance Information Retrieval System (PPIRS) data. AR, Exh. 13-9, Past Performance Evaluation Report, at 2-4.

For GDIT, the agency found that three contracts were “very relevant” and two were “relevant.” With that relevance determination as a backdrop, the agency obtained available past performance information and conducted interviews. The CPARS for one “very relevant” contract rated GDIT as satisfactory in one area and very good to exceptional in others; the contracting officer for that procurement expressed a concern about GDIT’s ability to consistently backfill personnel, and rated its performance as good, but not excellent. The contracting officer for another “very relevant” contract rated GDIT’s overall performance as satisfactory. The contracting officer for one task order under the third “very relevant” contract said that GDIT’s overall customer satisfaction was good. For one of GDIT’s “relevant” contracts, the contracting officer stated that the firm’s overall performance was very good; no third party information was available for the fifth contract. Id. at 6-8. The Army concluded that, based on GDIT’s “recent/relevant performance record,” it had a high expectation that GDIT would successfully perform the required effort and assigned it a substantial confidence rating. The team stated that the one satisfactory rating did not outweigh the other good/exceptional findings. Id. at 6, 8.

For SAIC, the agency found that one contract was “very relevant” and four were “relevant.” With that relevance determination as a backdrop, the agency obtained available past performance information and conducted one interview. For SAIC’s “very relevant” contract, SAIC’s proposal stated that its performance was consistently rated as excellent, but the agency was unable to confirm this statement. The CPARS for one of SAIC’s “relevant” contracts confirmed that SAIC was rated exceptional in a number of areas and very good in some others. The CPARS for a second “relevant” contract confirmed that SAIC was rated exceptional in all areas; the contracting officer for this procurement stated that he was very satisfied with SAIC’s overall efforts. For a third “relevant” contract, SAIC stated that it was rated exceptional in all areas, but the agency was unable to confirm this statement. Information was not available on another “relevant” contract. Id. at 11-12. The Army found that, based on SAIC’s “recent/relevant performance record,” it had a high expectation that SAIC would successfully perform the required effort and assigned it a substantial confidence rating. Although it was not able to confirm all the meritorious past performance assessment ratings identified by SAIC, the agency stated that the past performance information it did find indicated that SAIC had received good/excellent ratings, and this gave the Army substantial confidence in SAIC’s ability to perform. Id. at 11-12.

GDIT’s argument that the agency did not consider relevance when assigning its performance confidence assessment ratings is belied by the record. GDIT’s assertion that SAIC should have been rated satisfactory confidence because its past efforts are less relevant than those of GDIT ignores the RFP’s requirement for an integrated assessment of both the relevance and quality of an offeror’s past performance. The Army conducted this integrated assessment, placing the quality of the offerors’ performance in the context of the relevance of its efforts, and GDIT has given us no basis to find it unreasonable.  (General Dynamics Information Technology, Inc., B-407057, Oct 12, 2012)

Landscapes asserts that the agency’s rejection of its Chapel Ridge experience to meet the park cleaning experience requirement was unreasonable. According to the protester, both janitorial work (as performed in the Chapel Ridge apartments) and park cleaning fall under the North American Industry Classification Systems (NAICS) code 561210, “Facilities Support Services,” and thus janitorial work experience should be acceptable to meet the park cleaning requirement.

Regardless of whether Landscapes is correct with respect to the applicable NAICS codes for these services, its argument is unpersuasive here. As noted by the agency, the solicitation not only placed janitorial work (as performed in the Chapel Ridge apartments) and park cleaning in separate categories of services, it also defined the services in materially different ways. In this regard, the solicitation park cleaning performance requirements included such work as cleaning vault toilets, picnic shelters, camping sites, and trailer dump stations; groundskeeping on beaches, fishing areas, and nature and hiking areas; and high pressure washing. RFP at 57-59. In contrast, the janitorial work performance requirements included such work as dusting furniture; cleaning and waxing floors; cleaning and vacuuming carpets; cleaning light fixtures, mirrors and windows; and groundskeeping. Id. at 57. Although there may be some overlap, it appears that the janitorial requirements are largely performed indoors while the park cleaning requirements are largely performed outdoors.

Further, and more importantly, Landscape’s description of its actual Chapel Ridge interior building maintenance experience makes clear the difference between that work and the park cleaning required here. According to Landscapes’ proposal, its Chapel Ridge interior maintenance work involved being “[o]n-call for clean-up of vacant apartments between tenants.” Landscapes Revised Proposal at 8. Further, according to the proposal, “[c]lean-up services include window cleaning, carpet/spot cleaning, dusting/cleaning of countertops, cleaning and sanitizing of bathroom fixtures and kitchen appliances and trash removal.” Id. Given the terms of the solicitation, we find reasonable the agency’s determination that such services are materially different than park cleaning.

Furthermore, we see nothing unreasonable in the agency’s determination that the Chapel Hill interior building maintenance experience involved a materially lesser magnitude of work. In this regard, while Landscapes listed the annual value of the Chapel Hill interior maintenance work as $10,000, the record indicates that the government estimate for park cleaning here is in excess of $80,000 per year. AR, Tab 17, Government Estimate at 3-5. Further, although the protester’s Chapel Ridge experience was found to be of similar magnitude with regard to the janitorial work category--based on the fact that there were 108 apartments in the complex--the agency reasonably concluded that the work was not of a similar magnitude with respect to park cleaning, where magnitude was measured by contract dollars. As a result, we find no basis to question the agency’s determination that Landscapes’ proposal was unacceptable under the experience factor.  (Landscapes Inc., B-406835, Sep 10, 2012)  (pdf)

The agency received 4 past performance questionnaires for ERT; because this was fewer than the 5 past performance questionnaires required by the solicitation, ERT was rated unknown confidence for past performance.[2] ERT asserts that since it received very favorable ratings from the references that filled out the four past performance questionnaires, it should have been rated low risk, rather than unknown confidence for past performance.

We find the evaluation in this regard to be unobjectionable. The solicitation required offerors to submit five questionnaires, which presumably is the number of questionnaires the agency believed it needed for an adequate basis to evaluate past performance, and specifically provided that if the offeror did not have relevant past performance, or if the information was not available, the offeror would not be rated favorably or unfavorably. Since there was inadequate evidence of relevant past performance as defined by the solicitation, it was consistent with the solicitation, and reasonable, for the agency to assign ERT a rating of unknown confidence for past performance. Thomas Brand Siding Company, Inc., B-286914.3, Mar. 12, 2001, 2001 CPD ¶ 53 at 4.  (Earth Resources Technology, Inc., B-406659, Jul 30, 2012)  (pdf)

TEG again protests that the DOS’s past performance evaluation of the awardees’ proposals was unreasonable and not consistent with the RFP’s evaluation scheme. Specifically, TEG contends that none of the awardees had the required number of relevant prior projects--acting as prime contractors, as defined in the RFP--to justify the agency’s confidence ratings, and that only TEG’s proposal included the requisite number of relevant projects.

As stated in our prior decision, the critical question in our review of an agency’s past performance evaluation “is whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation’s evaluation scheme, and whether it was based on relevant information sufficient to make a reasonable determination of the offeror’s past performance.” The Emergence Group, supra, at 5. The DOS’s past performance reevaluation was not consistent with the terms of the RFP.

As noted previously, to facilitate the evaluation of past performance, section L.26.2.4 of the RFP required offerors to demonstrate their relevant experience by providing a specific number of references “documenting the offerors ability as a prime contractor to hire and deploy advisors and to provide life and mission support,” and requiring each proposed subcontractor to furnish the same information, if performing at least 20 percent of the work. In addition, section M.4 provided that offerors were required to comply with the instructions in section L, such as section L.26.2.4.

Our Office held a hearing to clarify how the agency determined that the awardees’ past performance was relevant. According to the TEP Chair, who presided over the DOS’s past performance reevaluation, the TEP did not consider section L in reevaluating proposals, nor did the evaluators review offerors’ proposals to determine compliance with the section L requirements, such as whether or not offerors had furnished the required minimum number of contracts reflecting their ability to hire and deploy advisors and to provide life and mission support. See Hearing Transcript (Tr.) at 31, 33, 97-102, 103-04. Moreover, the TEP Chair did not know if any agency official performed this review. Tr. at 97-98.

This is problematic because neither the record nor the Chair’s testimony establishes that the awardees’ past performance satisfied the section L.26.2.4 requirements. For example, the Chair testified that, of the three contracts that Crucible submitted to demonstrate the relevance of its past performance, Crucible was a prime contractor on only one of these contracts, even though section L.26.2.4 required at least three references for the offeror itself. See Tr. at 103-07. The Chair further testified that Crucible did not have experience deploying advisors or providing life and mission support under any of its referenced contracts, as required by section L.26.2.4. See Tr. at 130-33. The Chair also admitted that the compliance matrix included in Crucible’s proposal was inaccurate where it stated that the proposal complied with section L.26.2.4. Tr. at 107-08; AR, Tab 9, Crucible Proposal, vol. 2, Past Performance Conformance Cross Reference Table, at 3.

Another example pertains to the evaluation of GCJS’s past performance, which relied upon subcontracts performed by some of GCJS’s individual joint venture members. The Chair conceded that these subcontracts relied upon in the past performance evaluation did not demonstrate experience in deploying advisors or providing life and mission support. See Tr. at 153-60.

On the other hand, TEG asserts, and our review confirms, that its proposal complied with section L.26.2.4. AR, Tab 9, TEG Proposal, vol. 2, Past Performance at 20-43.

The DOS argues, in its post-hearing comments, that section L.26.2.4 of the RFP is not relevant to the actual evaluation of past performance because section M, which does not expressly incorporate the requirements stated in section L.26.2.4, governs the evaluation of past performance.[4] In this regard, the DOS asserts that section M.9.3.1.(a) does not distinguish between the past performance of an offeror as a prime or as a subcontractor, so that all of this past performance can be attributed to the offeror as a whole. The agency also argues that the TEP reasonably assigned confidence ratings to the awardees, given that it reasonably found that the awardees’ past performance references reflected their experience in “providing criminal justice related support services and associated support systems required under Section C.” See e.g., Tr. at 42-43; Agency Post-Hearing Comments at 30; RFP § M.9.3.1.a.

A solicitation generally must be read as a whole and in a reasonable manner, giving effect to all its provisions. See CourtSmart Digital Sys., Inc., B-292995.2, B-292995.3, Feb. 13, 2004, 2004 CPD ¶ 79 at 12.

As noted by the DOS, our decisions have recognized that section L provisions do not have to correspond to the evaluation criteria set forth in section M because section L generally only provides guidance to assist offerors in preparing and organizing their proposals. See e.g. University Research Co., LLC, B-294358.6, B-294358.7, Apr. 20, 2005, 2005 CPD ¶ 83 at 18. Here, however, not only are the requirements contained in section L.26.2.4 stated in mandatory terms, i.e., “shall,” but section M.4 provides that offerors “shall” provide the information required by section L.

The agency’s interpretation that the solicitation did not require the past performance evaluation to consider the information required by section L.26.2.4--that is, separate past performance references for the prime contractor and the principal subcontractors--nullifies and renders meaningless the unambiguous and mandatory language of section L.26.2.4, and is thus unreasonable. See Wackenhut Int’l, Inc., B-286193, Dec. 11, 2000, 2001 CPD ¶ 8 at 6-7 (offeror must comply with mandatory requirements stated in section L of an RFP, particularly where section M makes clear that compliance with section L requirements is part of the evaluation). Section M.9.3.1.a of the solicitation provides that the experience of both the offeror and its major subcontractors would be considered in the evaluation. Nothing in section M.9.3.1.a contradicts the instructions in section L.26.2.4 that require the offeror and the subcontractors provide separate references, which will be separately evaluated; and nothing in section M.9.3.1.a negates the section L.26.2.4 threshold requirement that the offeror provide experience that shows its ability as a prime contractor to hire and deploy advisors and to provide life and mission support.

Contracting agencies do not have the discretion to announce in the solicitation that they will use one evaluation plan, and then follow another. Once offerors are informed of the criteria against which the proposals will be evaluated, the agency must adhere to those criteria in evaluating proposals and making its award decision, or inform all offerors of any significant changes made in the evaluation scheme. Wackenhut Int’l, Inc., supra, at 8. Because the record evidences that the agency did not adhere to the announced evaluation scheme in evaluating past performance, we conclude that the agency’s past performance evaluation of the proposals was unreasonable and improper.  (The Emergence Group, B-404844.7, Feb 29, 2012)  (pdf)

In reviewing protests against allegedly improper evaluations, our Office examines the record to determine whether the agency’s judgment was reasonable and in accord with the evaluation factors set forth in the RFP. See, e.g., TPL, Inc., B-297136.10, B-297136.11, June 29, 2006, 2006 CPD ¶ 104 at 10. An agency’s evaluation of past performance, which includes its consideration of the relevance, scope, and significance of an offeror’s performance history, as well as consideration of actions taken to resolve prior problems, is a matter of agency discretion which we will not disturb unless the agency’s assessments are unreasonable, inconsistent with the solicitation criteria, or undocumented. See, e.g., Yang Enter., Inc.; Santa Barbara Applied Research, Inc., B-294605.4 et al., Apr. 1, 2005, 2005 CPD ¶ 65 at 5; Acepex Mgmt. Corp., B-283080 et al., Oct. 4, 1999, 99-2 CPD ¶ 77 at 3, 5. In short, we will not substitute our judgment for that of the agency, and a protester’s mere disagreement with such judgment does not provide a basis to sustain a protest. Birdwell Bros. Painting & Refinishing, B-285035, July 5, 2000, 2000 CPD ¶129 at 5.

We have reviewed the evaluation of ProLog’s past performance and find, based on the record, that the agency’s evaluation was reasonable, consistent with the RFP’s stated evaluation criteria, and fair. In this regard, of the 10 contracts evaluated, only 1 was very relevant, 6 were somewhat relevant, and 3 were not relevant. The very relevant contract was performed by ProLog’s proposed subcontractor [DELETED]; but, as the agency noted, ProLog itself had not performed contracts that were of similar scope or magnitude. For example, the MARAD contract was close in magnitude ($24.5 million) and included some of the required services, but it did not include all transportation functions[10]; the other contracts performed by ProLog were of much smaller dollar values and therefore were not of similar magnitude. The agency recognized that ProLog’s and [DELETED] performance ratings ranged from satisfactory to exceptional and outstanding, but given ProLog’s lack of very relevant experience, the agency only had confidence (not high confidence) that the offeror would successfully perform. Accordingly, the proposal was rated satisfactory and not substantial confidence. AR, Tab 31, PCAG Report, at 57-58, 100-01; Tab 32, PAR, at 129; Tab 34, Source Selection Decision, at 4. Although ProLog disagrees with the agency’s evaluation judgments, it has not shown them to be unreasonable.  (ProLog, Inc., B-405051, Aug 3, 2011)  (pdf)

NikSoft also challenges its acceptable rating under the past performance evaluation factor. In this regard, NikSoft asserts that it “received nearly perfect scores” on each of its three past performance questionnaires, and thus it should have received an exceptional rating. Protester’s Comments at 9. In response, the agency asserts that its decision to rate NikSoft as merely acceptable was reasonable because NikSoft had failed to provide ratings from government officials, and had only provided ratings from prime contractors. We disagree.

The evaluation of past performance is a matter of agency discretion, and we will review the evaluation only to ensure that it was reasonable and consistent with the solicitation’s stated evaluation criteria and applicable statutes and regulations. The evaluation by its very nature is subjective; an offeror’s disagreement with the agency’s evaluation judgments does not demonstrate that those judgments are unreasonable. All Points Int’l Distributors, Inc., B-405954, Dec.16, 2011, 2011 CPD ¶ 281 at 3.

Here, the agency has failed to provide a meaningful explanation for why NikSoft received only an acceptable rating for past performance. Given that the RFQ specifically permitted vendors to provide questionnaires from prime contractors--and the RFQ did not indicate that these reviews would be viewed less favorably--the agency has failed to provide a meaningful response to the protester’s challenge to its rating. Thus, we find that the agency has not reasonably explained why it assigned NikSoft an acceptable rating under this factor and we sustain the protest on this basis.

NikSoft also challenges LS3’s acceptable rating under the past performance evaluation factor given that LS3 only provided two questionnaires with its quotation, even though the solicitation required vendors to provide three past performance questionnaires. In response, the CO states, without elaboration, that the agency did not consider this to be a weakness and that it had no impact on LS3’s acceptable past performance rating. CO’s Statement at 417. However, the CO provides no explanation for why the awardee’s non-conformance with the solicitation’s mandatory requirements in this regard should not affect the awardee’s ratings, given the technical evaluation rating criteria. This is a matter that should be considered in the corrective action recommended below.  (NikSoft Systems Corp., B-406179, Feb 29, 2012)  (pdf)

Excalibur asserts that the agency unreasonably determined MEA's proposal to be technically acceptable under the relevant experience factor. Protest at 10; Comments at 3. In this regard, Excalibur claims that because the relevant experience section of MEA's proposal did not describe the type of work that MEA performed under the Navy contract, MEA's proposal did not meet the relevant experience criteria, and therefore was not technically acceptable. Comments at 3-4; Supp. Comments at 4.

As a general matter, the evaluation of an offeror's experience and past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based evaluation ratings. MFM Lamey Group, LLC, B-402377, Mar. 25, 2010, 2010 CPD ¶ 81 at 10. Where a protester challenges an evaluation and source selection, we will review the evaluation and award decision to determine if they were reasonable and consistent with the solicitation's evaluation criteria and procurement statutes and regulations, and to ensure that the agency's rationale is adequately documented. JVSCC, B-311303.2, May 13, 2009, 2009 CPD ¶ 138 at 5; S4, Inc., B-299817, B-299817.2, Aug. 23, 2007, 2007 CPD ¶ 164 at 9. The evaluation of experience and past performance, by its very nature, is subjective; an offeror's mere disagreement with the agency's evaluation judgments does not demonstrate that those judgments are unreasonable. FN Mfg., LLC, B-402059.4, B-402059.5, Mar. 22, 2010, 2010 CPD ¶ 104 at 7.

It is true, as Excalibur claims, that the relevant experience section of MEA's proposal does not describe the type of work that MEA performed under the Navy contract. The past performance section of MEA's proposal, however, provides a detailed description of the work that MEA performed under each of the contracts listed in the relevant experience section, including the Navy contract. AR, Tab 13, MEA Proposal, Section 4, Past Performance, at 5-6. A bid or proposal submitted to the government is properly evaluated by reading the bid or proposal as a whole. See Management Tech. Servs., B-251612.3, June 4, 1993, 93-1 CPD ¶ 432 at 6; Earth Res. Corp., B-248662.5, B-248662.7, Dec. 29, 1992, 93-1 CPD ¶ 17 at 4. We see no solicitation provision--and Excalibur has cited to none--that would have precluded the agency from considering information in the past performance section of an offeror's proposal for purposes of evaluating the offeror's technical acceptability under the relevant experience factor. Further, due to the direct relationship here between MEA's relevant experience and past performance, we view the agency's consideration of MEA's proposal as a whole to have been logical and reasonable. This ground of protest is denied.

Excalibur also challenges the agency's consideration of MEA's performance of the Army subcontract. Comments at 4-5; Supp. Comments at 5. Excalibur presents two bases for this challenge. First, Excalibur asserts that the Army subcontract is not relevant because MEA performed the work as a subcontractor, rather than as a prime contractor. Comments at 4. Second, Excalibur asserts that the contracting officer's verification that MEA performed washer and dryer services under the subcontract was improper because this information was beyond the "four corners" of MEA's proposal and because at the time of the verification, the prime contractor point of contact allegedly had become an employee of MEA. Comments at 4; Supp. Comments at 5.

As described above, the solicitation provided that the relevant experience criteria would be met through the listing of "at least one," but no more than five, relevant contracts. See RFP at 12. Accordingly, we need not consider Excalibur's allegations regarding the Army subcontract because the record shows that the agency reasonably could--and apparently did--determine MEA's proposal to be technically acceptable under the relevant experience factor on the basis of the Navy contract alone. See AR, Tab 5, Technical Evaluation of MEA Proposal, ¶¶ 1(a) to 1(c); see also Contracting Officer's Statement at 5. In other words, the agency's finding that the Navy contract met the relevant experience criteria rendered the agency's consideration of the Army subcontract unnecessary. Thus, to the extent that the agency considered the Army subcontract, such consideration did not prejudice Excalibur. Prejudice is an essential element of every viable protest; we will not sustain a protest unless the protester demonstrates a reasonable possibility that it was prejudiced by the agency's actions. Armorworks Enters., LLC, B-400394.3, Mar. 31, 2009, 2009 CPD ¶ 79 at 3. This ground of protest is denied.  (Excalibur Laundries, Inc., B-405814,B-405814.2, Jan 3, 2012)  (pdf)

The protester complains that the agency unreasonably evaluated Solstice's proposal under the company experience, qualifications of key personnel, and past performance factors.

Our Office will review an agency's evaluation and exclusion of a proposal from the competitive range for reasonableness and consistency with the solicitation criteria and applicable statutes and regulations. Int'l Med. Corps, B-403688, Dec. 6, 2010, 2010 CPD para. 292 at 7. Contracting agencies are not required to retain in the competitive range proposals that are not among the most highly-rated or that the agency otherwise reasonably concludes have no realistic prospect of being selected for award. Federal Acquisition Regulation (FAR) sect. 15.306(c)(1); D&J Enters., Inc., B-310442, Dec. 13, 2007, 2008 CPD para. 8 at 2. In this regard, a protester's mere disagreement with an agency's evaluation and competitive range judgment does not establish that the agency acted unreasonably. SPAAN Tech, Inc., B-400406, B-400406.2, Oct. 28, 2008, 2009 CPD para. 46 at 9.

In its response to the RFP's requirements under the company experience factor, Solstice provided detailed information in its proposal about each of its projects, and about those of The Bohle Company, one of its proposed subcontractors. AR, Tab J, Solstice Proposal at 4-12. It did not, however, provide any detailed information about its other proposed subcontractor, Tetra Tech. Our review of Solstice's proposal confirms that Solstice cited experience with several projects unrelated to the main experience sought by the company experience factor– that is, topics and issues involving national flood risk management and levee safety issues. Id. Furthermore, except in the most general of terms, Solstice did not describe how the projects it identified were comparable in size, recency and/or relevance to the solicitation requirements. The record thus supports the agency's conclusion that Solstice did not provide at least three projects (counting both, its own and its subcontractors' projects) that demonstrated the requisite company experience in accordance with the RFP criteria.

In short, we find that the agency's decision to exclude Solstice from the competitive range was consistent with the terms of the RFP and reasonable. Although, Solstice disagrees with the agency's judgment in this regard, its disagreement does not demonstrate that the agency's decision was unreasonable. SPAAN Tech, Inc., supra.  (Solstice Advertising, B-405529.2, November 21, 2011)  (pdf)

CAE's protest predominantly attacks the agency's relevancy determinations of the firm's past performance. For example, CAE complains that the agency discounted CAE's subcontractor performance which led to lower relevancy ratings, considered contracts that were not identified by CAE in its proposal and then found some of them to be only somewhat relevant, and made other errors in evaluating relevant past performance. CAE then complains that these erroneous relevancy determinations were the determining factor for award. Based upon our review of the record, we find that each of CAE's challenges lack merit. Although we do not address each and every protest argument, we discuss several of them below.

For example, CAE contends that the agency unreasonably discounted two of its contacts because CAE performed the work as a subcontractor, and that the agency further distorted the evaluation results by considering five less relevant contracts where CAE was the prime contractor even though CAE did not identify these contracts in its proposal. Protest at 2. However, the record shows that the agency reasonably evaluated CAE's past performance in light of the role it was proposed to perform here. For example, since CAE proposed to perform [DELETED] percent of the effort here as the prime contractor overseeing all facets of the training system, CAE's subcontractor performance was properly found to receive less than the maximum relevancy rating for program management. Furthermore, as noted by the agency, CAE's subcontractor performance was on contracts of smaller scope, magnitude, and complexity than the effort here. To the extent the protester now disputes the agency's findings as to the relevancy of these contracts, we are unpersuaded by its arguments, especially given that CAE failed to dispute any of the agency's relevancy findings during discussions.

In addition, we find no error to the agency's consideration of additional contracts, not listed in CAE's proposal, where CAE was the prime contractor. Since the RFP announced that the agency would consider past performance information other than that provided in an offeror's proposal, and these additional contracts were somewhat relevant to the evaluation, we find this aspect of the evaluation unobjectionable.

CAE argues that it was unreasonable for the agency to rate its Air Education and Training Command Air Force Security Assistance Training (AETC AFSAT) contract as not relevant for contractor logistic support. It also complains that the agency improperly considered CPARS reports for this contract, and failed to conduct discussions to resolved a conflict between the CPARS reports and CAE's proposal. According to CAE, its work on this contract is relevant to contractor logistics support. CAE's Comments at 16-18.

The record shows that, although CAE's proposal represented that the firm had performed contractor logistics support under this contract, the last four CPARS reports stated that no such work was performed. AR, Tab 6, Subtab 1, CPARS for AETC AFSAT Contract (stating "N/A" for logistics support); Contracting Officer's Statement at 32 (CPARS stated contractor logistics support was "not applicable" for AETC AFSAT contract). The protester was advised during discussions that this contract was rated not relevant for contractor logistics support, and it failed to dispute the evaluation findings. Given CAE's silence on the matter, we find reasonable the agency's reliance on the CPARS reports as credible evidence that the contract was not relevant to the evaluation of contractor logistics support. CAE's disagreement with the agency's conclusion now is insufficient to show that the evaluation was unreasonable. See Command Enters., Inc., supra.

CAE also argues that the agency's past performance evaluation "completely ignore[d]" the program management experience of one of CAE's proposed subcontractors, [DELETED]. CAE contends that [DELETED]'s "very high ratings" in this area should have resulted in CAE receiving a higher performance confidence assessment rating in this area. Protest at 18; CAE Comments at 20-21.

As recognized by the agency, CAE did not propose to use any subcontractors in the program management role; rather, that role was reserved to CAE. AR, Tab 23, Proposal Analysis Report, at 98. [DELETED]'s role was limited to performing [DELETED] percent of the overall work, which included [DELETED], and managing and operating [DELETED] of the 11 sites where work was to be performed. Id. at 95; Protest at 8. Given [DELETED]'s limited role in managing sites, which the agency reasonably distinguished from the overall program management duties for the contract, we find the agency's decision not to give weight to [DELETED]'s past performance in program management to be unobjectionable.

In sum, based on our review of the record and considering all of the protester's arguments, we find the agency's evaluation of past performance, including the relevancy determinations, to be reasonable and consistent with the RFP.  (CAE USA, Inc., B-404625, March 16, 2011)  (pdf)

FSI asserts that its past performance rating should have been excellent rather than satisfactory. In this regard, according to the protester, a past performance rating of only satisfactory was unreasonable in view of the fact that two of its references submitted performance questionnaires which rated its overall performance with scores of 5 out of 5; a third reference rated it with an overall score of 4; and the contracting officer's technical representative (COTR) for the incumbent bridge contract had rated its performance as very good in a questionnaire that was submitted on FSI's behalf for a procurement with another agency.

As a general matter, the evaluation of an offeror's past performance, including relevance and scope of the performance history to be considered, is within the discretion of the contracting agency. We will not question an agency's judgment unless it is unreasonable or inconsistent with the terms of the solicitation, or is undocumented. Family Entm't Servs., Inc., d/b/a/ IMC, B‑291997.4, June 10, 2004, 2004 CPD para. 128 at 5.

The past performance evaluation here was unobjectionable. The record indicates that while the evaluators acknowledged FSI's positive performance questionnaire ratings, TET Report at 17; PNM at 7, they also noted that only one of FSI's contracts‑‑the incumbent contract‑‑ was for maintaining facilities comparable to the Yard. In this regard, as acknowledged by FSI, the Yard encompasses approximately 100 buildings with 1 million square feet and 113 acres. In contrast, the three past performance questionnaires submitted for FSI for other contracts were for relatively limited facilities, including one for work at a facility described as covering approximately 140,000 square feet and another for a facility with 215,000 square feet. Similarly, only FSI's incumbent work was comparable in value to its proposed efforts here ($16.9 million over 5 years). In this regard, two of FSI's other contracts were valued at less than $2 million over 5 years; another was valued at less than $4 million over 9 years; and a fourth was valued at $17 million over 11 years. FSI FPR, Past Performance Proposal at 3, 5, 7, 9.

Further, the agency reasonably found a number of aspects of FSI's past performance to be of concern. PNM at 7. For example, the record indicates that FSI experienced high turnover in project managers (PM) (six in less than 7 years) under the incumbent contract. While FSI maintains that it replaced the departing PMs quickly, the agency found that the repeated vacancies hindered other key personnel in performing their duties, and that using APMs to fill the PM position in turn led to the need to find qualified personnel to fill the APM position. PNM at 8; Contracting Officer's Statement at 8. In addition, the record indicates such other performance problems as FSI's failure to submit quotations for level III reimbursable work within the allowed 5-day response time; quotations that were over the not-to-exceed level; and failures to timely notify the contracting officer of expected delays in various tasks. PNM at 8-9; Contracting Officer's Statement at 9.

While the COTR on the incumbent bridge contract rated FSI's past performance favorably in the questionnaire furnished for another procurement, the record indicates that he was unaware of several of the above contract administration issues. COTR Declaration para. 2; Contracting Officer's Final Declaration para. 3. Since the contracting officer for this RFP was also the contracting officer for the bridge contract, and was familiar with FSI's performance problems on the prior contract, we believe that she could reasonably discount the COTR's assessment, and instead rely on her knowledge of FSI's continuing performance issues in evaluating its past performance as satisfactory.

Finally, while the past performance questionnaires for FSI's other contracts assigned high overall ratings, the contracting officer here noted that FSI's business relations ratings for each of the three contracts appeared to decline as the contract neared completion. For example, for a Fort Campbell, Kentucky barracks maintenance contract, which had 2 years of performance left, FSI was rated with all 5s. In contrast, a facility operations contract at Fairchild Air Force Base, Washington, which had 8 months before completion, was scored with a mixture of 5s and 4s, and a facility operation/maintenance contract at the Southeast Archives Building, Georgia, which had 3 months before completion, was scored with a mixture of 4s and one 3. Protest exh. B. When considered with the negative information concerning FSI's performance on the incumbent contract, the contracting officer concluded there was an overall trend toward declining performance by FSI toward the end of contract performance. Contracting Officer's Final Declaration para. 4.

While FSI disagrees with the contracting officer's rationale, we find nothing unreasonable in the contracting officer's determination that, given the specific performance concerns identified above, a past performance rating of excellent was not warranted.

KIRA Past Performance

FSI asserts that the evaluation of KIRA's past performance as excellent was unreasonable because KIRA's past performance record included some "satisfactory" ratings and the agency improperly credited KIRA for contracts that were not of similar scope to the RFP.

The agency's evaluation was reasonable. KIRA submitted information on three past base operations contracts valued from $81 million to $250 million for performance periods ranging from 7 to 10 years. Thus, each of KIRA's contracts was greater in scope than the current contract. While FSI asserts that it was improper to give KIRA credit for its performance on more complex contracts because they were not of the same scope, we believe the agency reasonably concluded that KIRA's proven performance on much larger projects was a strong indication that it would be capable of meeting the PWS requirements here. SAR at 5; see Family Entm't Servs., Inc., d/b/a IMC, supra (agency may reasonably consider contract performance under contracts more complex than work solicited).

Further, favorable past performance questionnaires were received for all three contracts, including one which rated KIRA as overall 5 out of 5 (substantially exceeded expectations) and two of which rated its overall performance as 4 out of 5 (met all expectations and exceeded some). The fact that one of KIRA's references included two ratings of 3 (met all expectations), while all other individual ratings were 4s and 5s, did not preclude the agency from evaluating KIRA's performance as overall excellent. Here, in contrast to FSI's past performance record, the evaluators found no weaknesses and no indication of declining performance over the life of the contracts. Rather, KIRA's past performance was characterized by very positive performance managing complex base maintenance contracts. TET Report at 15. In these circumstances, we find no basis to question the past performance evaluation.  (Facility Services Management, Inc., B-402757.6; B-402757.7, February 10, 2011)  (pdf)

Bannum argues that its past performance should have been rated as "Blue-Outstanding," and that this higher rating would have justified a tradeoff in its favor. In its comments on the agency report, however, Bannum focuses primarily on one aspect of the past performance evaluation, arguing that by the time the BoP evaluated past performance, the default termination of the firm's contract had occurred more than 3 years earlier, and which Bannum argues should have placed that event outside the BoP's consideration. We disagree.

Where a solicitation requires the evaluation of offerors' past performance, we will examine an agency's evaluation only to ensure that it was reasonable and consistent with the solicitation's evaluation criteria, since determining the relative merits of offerors' past performance information is primarily a matter within the contracting agency's discretion. DGR Assocs., Inc., B-285428, B-285428.2, Aug. 25, 2000, 2000 CPD para. 145 at 11. While a solicitation may specify the time frame for references an offeror may submit regarding past performance, such a limitation does not imply a limitation on the agency's evaluation of past performance. Alaska Mech., Inc., B‑404191, Dec. 15, 2010, 2010 CPD para. 296 at 4. More generally, an offeror's mere disagreement with the agency's assessment of its past performance, or the merits of its proposal relative to others, does not render the source selection unreasonable. Encorp-Samcrete Joint Venture, B‑284171, B‑284171.2, Mar. 2, 2000, 2000 CPD para. 55 at 4.

Although, as noted above, the RFP instructed offerors to submit past performance references that had occurred in the preceding 3 years (which at the time of Bannum's past performance proposal in June 2008 could have included the 2007 default termination), Bannum points to no limitation on the BoP's consideration of relevant past performance that was beyond that time--either in the RFP or by statute or regulation. Rather, the RFP description of the past performance evaluation merely indicated that more weight would be given to more recent experience. It did not foreclose consideration of older (or, for that matter, more recent) performance than the offeror had submitted. In short, Bannum has not shown that the BoP's evaluation was unreasonable or contrary to the terms of the RFP. Nor, more generally, has Bannum shown that the evaluation of its past performance as "Green-Acceptable" was unreasonable. The contemporaneous record reflects the evaluators' fair consideration of both the strengths and weaknesses in the performance record for Bannum in a manner consistent with the RFP. (Bannum, Inc., B-404712, March 1, 2011)  (pdf)

Nova challenges the agency's evaluation of the past performance portion of its proposal, asserting that the agency improperly failed to credit Nova for successful performance of the incumbent contract for a period of approximately 10 weeks before final proposals were submitted. Had the agency done so, Nova claims that the agency would have evaluated its past performance as "excellent," not merely "good." See Protest at 7. The agency argues that it properly accorded no weight to the protester's incumbent contract past performance reference, given the brief period of Nova's performance.

Where a solicitation requires the evaluation of offerors' past performance, we will examine an agency's evaluation only to ensure that it was reasonable and consistent with the solicitation's evaluation criteria and procurement statutes and regulations, because determining the relative merits of offerors' past performance information is primarily a matter within the contracting agency's discretion. The MIL Corp., B‑297508, B-297508.2, Jan. 26, 2006, 2006 CPD para. 34 at 10. Mere disagreement with an agency's evaluation is not sufficient to render the evaluation unreasonable. Bevilacqua Research Corp., B-293051, Jan. 12, 2004, 2004 CPD para. 15 at 8 n.8. Solicitations must identify all significant evaluation factors and any significant subfactors that will be considered in awarding the contract, and the evaluation of proposals must be based on the factors set forth in the solicitation. Federal Acquisition Regulation (FAR) sections 15.304(d), 15.305(a). While agencies are required to identify the major evaluation factors, they are not required to identify all areas of each factor which might be taken into account, provided that the unidentified areas are reasonably related to, or encompassed by, the stated criteria. Mid-Atlantic Design & Graphics, B-276576, July 1, 1997, 98-1 CPD para. 132 at 3-4.

The RFP did not state that, in evaluating past performance, the agency would consider the length of time that an offeror had performed a contract; it is self-evident, however, that the length or duration of an offeror's prior contract effort logically relates to both the relevance and quality of an offeror's past performance. Chenega Tech. Prods., LLC, B-295451.5, June 22, 2005, 2005 CPD para. 123 at 6. In evaluating an offeror's likelihood of successful performance, a prior contract effort that is of brief or limited duration is simply not as probative of an offeror's record as a contract for a lengthier period of time. Id. Where, as here, the RFP stated that prior contracts would be assessed to determine whether they were the same or similar in nature, size, and complexity as the requirement being procured under this solicitation, we see nothing unreasonable in the agency's consideration of the length of contract performance in its evaluation of past performance. Given the one year base period of the contract and the fact that Nova had performed the incumbent effort for approximately 10 weeks at the time of the evaluation here, we find the agency's decision to give Nova's performance of that contract no weight and not to consider it as relevant to the evaluation of Nova's past performance was neither unreasonable nor inconsistent with the solicitation.  (Nova Technologies, B-403461.3; B-403461.4, February 28, 2011)  (pdf)

Herve Cody argues that the firm's experience on the St. Lucie spillway repair project is relevant to the spillway replacements solicited here. See Protest at 2-4; Comments at 7. According to the protester, the SOW requires proficiency in all aspects of replacing spillway systems, which the protester contends includes both earthworks and metalworking. See Comments at 2. The protester maintains that replacing the spillways requires construction of new steel spillway boxes and, in this respect, the St. Lucie project is relevant because it demonstrates Herve Cody's ability to perform metal work and steel fabrication. The protester states that the other two projects demonstrate Herve Cody's ability to perform earthworks and suggests that, cumulatively, the three examples submitted demonstrate its ability to perform all aspects of spillway replacement, that is, earthworks and metalworking. See id. at 3 n.1. The protester complains that, in evaluating the St. Lucie project, the agency focused only on the earthwork component and ignored the metalworking component of the SOW's requirements.

The agency disputes that the St. Lucie project is relevant. In this regard, the agency states that the evaluators found substantive differences between the St. Lucie repair project and the spillway replacements solicited here. The agency also notes that the protester failed to submit any description of the work it performed for the St. Lucie project showing why it should be viewed as relevant. Thus, the Corps contends that it reasonably concluded that the protester did not submit the minimum number of relevant and similar project examples required by the RFP. See AR, Tab 1, at 5; CO's Statement at 2.

Our Office examines an agency's evaluation of experience and past performance to ensure that it was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations. See JVSCC, B‑311303.2, May 13, 2009, 2009 CPD para. 138 at 5. A protestor's mere disagreement with the agency's judgment is not sufficient to establish that the agency acted unreasonably. MFM Lamey Group, LLC, B-402337, Mar. 25, 2010, 2010 CPD para. 81 at 10.

Here, the record shows that the Corps reasonably found that Herve Cody failed to provide three relevant projects to demonstrate its corporate experience. As noted above, the RFP required offerors to identify at least three projects that were as similar as possible to the solicited work, which involves replacement of spillways, and includes excavation and backfill of dikes. The Corps found the St. Lucie Spillway repair project was not similar to the scope of the work solicited. That, is the SSEB noted that Herve Cody's work on the St. Lucie project did not show experience with, among other things, excavation and construction of dikes, fabrication and/or installation of spillway systems, and drainage. AR, Tab D, Source Selection Evaluation Report, at 8.

Herve Cody does not contend that its work on the St. Lucie project includes excavation and backfill of dikes.[3] Rather, the protester argues that the agency should have considered that the firm's cumulative experience was relevant (regardless of the scope and type of each individual project) because, considered together, its projects demonstrate that Herve Cody has experience performing all relevant tasks. This, however, would be inconsistent with the RFP's unambiguous requirement to identify a minimum of three projects of similar scope.

In sum, we find that Herve Cody has not shown that the agency's evaluation of the firm's corporate experience was unreasonable or was inconsistent with the RFP's evaluation criteria.  (Herve Cody Contractor, Inc., B-404336, January 26, 2011)  (pdf)

AMI asserts that the evaluation was flawed because the agency unreasonably failed to consider a positive record of its past performance involving work at the Rock Creek Mine. In this regard, AMI's proposal included a completed past performance questionnaire--reflecting all "very good" or "exceptional ratings"--and identified it as coming from the former mill manager and as having been submitted to another federal agency in support of a previous, unrelated past performance evaluation. AMI asserts that the agency was required to consider this performance record under the terms of the RFP.

The agency's treatment of this past performance record was unobjectionable. For contracts completed within the past 3 years, the RFP allowed offerors to submit any currently available past performance information, including customer surveys as well as a listing of up to 10 relevant past/present performance contracts. RFP sect. L(4)(b)(i)-(ii). For each of the listed projects, offerors were required to provide identifying information, including up-to-date contact information for contracting officers and to forward agency-provided questionnaires to a "verified point of contact" at each past performance reference, with instructions to complete the questionnaire and return it to the agency. Id. sect. L(4)(b)(ii)-(iv). AMI completed and forwarded questionnaires for the four projects discussed above, but for the Rock Creek Mine, it simply submitted a previously completed (and different) questionnaire from the former mill manager. It also submitted background information on the project as part of its own "currently available" information. AMI explained in its proposal that it did not resolicit the Rock Creek information because the questionnaire had been used in prior RFPs. Past Performance Proposal at 7.

Notwithstanding that the RFP solicited a variety of past performance information from offerors with no parameters, it set explicit parameters for consideration of past performance questionnaires. In particular, the RFP required questionnaires to come from verified points of contact rather than from the offeror itself. As explained by the contracting officer, since the Rock Creek questionnaire was submitted directly by AMI, rather than by the third party reference, the agency was unable to validate the integrity of the information in the questionnaire. Contracting Officer's Statement at 6. In determining the quality and relevance of an offeror's past performance information, an agency reasonably may consider the credibility of the information's source. See Hughes Missile Sys. Co., B-259255.4, May 12, 1995, 95-1 CPD para. 283 at 14‑15 (agency performance risk assessment that included consideration of the credibility of the information source was proper). Indeed, under the Federal Acquisition Regulation (FAR), agencies are required to consider the source of past performance information. FAR sect. 15.305. Since the Rock Creek questionnaire was not in the same format as the questionnaires used in the RFP, and was submitted by AMI itself--with an obvious stake in the evaluation--instead of directly from the third party reference, as provided for under the RFP, the agency reasonably could conclude that it lacked sufficient credibility. It follows that the agency reasonably disregarded the questionnaire and based AMI's past performance evaluation on the other four questionnaires. See J. Womack Enters., Inc., B‑299344, Apr. 4, 2007, 2007 CPD para. 69 at 8 (agency reasonably questioned validity of responses submitted by offeror's critical subcontractor, given stake in outcome of competition).  (Alaska Mechanical, Inc., B-404191, December 15, 2010)  (pdf)

CommSol also complains that the Coast Guard did not favorably consider the firm's 2010 dry dock repair contract with the agency. In this regard, the protester argues that the Coast Guard should not have considered the July 2010 default termination of this contract, because CommSol appealed the termination to the Civilian Board of Contract Appeals (CBCA) and the Coast Guard ultimately agreed to convert the termination for default to a termination for convenience.

As indicated above, the record here shows that the Coast Guard did not consider CommSol's default-terminated 2010 dry dock repair contract. That said, the agency properly could have considered this contract. CommSol's 2010 contract was terminated several weeks before the Coast Guard made its selection decision, and the Coast Guard's agreement to convert the default termination to one for convenience occurred after the agency's evaluation of quotations and selection decision. See Protester's Comments, exh. B. An agency may properly consider the termination of a firm's contract in its past performance evaluation, even where the termination has been appealed; the fact that a termination may ultimately be overturned on appeal does not demonstrate that the agency's earlier consideration of that default termination in a past performance evaluation was unreasonable. See MAC's Gen. Contractor, B‑276755, July 24, 1997, 97-2 CPD para. 29 at 3.  (Commissioning Solutions Global, LLC, B-403542, November 5, 2010) (pdf)

Innovative protests the agency's evaluation of its past performance. Specifically, Innovative complains, with regard to the project that was determined to be not recent, that by the RFP issue date it had completed over 50 percent of the portion of the overall project that was assigned to the firm, and thus the agency should have found the project to be recent. Innovative also complains, with regard to the remaining four projects that were found to not be relevant, that each of these projects contained features that were relevant to the work being procured here.

In reviewing a protest challenging an agency's past performance evaluation, we will examine the record to determine whether the agency's judgment was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations. Ostrom Painting & Sandblasting, Inc., B‑285244, July 18, 2000, 2000 CPD para. 132 at 4.

We find from our review of the record that the agency's conclusions regarding Innovative's projects were reasonable. With respect to Innovative's first project, the Bert Jones Yacht Basin Bulkhead, Dredging and Pier Replacement contract, the Corp found that this project, which was identified as being performed by an unnamed joint venture, did not satisfy the solicitation's requirement that the project be 50 percent complete by the RFP issuance date. The Corps noted that Innovative's proposal showed that at the time of the RFP's September 23, 2009 issuance date, Innovative had been performing the project for only 6 months, with an estimated completion date of January 24, 2011, that is, some 16 months later. See AR, Tab D, Re-evaluation of Innovative Builders, at 6; Innovative's Proposal, Vol. 1, Past Performance. In its comments, Innovative does not assert that 50 percent of the project was completed by the RFP issuance date, rather Innovative asserts that it had completed 50 percent of its portion of the overall project by that date. Protester's Comments at 5. Innovative's proposal did not, however, identify what work comprised Innovative's portion of the project, or identify that 50 percent of this work had been completed by the RFP issuance date. Accordingly, we find that the agency reasonably concluded that this project did not satisfy the RFP's requirements.

We also find reasonable the agency's conclusion that Innovative's fourth project (reconstruction of piers) and fifth project (construction of municipal marina) were not sufficiently similar in scope, type, effort and complexity to be considered relevant to the construction project here. Construction of the seawall solicited by the RFP here would require forming and placement of structural concrete, reinforcement, vinyl sheet pile, storm drainage or trench drains, compacted fill, and other things. See AR, Tab D, Re-Evaluation of Innovative Builders, at 14. This would require 15,249 cubic yards of concrete, 95,000 linear feet of concrete pile, 140,000 cubic yards of compacted fill, 61,000 cubic yards of beach fill, and 15,000 tons of rip rap, among other things. Id. In comparison, Innovative's project No. 4 (a $966,700 project), required only 300 cubic yards of concrete (2.0% of what was required here), no vinyl sheet pile, no storm drainage or trench drains, no compacted fill, only 500 cubic yards of beach fill (0.8% of what was required here), and only 1,000 tons of rip rap. The Corps found that this did not demonstrate that this project had a similar scope, level of effort, or complexity to establish that this project was relevant. See id. at 15. Although Innovative disagrees with the Corps' judgment in this regard, its disagreement does not show that the agency's judgment was unreasonable. Entz Aerodyne, Inc., B‑293531, Mar. 9, 2004, 2004 CPD para. 70 at 3.

With regard to Innovative's fifth project (a $326,500 project), the Corps found that the marina only required 100 cubic yards of concrete (0.7% of what was required here), only 400 linear feet of vinyl sheet pile (7% of what was required here), and 2,800 linear feet of timber pile (compared to the 95,000 linear feet of concrete pile). The Corps also found that the project did not contain any forming or placement of structural concrete or concrete piles. Id. at 16. Again, given the significant disparity in size and scope of these projects with the seawall being procured here, we find reasonable the agency's conclusion that this project was not relevant.

In its comments, Innovative appears to contend that it may have been treated disparately in the reevaluation, asserting that its past performance projects were subject to greater scrutiny than other proposals. This protest allegation was not timely raised, however, since Innovative received the agency's report (containing the revised source selection decision and reevaluation) on August 6, 2010, but did not file its comments until August 17, some 11 days later. See 4 C.F.R. sect. 21.2(a)(2) (2010) (a protest based on other than alleged improprieties in a solicitation must be filed no later than 10 calendar days after the protester knew of the basis for protest). Moreover, Innovative does not argue that the awardee's satisfactory performance rating would have changed if its proposal had been reevaluated using the alleged stricter scrutiny that was applied to Innovative's proposal.

The protest is denied.  (Innovative Builders, Inc., B-402507.2, September 15, 2010) (pdf)

The protester also contends--as set forth in more detail below--that the contracting officer improperly considered older past performance information when more recent information was available. In addition, the protester contends that the contracting officer should have considered interim and final monitoring reports in her past performance evaluation.

With regard to the age of the contractor evaluation forms, the protester notes that the contracting officer’s evaluation considered contractor evaluation forms for the following periods:

Bannum’s Comments at 2. However, the protester contends that the contracting officer should have considered the following more recent contractor evaluation forms:

Bannum’s Comments at 3.

The agency states that, pursuant to Federal Acquisition Regulation (FAR) sect. 42.1503, the past performance evaluation system involves several levels of internal agency review, as well as an opportunity for the contractor to comment on the ratings, before a contractor evaluation form can be finalized. The agency contends that none of the contractor evaluation forms identified by the protester had been finalized by the time the contracting officer performed her past performance evaluation on December 18, 2009. Furthermore, three contractor evaluation forms (Fayetteville, Jackson, and Clarksburg) were still provisional or in dispute at the time of the source selection decision on March 25, 2010. Supp. AR at 4.

With regard to the Savanna contract, an initial contractor evaluation form for the period identified by the protester--September 1, 2008 to September 1, 2009--had been prepared by the agency and sent to Bannum for comment on December 14, which was 4 days prior to the contracting officer’s past performance evaluation. Supp. AR, attach. 1, Contractor Evaluation Form Correspondence, at 1. Bannum submitted its comments and rebuttals on January 11, 2010, more than 3 weeks after the contracting officer had prepared her past performance evaluation of the protester’s proposal. Id. at 6. Initial contractor evaluation forms for the Fayetteville, Jackson and Clarksburg contracts had not yet been prepared or sent to Bannum at the time of the contracting officer’s past performance evaluation on December 18, 2009. Supp. AR at 7, n.10. Indeed, with regard to the Jackson and Clarksburg references, the periods covered by the more recent contractor evaluation forms did not end until December 31, 2009 and January 31, 2010, respectively--which was 2 to 3 weeks after the contracting officer had performed her past performance evaluation of Bannum’s proposal. Id. at 5-6.

The FAR provides:

Agency evaluations of contractor performance prepared under this subpart shall be provided to the contractor as soon as practicable after completion of the evaluation. Contractors shall be given a minimum of 30 days to submit comments, rebutting statements, or additional information. Agencies shall provide for review at a level above the contracting officer to consider disagreements between the parties regarding the evaluation. The ultimate conclusion on the performance evaluation is a decision of the contracting agency. . . . These evaluations may be used to support future award decisions.

FAR sect. 42.1503(b) (emphasis added).

Given that the agency was required under the FAR to afford Bannum an opportunity to comment on its proposed contractor evaluation form ratings prior to finalizing them and using them to support future award decisions, we find the agency was under no obligation to consider the initial contractor evaluation form ratings in evaluating Bannum’s past performance. Therefore, we find no merit to the protester’s argument that the agency failed to use the most current contractor evaluation forms.

Similarly, we find unpersuasive the protester’s argument that the agency should have considered the interim and final monitoring reports. Monitoring reports document individual visits to a contractor’s facility and, in the aggregate, form the supporting basis for the ratings assigned in the contractor evaluation form. Declaration of Deputy Chief of Community Corrections (June 15, 2010), at 1-2. A full monitoring report documents an announced planned visit by a BOP monitoring team to assess the contractor’s performance. Id. at 1. An interim monitoring report documents a visit that may be announced or unannounced and often focuses on areas of contractor performance that the agency previously identified as areas of concern or deficiency. Id.

Agencies have the discretion to determine the scope of the performance history to be considered, provided that all proposals are evaluated on the same basis and the evaluation is consistent with the terms of the RFP. Weidlinger Assocs., Inc., B‑299433; B-299433.2, May 7, 2007, 2007 CPD para. 91 at 8; USATREX Int’l., Inc., B‑275592, B-275592.2, Mar. 6, 1997, 98-1 CPD para. 99 at 3. Given this discretion, we cannot find unreasonable the agency’s decision to rely on contractor evaluation forms--the final, formal performance evaluations of the entire performance period--rather than monitoring reports, which memorialize only the performance observed during a single visit.[6] Therefore, we will not disturb the agency’s decision to rely on final contractor evaluation forms, combined with the information submitted in the offerors’ past performance proposals, to evaluate past performance.  (Bannum, Inc., B-402730, July 6, 2010)  (pdf)

FN argues that the Army unreasonably evaluated Colt’s proposal as "low risk" under the past performance factor. According to FN, based on the terms of the RFP, none of Colt’s past performance references should have been considered relevant because they did not involve the manufacture of "belt-fed" machine guns, or indeed, machine guns at all.

Where a solicitation calls for the evaluation of past performance, we will examine the record to ensure that the evaluation was reasonable and consistent with the solicitation’s evaluation criteria and procurement statutes and regulations. Divakar Techs., Inc., B-402026, Dec. 2, 2009, 2009 CPD para. 247 at 5. The evaluation of past performance, by its very nature, is subjective; an offeror’s mere disagreement with the agency’s evaluation judgments does not demonstrate that those judgments are unreasonable. SDV Telecomms., B-279919, July 29, 1998, 98-2 CPD para. 34 at 2.

Here, Colt submitted three references for evaluation: (1) a contract for the manufacture of M/4/M4A1 carbines, with a value of approximately $478 million; (2) a contract for the manufacture of the M16A4 rifle with a value of approximately $16 million; and (3) a second contract for the manufacture of M4/M4A1 carbines with a value of approximately $6 million. The SSA determined all three references to be relevant since they were of "comparable complexity and value to the proposed effort." AR, Tab III.1, SSDD, at 18.


In documenting his relevance determination, the SSA concluded that the M4/M4A1 and the M16 are considered machine guns because they are weapons which are capable of automatically shooting more than one shot by a single pull of the trigger without having to be manually reloaded. In this regard, the SSA further noted that "the M4A1 carbine is capable of full automatic fire at a rate of approximately 800 rounds per minute from a 30 round detachable box magazine." AR, Tab III.1, SSDD, at 17. Expressly recognizing that the referenced weapons are not "belt-fed" machine guns, the SSA determined that the complexity of a box magazine-fed machine gun, like the M4/M4A1 and M16, "requires a similar attention to strict government technical tolerances and specifications of a belt fed magazine" and further noted that "any effort to manufacture complete small caliber automatic weapons demonstrates greater knowledge and capability than the production of lesser weapons or weapon parts." Id. at 17. According to the SSA, the weapons manufactured by Colt under these contracts "were made and tested to a weapon specification similar to the M240 Machine Gun." Id. at 18.

The SSA also expressly considered the dollar values of the various contracts referenced by Colt as part of his relevance assessment. In this regard, the SSA highlighted the fact that the $478 million M4/M4A1 contract had a "significantly" greater value as compared to the estimated value of the M240 acquisition (approximately $187 million). Id. The SSA again noted that the M4/M4A1 weapon manufactured by Colt is made in accordance with a government-approved technical data package, which involves a "significant degree of complexity." Id. Acknowledging that the other two contracts referenced by Colt were not equivalent in dollar value, the SSA similarly emphasized the complexity associated with the weapons manufactured under these contracts in justifying his determination that they should be considered relevant for the purposes of evaluating Colt’s past performance.

FN principally argues that the SSA’s findings do not comport with the evaluation criteria established under the RFP, which defined "relevant" as "[g]as operated, belt fed machine guns produced to other Military technical data packages." RFP, sect. L.3. According to FN, the M4/M4A1 and M16 weapons "in general parlance" are not considered machine guns. Protest at 8. FN asserts that, even if they can properly be considered machine guns, they are not "belt-fed," and thus, by the terms of the RFP, Colt’s contracts for the manufacture of these weapons should not have been regarded as relevant. Instead, they should have been considered, at most, "somewhat relevant," which was defined to encompass "other small arms weapons, machine gun parts, or experience in producing commercial small arms parts." RFP, sect. L.3. FN further argues that the SSA could not rely on the language in the solicitation indicating that "comparable complexity, size or value to the proposed effort may also be considered in determining relevance" for the purpose of finding Colt’s contracts for the M4/M4A1 and M16 weapons relevant. In FN’s view, such a finding would render the relevance categories meaningless since it would allow the Army to effectively redefine the specific relevance definitions identified in the RFP.

We find that FN’s challenge to the agency’s past performance evaluation is based on an unreasonably narrow reading of the SSA’s evaluation and the evaluation criteria established under the RFP. First, as established by the record, the SSA expressly recognized the fact that Colt’s referenced contracts did not fit precisely within the definition of a belt-fed machine gun under the relevance categories identified in the RFP. Thus, whether the guns manufactured under Colt’s M4/M4A1 and M16 contracts are in fact "machine guns" (a term not specifically defined in the RFP) or "belt-fed" weapons, is not relevant to assessing the reasonableness of the SSA’s analysis. Rather, the record shows that the SSA based his relevance assessment on the degree to which Colt’s contracts for the manufacture of the M4/M4A1 and the M16 weapons were of comparable complexity to the manufacture of the M240 machine gun.

Second, by its terms, the language of the RFP afforded the agency broad discretion in assessing relevance beyond the narrowly defined relevance categories identified by the RFP. In reaching this conclusion, we note that the discretionary language appears immediately after the defined relevance categories and expressly provides that "complexity, size or value to the proposed effort may also be considered in determining relevance." RFP, sect. L.3 (emphasis added). Moreover, recognizing that the agency retained the discretion under the RFP to effectively look beyond the specific relevance categories does not render the defined categories meaningless, as FN asserts. Rather, it is FN’s rigid adherence to the defined relevance categories that would effectively read out the discretionary language set forth in the RFP. See Northrup Grumman Info. Tech., Inc., B-401198, B-401198.2, June 2, 2009, 2009 CPD para. 122 at 2 (in order for an interpretation to be reasonable, solicitation must be read as a whole and in a manner that gives effect to all of its provisions). The SSA reasonably understood the RFP language as affording him the overall discretion to consider a multitude of factors in making his final assessment regarding the relevance of offerors’ references in addition to the specific relevance categories established by the RFP. Because the agency’s evaluation was consistent with the evaluation criteria established by the terms of the RFP, we see no basis to find the agency’s past performance evaluation improper as FN has alleged.

FN further argues that it was unreasonable to find Colt’s referenced contracts relevant since they are not of comparable complexity. FN believes that the manufacture of the M240 machine gun is significantly more complex as compared to the manufacture of Colt’s M4/M4A1 and M16, which are not belt-fed, and notes that this fact was expressly recognized by Colt in its proposal and the solicitation, which specifically defined relevant contracts as involving belt-fed machine guns. While Colt’s proposal highlights various similarities and recognizes the differences between the manufacture of the M4/M4A1 and M16 as compared to the manufacture of the M240, Colt’s proposal does not indicate, or otherwise suggest, that its activities under its referenced contracts were not of comparable complexity. In addition, FN’s reliance on the RFP to support its relevance argument simply rehashes its contention that the SSA’s relevance determination was contrary to the terms of the RFP. As explained above, the RFP afforded the Army the discretion to make judgments regarding the relative degree of complexity of a contract as compared to the M240B requirement when assessing relevance. While FN may ultimately disagree with the SSA’s assessments regarding the comparable complexity of Colt’s references and his determination that they were relevant for the purposes of evaluating Colt’s past performance, FN’s mere disagreement does not render the SSA’s evaluation findings unreasonable.  (FN Manufacturing, LLC, B-402059.4; B-402059.5, March 22, 2010) (pdf)

Carthage also protests that the agency's evaluation of the awardee's past performance failed to consider negative past performance information. Specifically, the protester notes that the VA's Office of Inspector General (OIG) issued a report critical of certain aspects of Valor's performance of the operation of a CBOC in Monaca, Pennsylvania. See VA OIG Report No. 09-01446-26, "Healthcare Inspection Community Based Outpatient Clinic Reviews." Carthage contends that the VA "either was or should have been aware" of other allegations regarding Valor's allegedly poor performance on CBOC contracts. In support of this allegation Carthage provides a copy of a letter from a United States Senator to the VA requesting that the agency examine complaints about two CBOCs operated by Valor in Texas. Carthage also provides a number of newspaper articles from papers in Pennsylvania and Texas, discussing the OIG report and the allegations of poor care in Texas CBOCs.

Our Office has recognized that in certain limited circumstances an agency evaluating an offeror's proposal has an obligation (as opposed to the discretion) to consider "outside information" bearing on the offeror's proposal. International Bus. Sys., Inc., B-275554, Mar. 3, 1997, 97-1 CPD para. 114 at 5; G. Marine Diesel; Phillyship, B-232619, B-232619.2, Jan. 27, 1989, 89-1 CPD para. 90 at 4-5. Where we have charged an agency with responsibility for considering such outside information, the record has demonstrated that the information in question was "simply too close at hand to require offerors to shoulder the inequities that spring from an agency's failure to obtain, and consider this information." International Bus. Sys., Inc., supra; see GTS Duratek, Inc., B-280511.2, B-280511.3, Oct. 19, 1998, 98-2 CPD para. 130 at 14.

Here, the record does not demonstrate that this information was "too close at hand" for the agency to ignore. Specifically, there has been no showing that the VA employees involved in the evaluation or source selection were aware or should have been aware of the OIG report, the Senator's letter, or the articles in small newspapers located outside of New York State. Accordingly, we have no basis to conclude that the agency should have considered this information in its evaluation of Valor's past performance.  (Carthage Area Hospital, Inc., B-402345, March 16, 2010)  (pdf)

Shaw-Parsons argues that FEMA’s past performance evaluation was fundamentally flawed because it failed to consider the PPQs it received regarding Shaw-Parsons’ performance, as well as those of the other firms, and instead relied solely upon information contained in the firms’ SF 330 submissions. We agree.

In reviewing an agency’s evaluation under the past performance factor, the critical questions are whether the evaluation was conducted fairly, reasonably, and in accordance with the stated evaluation terms, and whether it was based on relevant information sufficient to make a reasonable determination of the firm’s overall past performance rating. University Research Co., LLC, B-294358.6, B‑294358.7, Apr. 20, 2005, 2005 CPD para. 83 at 16. An agency’s past performance evaluation is unreasonable where the agency fails to give meaningful consideration to all the relevant past performance information it possesses. DRS C3 Sys., LLC, B-310825, B-310825.2, Feb. 26, 2008, 2008 CPD para. 103 at 22.

Here, the SSN required firms to provide references for at least five contracts performed within the last 3 years. FEMA specifically provided firms with the PPQs, which they were to have completed by their past performance references and returned to FEMA. In its evaluation of Shaw-Parsons’ past performance information, FEMA determined that Shaw-Parsons had "[p]rovided requested information on past performance on 5 contracts of similar size, type, and scope with Government agencies and private industry in terms of project management, accuracy of costs estimates, cost control, quality control, completion of projects within budget, and compliance with performance schedules." Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw-Parsons, at 23. The record also reflects that five of Shaw-Parsons’ past performance references provided FEMA with completed PPQs. Shaw-Parsons’ PPQs reflect that it was rated "superior" under each performance element (quality of product or service, cost control, timeliness of performance, and business relations) for all five contracts, except for one contract where it received a rating of "acceptable" under the cost control element. This translates to ratings of "superior" on 19 out of 20 possible past performance ratings in Shaw-Parsons’ PPQs.

Further, the PPQs contain narrative comments regarding the quality of Shaw-Parsons’ performance. For example, on a U.S. Postal Service (USPS) contract with a value of nearly $500 million, the reference provides narrative comments for each performance element. Specifically, with respect to the "quality of product or service" performance element, the reference commented that Shaw-Parsons "has provided SUPERIOR services and products . . . evidenced by the fact that the USPS has awarded Parsons nearly 17,000 individual task orders! . . . Parsons has consistently met and exceeded the expectations of the USPS in the performance of large projects and programs such as the Leased Space Accessibility Program, where Parsons manages the work at over 12,000 facilities in nearly every state of the Union." Shaw-Parsons’ USPS Contract PPQ. Under timeliness of performance, the USPS reference indicated that "Parsons’ performance in terms of meeting the project schedules has been outstanding" and noted two prime examples in this regard. On a $250 million FEMA contract for "individual assistance" and "technical assistance" (ITAC), the reference, in justifying the "superior" ratings for Shaw-Parsons, noted that "Shaw has provided superior support by having highly qualified staff and resources onsite in less than 24 hours during critical need, and between 48-72 hours for routine requirements. Far exceeds their competitors." Shaw-Parsons’ FEMA IATC PPQ. The other PPQs provide comments with specific examples in justification of their "superior" ratings regarding the quality of Shaw-Parsons’ performance.

FEMA, however, did not consider Shaw-Parsons’ PPQs as part of its past performance evaluation. In fact, FEMA did not consider the PPQs for any firm. Rather, FEMA performed its assessment of the quality of the firms’ past performance based entirely on the information provided by the firms in their SF 330s. As a consequence, FEMA rated Shaw-Parsons’ past performance "acceptable," finding, in part, that the information contained in Shaw-Parsons’ SF 330 was insufficient to adequately judge the quality of its past performance. Shaw-Parsons AR, Ex. 17, SEB Evaluation of Shaw-Parsons, at 23. We conclude that FEMA’s failure to consider Shaw-Parsons’ PPQs in assessing the quality of Shaw-Parsons’ past performance was improper.

While there is no legal requirement that an agency consider all past performance references, we have held that some information is simply "too close at hand to require offerors to shoulder the inequities that spring from an agency’s failure to obtain and consider information." SCIENTECH, Inc., B-277805.2, Jan. 20, 1998, 98-1 CPD para. 33 at 5 (citing Int’l Bus. Sys., Inc., B‑275554, Mar. 3, 1997, 97-1 CPD para. 114 at 5‑6). Specifically, we consider PPQs in an agency’s possession to be past performance information too close at hand to ignore. Intercontinental Constr. Contracting, Inc.--Costs, B-400729.3, Mar. 4, 2009, 2009 CPD para. 44 at 2. Consideration of the PPQs was particularly material in this case since FEMA had not obtained any other third-party assessments of the quality of the firms’ past performance. Common sense dictates that an offeror’s self-assessment regarding the quality of its past performance, as reflected in its SF 330, is, by its nature, of less value as compared to the disinterested assessments provided by third parties. As noted above, a critical consideration in our review of an agency’s past performance evaluation is whether it is based on relevant information sufficient to make a reasonable determination of the firm’s overall past performance rating. University Research Co., LLC, supra at 16. By ignoring the PPQs it had at hand, the agency here failed to satisfy this standard in its evaluation of past performance.

FEMA maintains that its decision not to consider the PPQs was reasonable because the factors for evaluation under the PPQs did not match the factors for evaluation under the SSN. FEMA also asserts that such consideration would have been unfair given that the agency did not receive PPQs for some references for some firms and some of the PPQ responses addressed firms’ performance under individual task orders, rather than the firms’ overall contract performance. Based on these inconsistencies, the contracting officer determined that the SEB "would not further consider [the PPQs] in the evaluation." SEB Chairperson, Decl., Jan. 22, 2010, at 3. The agency’s arguments are unpersuasive.

As an initial matter, the record shows that FEMA prepared the PPQs and requested that firms distribute them to their references. It therefore strains credulity for FEMA to now claim that the PPQs were so fundamentally inadequate as to render them per se unusable. Second, the inconsistencies between the past performance elements as set forth in the PPQs and those identified in the SSN do not justify ignoring the only third-party assessments of the quality of the firms’ past performance obtained by FEMA. Specifically, the past performance elements identified for evaluation in the PPQs were "cost control," "quality of product or service," "timeliness of performance," and "business relations." While not identical, these topics closely parallel and appear to be directly relevant to the past performance evaluation categories identified in the SSN, which included "accuracy of cost estimates," "cost control," "completion of projects within budget," "quality control," "compliance with performance schedules," and "project management." SSN at 3.

In addition, to the extent some firms’ references did not return, or did not properly prepare, individual PPQs, these failings cannot justify the agency’s decision to ignore what are otherwise relevant PPQs with detailed information documenting the quality of the firms’ past performance. Such a conclusion would be at odds with the very nature of the "close at hand" principle, which reflects the duty of an agency to consider relevant information in its possession notwithstanding whether it was actually submitted by an offeror or whether the agency has sought similar information for other offerors. Moreover, the agency’s "fairness" concerns weigh in favor of giving consideration to those firms whose references responded appropriately with relevant past performance information, as opposed to denying these firms the benefit of this information because some of the references for the other firms failed to do the same.

FEMA also asserts that its decision to evaluate past performance based solely on the information provided by the firms in their SF 330s was consistent with the SSN and that sufficient information existed in the firms’ SF 330s to reasonably assess the quality of their past performance. Regarding the former point, FEMA notes that it was not required to consider the PPQs because under the terms of the SSN, FEMA merely reserved the right to use information outside of the response in evaluating past performance, including agency knowledge of the firm’s performance. As discussed above, however, the PPQs in this case were simply too close at hand for FEMA to have ignored. In other words, regardless of what discretion the solicitation may have afforded FEMA in seeking out additional information, once it had the PPQs, it could not simply ignore them.

Regarding the latter point, the SEB Chairperson asserted that the "SEB believed that each of the firms provided sufficient information in the SF 330 to allow the SEB to conduct this qualitative assessment." Decl. of SEB Chairperson, Jan. 22, 2010, at 3. This argument, however, is inconsistent with FEMA’s fundamental concern regarding the inadequacy of the past performance information contained in Shaw-Parsons’ SF 330. FEMA cannot on the one hand claim that it was justified in disregarding the PPQs because the SF 330s provided a sufficient basis to evaluate firms’ past performance, and then, on the other hand, assert that Shaw-Parsons’ "acceptable" past performance rating was justified based on a lack of information in its SF 330.  (Shaw-Parsons Infrastructure Recovery Consultants, LLC; Vanguard, B-401679.4; B-401679.5; B-401679.6; B-401679.7, March 10, 2010)  (pdf)

As discussed above, the record here reflects that the NMLC reviewed the past performance information of the offerors' five references for relevancy, quantity and quality, as well as relevant PPIRS information, and any other available past performance information; contacted various references; documented the evaluation in detail; and explained the basis for its judgments of each offeror's past performance, including Kuhana-Spectrum's.

Contrary to Kuhana-Spectrum's contentions, the agency can reasonably give differing weight to an offeror's prior contracts based upon their similarity or relevance to the required effort. TPL, Inc., B-297136.10, B-297136.11, June 29, 2006, 2006 CPD para. 104 at 12. Thus, the agency could reasonably give more weight to the documented instances of poor performance by Spectrum under the incumbent NMCP contract than the instances of good performance under less relevant contracts. See Del-Jen Int'l Corp., B-297960, May 5, 2006, 2006 CPD para. 81 at 8. While the protester states that medical positions are difficult to fill in the Portsmouth area, it does not dispute that Spectrum encountered problems associated with providing sufficient qualified health care personnel at the NMCP in accordance with the terms of the incumbent contract, which resulted in numerous contract discrepancy reports and the assessment of liquidated damages by the NMLC. The past performance record shows that this was a problem that Spectrum had encountered on a number of other contracts that were considered highly relevant because they involved the same labor bands as this RFP. Thus, the agency reasonably concluded that there was a significant risk concerning the protester's successful performance of the follow on contract here, so as to justify a moderate risk past performance rating, notwithstanding the numerous instances of successful contract performance by Spectrum and Kuhana reflected in the past performance evaluation record. The protester's mere disagreement with the agency's judgment is not sufficient to establish that the agency acted unreasonably. See Birdwell Bros. Painting & Refinishing, B-285035, July 5, 2000, 2000 CPD para. 129 at 5.

While the protester notes that the record shows that the awardees had some similar performance problems on other contracts (including some where references indicated that they would not make another award to the firm), yet received low risk past performance ratings, not only were Spectrum's performance problems related to the incumbent and other very relevant contracts, the evaluation record demonstrates that the agency, in the exercise of its discretion, reasonably found that the protester generally had deficiencies greater in number and/or severity in the context of the demonstrated past performance than the deficiencies and weaknesses found in the awardees' past performance. Moreover, the past performance evaluation documentation shows that each of the awardees had significant positive ratings and feedback regarding their past performance. We have reviewed the protester's detailed disagreements with how the agency considered the various offerors' past performance and find that the agency's risk ratings, including the awardees' low risk ratings, were reasonable and supported by the record.

Kuhana-Spectrum also protests that the NMLC improperly downgraded its proposal for not providing the name and qualifications of the site manager, scheduler, employer relations specialist and credentialer. Kuhana-Spectrum contends that identifying these individuals and their qualifications was not required by the solicitation; rather the protester argues that the solicitation only required Kuhana-Spectrum to identify the corporate personnel who would be responsible for start up and the ongoing administration of the key functional areas. The protester asserts that since its proposal identified these individuals, the NMLC was not justified in downgrading its proposal for not identifying and discussing these other individuals.

The evaluation of proposals is primarily a matter within the contracting agency's discretion which we will not question unless we find the evaluation to be unreasonable or inconsistent with the RFP's evaluation factors. Centro Mgmt., Inc., B-249411.2, Dec. 2, 1992, 92-2 CPD para. 387 at 5. It is an offeror's responsibility to submit an adequately written proposal that establishes its capability and the merits of its proposed technical approach in accordance with the evaluation terms of the solicitation. See Verizon Fed., Inc., B‑293527, Mar. 26, 2004, 2004 CPD para. 186 at 4. A protester's mere disagreement with the evaluation provides no basis to question the reasonableness of the evaluators' judgments. See Citywide Managing Servs. of Port Washington, Inc., B-281287.12, B-281287.13, Nov. 15, 2000, 2001 CPD para. 6 at 10‑11.

Here, the RFP specifically required the offeror's management plan to provide a persuasive written discussion demonstrating its contract management capabilities, and to describe and discuss how the responsbilities, experience and qualifications of each individual, responsible for ongoing adminstration of the contract, would contribute to successful contract performance. The solicitation cautioned that the evaluators would not assume that the offeror possesses any capability or knowledge unless specified in the proposal. Given the solicitation's instruction that the plan should demonstrate the offeror's management capabilities, and discuss how the responsibilities, experience and qualifications of each individual will contribute to successful contract performance, the Navy reasonably found, consistent with the terms of the solicitation, that Kuhana-Spectrum's failure to discuss the qualifications of individuals in its proposal was a weakness in its proposal.

In sum, we find the agency's evaluation of the proposals to be reasonable and consistent with the solicitation's evaluation scheme.  (Kuhana-Spectrum, B-401270, July 20, 2009)  (pdf)

ITT first argues that the agency's evaluation of its and RMS's proposals under the corporate experience evaluation factor was inconsistent with the terms of the solicitation and unreasonable. The protester asserts that the agency, in evaluating proposals under this factor, gave undue weight to whether the offeror's experience included contracts performed in Iraq or Afghanistan, and improperly credited RMS with corporate experience attributable to one of its subcontractors. Protest (B‑310106.6) at 16-20; Protester's Comments at 6-19; Protester's Supp. Comments at 2-11.

The RFP informed offerors that under the most heavily weighted corporate experience factor "[t]he Government [would] evaluate each offeror's (prime and significant subcontractors) experience," in order to "determine the degree to which the offeror has previously encountered the kinds of work, uncertainties, challenges, and risk that it is likely to encounter under the prospective contract," and to "develop insight into the offeror's relative capability and the relative risk associated with contracting with the offeror." RFP at 128. The solicitation stated that the agency would "evaluate the work performed for each reference for similarity and relevance to the work required under the contemplated contract," and that "[t]he more similar and relevant the work performed is to the contemplated work, the more valuable the experience is to the Government." Id. The solicitation specifically noted in this regard that "[e]xperience in [USCENTAF] [ATC], Airfield Management and [ATCALS] [AOR], specifically Iraq and Afghanistan, is more valuable than experience in other areas." RFP amend. 4, at 15. The solicitation further stated that in order to be considered a "significant subcontractor" for evaluation purposes, the subcontractor must be proposed to perform a minimum of 33 percent of the total proposed labor hours. RFP at 117.

The record reflects that the agency evaluated RMS's proposal under the corporate experience evaluation factor as "satisfactory," with the proposal receiving 46 of 60 available points, based on five contract references submitted by RMS as the prime contractor and five references submitted by RMS's "significant subcontractor" Midwest Air Traffic Control Service, Inc. AR, Tab 6C, BCM, at 23. The agency noted, for example, that with regard to RMS's performance of the predecessor contract as the incumbent contractor, RMS's proposal "demonstrated current experience within the . . . AOR in Tower, Radar, Non-radar and Air Field Management as well as managing the Air Traffic Control and Landing Systems maintenance support services," and that this constituted a "strength" given that it demonstrated an "understanding [of] the tempo of operations, as well as addressing the additional requirements and attributes required just to get the job done in an environment such as the AOR presents." Id.; see AR, Tab 6A, Technical Evaluation Board Report (TEB Report) (Mar. 3, 2009), at 12. The agency's evaluation here included other positive comments due to RMS's 11 years of experience as the incumbent contractor, including the agency's determination that this aspect of RMS's proposal was a strength in that it "demonstrated the ability to solve the unique supply support problems" that result from the environment in which the incumbent contract was performed. AR, Tab 6C, BCM, at 23; see AR, Tab 6A, TEB Report, at 12. In this regard, the agency commented that "[t]he current solicitation's performance work statement is based largely on the current contract's requirements," and that RMS had successfully performed that contract "as the incumbent for the past 11 years," including its handling of more than 1.3 million air traffic control tower transactions, with a number of these including air traffic control "transactions recorded at the Kabul Area Control Center, an activity that allows international flights over Afghan airspace." AR, Tab 6C, BCM, at 30. This verbiage, recognizing RMS's successful performance of the predecessor contract under the somewhat unique circumstances presented by the AOR, and specifically, operations in Iraq and Afghanistan, was repeated by the SSA in making her source selection decision, which also found that although both RMS and ITT received "satisfactory" ratings under the corporate experience factor, "the RMS Team provides a more comprehensive and germane level of corporate experience than ITT's Team." AR, Tab 6D, SSA Memorandum, at 4-5.

It is clear from the record that RMS's successful performance of the predecessor contract was considered favorably by the agency both during its evaluation of the proposals and its source selection, and was considered more favorably by the agency than ITT's experience performing similar work outside of the AOR and outside of Iraq and Afghanistan. However, we disagree with the protester that this aspect of the agency's evaluation was inconsistent with the terms of the solicitation, particularly given the solicitation's statement that "[e]xperience in [USCENTAF] [ATC], Airfield Management and [ATCALS] [AOR], specifically Iraq and Afghanistan, is more valuable than experience in other areas." RFP amend. 4, at 15. Indeed, our Office has recognized that an agency, even under generally-worded experience criteria, may properly consider the extent to which offerors have experience directly related to the work required by the RFP. Systems Integration & Dev., Inc. B-271050, June 7, 1996, 96-1 CPD para. 273 at 4. We also have recognized that a particular offeror may possess unique advantages or capabilities due to its prior experience under a government contract and that the government is generally not required to attempt to equalize competition or compensate for it. Crofton Diving Corp., B-289271, Jan. 30, 2002, 2002 CPD para. 32 at 6. Here, the agency, consistent with the corporate experience evaluation factor set forth in the solicitation, favorably considered RMS's successful performance of the predecessor contract in its evaluation of the proposals and source selection, and we find nothing in the record evidencing that the agency gave RMS's experience as the incumbent contractor "undue weight."  (ITT Corporation, Systems Division, B-310102.6; B-310102.7; B-310102.8; B-310102.9, December 4, 2009) (pdf)

In addition to the basic question of whether TMA could properly consider the past performance information submitted by AGHP, Health Net also challenges the agency’s substantive findings regarding the information it in fact considered in evaluating the past performance information submitted by AGHP. In this regard, Health Net contends that TMA failed to reasonably take into account the size of the prior contracts submitted by AGHP.

As explained above, the RFP indicated that for the purpose of evaluating past performance, TMA would consider an offeror’s performance on “relevant” contracts, which was generally defined to mean contracts “similar” to the T-3 requirements and specifically request offerors to submit their five “largest relevant” contracts. RFP at 96. In its evaluation of relevance, the agency’s methodology reflects that the size of the beneficiary population covered by a particular contract was a significant consideration, as evidenced by the fact that to achieve a rating of “relevant,” an offeror’s contract had to have covered a beneficiary population which was at least 75% the size of the T-3 contract population.

In challenging the agency’s evaluation of AGHP’s past performance, Health Net highlights the fact that AGHP’s contracts involved beneficiary populations that are a small fraction of the size of the beneficiary population covered under the T-3 contract. The record reflects that all but one of AGHP’s contracts were for beneficiary populations that are less than 3% the size of the T-3 population, with the one larger contract equal to 11% of the T-3 population.[9] Given that none of AGHP’s contracts were comparable to the T-3 effort in terms of the size of the covered beneficiary population, Health Net contends that it was unreasonable to have assigned AGHP the highest past performance rating of “High Confidence,” which was defined as “no doubt exists that the offeror will successfully perform the effort.” AR, Tab 86, SSEG, at 19.

TMA explains that AGHP’s “High Confidence” rating reasonably reflects an integrated assessment of AGHP’s past performance information, which considered the relevance of AGHP’s past performance, the qualitative aspects of the performance, as well as the past performance information submitted for AGHP’s subcontractor, WPS. Regarding relevance, TMA maintains that it specifically considered the fact that AGHP’s contracts were limited in terms of size; however, it also found that the functional aspects of the contracts were similar to the T-3 requirements. AR, Tab 10, Final PAG Report for AGHP. When these considerations were combined, TMA maintains, it reasonably found AGHP’s contracts to be “somewhat relevant.” Given the “exceptional” ratings in terms of its qualitative performance that AGHP received for these “somewhat relevant” contracts, coupled with the relevant past performance and “exceptional” ratings of AGHP’s subcontractor, WPS, TMA argues that it reasonably assigned AGHP a “High Confidence” rating.

While we recognize that the past performance evaluation was not to be based on size alone, TMA’s assertion that its integrated assessment of AGHP’s past performance information justified giving AGHP the highest past performance rating is unpersuasive. Not one of AGHP’s contracts was evaluated as “relevant”; rather, they were all considered to be only “somewhat relevant.” Whether it was reasonable to consider some of the contracts even “somewhat” relevant given that their beneficiary populations were a small fraction of the size of the beneficiary population covered by the T-3 contract is itself questionable. At a minimum, absent some further support in the record, it was not reasonable to give AGHP the highest past performance rating in reliance on the “exceptional” performance ratings associated with the prior contracts of such smaller size. On the contrary, the value of the “exceptional” ratings as predictors of AGHP’s success on the T-3 contract is inherently diminished by their lack of relevance due to their relatively small size.

In this regard, the SSEG implicitly recognized the need to consider size when deciding what weight to give to an offeror’s prior contracts; it specifically advised that “[r]elevance will increase as the size of the historical efforts increase” and instructed TMA evaluators to “[g]ive the greatest weight to the information determined to be the most relevant and significant.” AR, Tab 86, SSEG, at 15, 18.

Moreover, we conclude that the agency’s consideration of the relevant past performance of AGHP’s subcontractor, WPS, could not have reasonably justified AGHP’s past performance rating. To the extent WPS had “relevant” and “exceptional” past performance, WPS’s role in performance was limited to [Deleted] of the many T-3 functional requirements, [Deleted]. This left AGHP, as the prime contractor, responsible for all other T-3 requirements, including [Deleted]. Thus, while AGHP, through WPS, demonstrated relevant experience for [Deleted] under the RFP, a significant portion of the contract was to be in the hands of AGHP, which had only “somewhat relevant” experience.

During a hearing conducted by our Office, the SSA appeared to recognize that AGHP lacked past performance of a magnitude contemplated under the T-3 contract and the concern this created. Specifically, he testified:

THE WITNESS: . . . I saw the four contracts, somewhat relevant, the biggest one being, I believe, [Deleted] [beneficiaries], and it dropped off to [Deleted]. I also asked the same question of the team, and the SSEB. How can we take somebody with just these five contracts?
Hearing Transcript (Tr.) at 1574-75.

His answer to this concern was that TMA looked at generalized information about “Aetna” by essentially aggregating all of “Aetna’s” commercial activities and thereby concluding that “they” could do the job, notwithstanding the fact that none of the contracts actually submitted for the purpose of evaluating AGHP’s past performance reflected a magnitude comparable to the T-3 contract. Tr. at 1575, 1597, 1609-19. In this regard, the SSA testified that had it not been for Aetna’s other commercial experience, TMA probably would not have assigned AGHP a “High Confidence” rating.

QUESTION: If they didn’t have that [commercial] experience, would you have given them the high confidence rating?

THE WITNESS: Probably not. We had an extensive discussion on that.
Tr. at 1626.

Setting aside the fact that any analysis in this regard was not documented in the contemporaneous record, reliance on such an analysis would have been problematic. Based on the SSA’s testimony, there is no indication that TMA understood which Aetna entities performed any of the “commercial” work considered, there was no information or analysis regarding the magnitude of any of these “commercial” activities or plans, it was not apparent how the work was relevant to the T-3 effort, nor was there any information or analysis regarding how Aetna performed qualitatively in connection with the undefined “commercial” experience.[10] Tr. at 1577, 1617-20. Given the limited information available to the agency in the record, it is difficult to understand how this possibly could have served as a basis for AGHP’s high confidence past performance rating.

In sum, based on the fact that AGHP’s past performance submitted for evaluation was with respect to contracts that were small fractions of the size of the T-3 effort, TMA’s decision to assign AGHP the highest past performance rating of “High Confidence” is not supported by the record. See Continental RPVs, B‑292768.2, B‑292768.3, Dec. 11, 2003, 2004 CPD para. 56 at 12 (finding past performance rating of low risk was not supported where awardee’s contracts were a mere fraction of the size of the contemplated contract).  (Health Net Federal Services, LLC, B-401652.3; B-401652.5, November 4, 2009)  (pdf)

CommSol protested on July 2 after receiving notice of the awardee and contract price. In its protest, CommSol complains that the agency improperly evaluated the vendors' past performance and unreasonably awarded the contract to Riverhawk--a higher-priced vendor--who had less or equal experience than CommSol on USCG projects. Specifically, CommSol contends that the agency deviated from the stated evaluation criteria by crediting Riverhawk's quotation with an "excellent" rating based on only one contract that was fully performed and one contract that had not yet been completed at the time of the evaluation, when the solicitation required that vendors identify at least two fully-performed contracts. CommSol also asserts that the agency improperly assigned its quotation a "neutral" rating.

Where a solicitation contemplates the evaluation of vendors' past performance, as is the case here, the contracting agency has the discretion to determine the relevance and scope of the performance history to be considered, and our Office will not question the agency's judgment unless it is unreasonable or inconsistent with the terms of the solicitation or applicable procurement statutes and regulations. National Beef Packing Co., B-296534, Sept. 1, 2005, 2005 CPD para.168 at 4; Sam Facility Mgmt., Inc., B-292237, July 22, 2003, 2003 CPD para. 147 at 3. A protester's mere disagreement with the agency's judgment does not establish that an evaluation was unreasonable. Sam Facility Mgmt., Inc., supra, at 3. Based on our review of the record, as discussed below, we find the USCG's evaluation of past performance to be reasonable.

With regard to the evaluation of Riverhawk's quotation, the protester is correct that vendors were required to identify at least two relevant contracts that had been 'performed during the last three (3) years." RFQ at 26. However, the RFQ states only that the contracts must be performed during the past 3 years, not that performance must be completed prior to the evaluation. Where, as here, the RFQ only requires performance and not completed performance, we will not find unreasonable an agency's decision to consider performance of ongoing contracts. See Sam Facility Mgmt., Inc., supra, at 7 (challenge to the agency's past performance evaluation that considered ongoing contracts was denied because the solicitation did not state that contracts must have been completed within the last 5 years, only that they must have been "performed" within the last five years). As such, we find the USCG's evaluation of Riverhawk's past performance, which included consideration of Riverhawk's uncompleted Pea Island contract but relied primarily on the Key Biscayne contract, to be reasonable and in accordance with the terms of the solicitation.

We also find reasonable the agency's assessment of CommSol's past performance--both the agency's determination that CommSol's contracts were not relevant and the agency's assessment of a "neutral" rating to CommSol's quotation. In this regard, the record confirms that none of CommSol's contracts were relevant because they were not of similar complexity, scope, or dollar value to this acquisition. All three contracts were significantly less in value than the independent government estimate for this acquisition, and the two largest contracts did not include the same work as was required here. For example, CommSol's two largest contracts were for flushing engines, which the agency explains are only a small part of the 67 dry dock requirements that would be performed here. CommSol's third contract was for dry dock work, but the contract was only for $35,546.00, which was only a fraction of the government estimate of $534,700.00. Furthermore, the USCG received no input from contractor references, despite the agency's reasonable attempts to obtain such information from references for the two largest contracts. Given the absence of relevant past performance and reference responses, we find that the agency could reasonably assess CommSol’s quotation a "neutral" past performance rating. CMC & Maint., Inc., B-292081, May 19, 2003, 2003 CPD para. 107 at 3.

Finally, the protester asserts that the agency erred in awarding the contract to a higher-priced vendor who, according to the protester, had "less or equal" past performance. Protest at 2. However, as discussed above, the record supports the agency's determination that Riverhawk had a more relevant and excellent record of performance, and the agency reasonably considered this record to be worth the price premium associated with the Riverhawk's quotation. Since the RFQ stated that past performance was "significantly more important than" price, we find the agency's best value analysis and award selection to be unobjectionable.  (Commissioning Solutions Global, LLC, B-401553, October 6, 2009) (pdf)

Section L of the RFP provided that an offeror's experience would be evaluated based upon its demonstrated housing maintenance experience with recent, relevant projects. RFP sect. L, at 36. Past performance was to be evaluated based upon client satisfaction on recent, relevant housing maintenance services projects within the last 3 years. Id. Relevance was defined as "sufficiently similar" to the RFP's work to provide an indication of expected performance, based on such indicators as construction similarity and complexity, contract type, dollar value, major or critical subcontractors, teaming partners and joint ventures. Id. The RFP distinguished experience from past performance as follows: "experience pertains to the types of work and volume of work completed by a contractor that are comparable to the type of work covered by this requirement, in terms of size, scope, and complexity [;] [p]ast performance relates to how well a contractor has performed." RFP sect. L, at 37. Section M of the RFP provided that the agency would evaluate an offeror's past performance on relevant projects and its experience information based on the degree of relevance--the more relevant an offeror's experience, the greater the degree of significance that would be applied in the evaluation. RFP sect. M, at 19.

AAJV asserts that the agency failed to follow these evaluation criteria in assigning AAJV a satisfactory rating under the experience and past performance factors, because it introduced an undisclosed limitation of $5 million in annual value as a threshold for considering projects relevant. Protest at 6, 8. In this regard, AAJV submitted 13 projects and the agency found that 12 were relevant from the standpoint of the scope of services. AR, Tab 4A, attach. 1. However, the agency found that only one of the projects was similar in size (e.g., some 7 of the projects were valued at less than $2 million per year) or complexity, and thus found them not relevant under both the experience and past performance factors. In the protester's view, instead of considering only its one project above the threshold value to be relevant, the agency also should have accorded some degree of relevance to its projects below the threshold, based on the other relevance considerations.

This argument is without merit. In evaluating proposals, an agency properly may take into account specific, albeit not expressly identified, matters that are logically encompassed by, or related to, the stated evaluation criteria. Independence Constr., Inc., B‑292052, May 19, 2003, 2003 CPD para. 105 at 4. Size is a proper consideration in determining whether an offeror has experience and a record of past performance under similar contracts. See Molina Eng'g, Ltd./Tri-J Indus., Inc. Joint Venture, B‑284895, May 22, 2000, 2000 CPD para. 86 at 7; Proteccion Total/Magnum Sec., S.A., B‑278129.4, May 12, 1998, 98-1 CPD para. 137 at 6. Here, the agency established a minimum relevance value of $5 million; the estimated contract value of the first option year was expected to be approximately $11 million, and the agency determined that a contract approximately half that size would be sufficient to be predictive of the quality of performance of the current requirement. Establishing a threshold value in this manner was sufficiently related to the relevance criterion, and we find nothing inherently unreasonable in a threshold of approximately one-half the value of the current requirement. While AAJV believes, essentially, that a lower contract value should not have precluded consideration of its other, lower value projects in the evaluation, there simply was no requirement that the agency give weight to such projects.

Considering the foregoing, there was nothing unreasonable in the agency's evaluation of AAJV's experience and past performance as satisfactory. The agency received two past performance records on behalf of AMI--the majority member of the joint venture--one of which was for maintenance and repair of retail gasoline stations (rated satisfactory) and one of which was for custodial services (rated exceptional), but neither was considered relevant. AR, Tab 10; AAJV Past Performance Proposal. For ACEPEX--the minority member--the agency received past performance information on 8 different projects, with past performance ratings ranging from good to excellent. AR, Tab 10; AAJV Past Performance Proposal. As discussed above, only one of these projects was considered relevant--a contract at the Federal Law Enforcement Training Center (FLETC), which involved both HOM and COOM, was worth $6.5 million per year, and had an overall past performance rating of good. Despite AAJV's above-satisfactory ratings on the FLETC and other projects, the fact remains that the majority of its projects were significantly smaller and less complex than the contract to be awarded, and that only one project ultimately was determined to be relevant for evaluation purposes. This being the case, we find nothing objectionable in the agency's evaluating AAJV's proposal as satisfactory for experience and past performance. Robinson's Lawn Serv., B‑299551.5, June 30, 2008, 2009 CPD para. 45 at 7 (where prior contracts were significantly smaller than the contract at issue and the work was less complex, the agency reasonably rated offeror's experience and past performance as satisfactory, notwithstanding high past performance ratings for those contracts)  (AMI-ACEPEX, Joint Venture, B-401560, September 30, 2009) (pdf)

JSW Maintenance, Inc. of Warner Robins, Georgia, protests the award of a contract to Ashley-Marie Group, Inc. (AMG) of Elizabethtown, North Carolina, by the Department of the Air Force under request for proposals No. FA8501-08-R-0039 for grounds maintenance services for Robins Air Force Base (AFB), Georgia. JSW alleges that the agency’s evaluation of AMG’s past performance was unreasonable and that the agency failed to reasonably consider the fact that JSW and AMG did not have a similar understanding of the agency’s requirements when they submitted their proposals.

(sections deleted)

JSW’s arguments regarding the relevance of AMG’s past performance record are also without merit. The record reflects that the Air Force considered past performance information submitted by AMG with respect to four contracts performed by AMG’s key personnel. These included: (1) a base-wide grounds maintenance contract for Cannon AFB, New Mexico, which the Air Force considered to be “relevant”; (2) a grounds maintenance services contract for the Federal Law Enforcement Training Center (FLETC), Glynco, Georgia, which the Air Force also evaluated as “relevant”; (3) a base-wide grounds maintenance services contract, for the Naval Support Activity in Mechanicsburg, Pennsylvania, which the Air Force, evaluated as “somewhat relevant”; and (4) a post-wide grounds maintenance services contract at Fort Jackson, South Carolina, which the Air Force considered to be “somewhat relevant.” Agency Report (AR), Tab 12b, Performance Confidence Assessment Group Report, at 3-4.

In challenging the agency’s assessments regarding the relevance of AMG’s contracts, JSW highlights the fact that AMG’s contracts were for lower dollar values and did not involve some of the requirements of the Robins contract. In this regard, JSW argues that the Air Force’s relevancy assessments of the Cannon AFB and the FLETC contracts were unreasonable given that the dollar values of the two contracts were approximately one-quarter and one-half of the Robins AFB contract, respectively. In addition, JSW argues that the Cannon AFB contract is dissimilar because the air base is located in a desert climate, which presents fewer grounds maintenance challenges than Robins AFB, and because the contract did not involve composting operations. According to JSW, the FLETC contract should not have been considered relevant because, in addition to being smaller in dollar amount than the Robins AFB contract, it did not require composting operations or airfield maintenance. JSW also takes issue with the Air Force’s assessment of the Navy contract in Mechanicsburg and the Fort Jackson contract as “somewhat relevant.” JSW contends that, unlike the Robins AFB contract, these contracts are for smaller values and do not involve many of the Robins contract requirements.

We find the Air Force’s assessments of the relevance of AMG’s past performance references to be reasonable and consistent with the evaluation scheme set forth in the solicitation. As noted above, the RFP defined the various degrees of relevance principally in terms of the type and extent of the work effort in comparison to the work required at Robins AFB, as opposed to particular dollar values. Thus, the fact that the dollar values of AMG’s contracts were lower than the Robins AFB contract is not determinative of the reasonableness of the agency’s evaluation. In this regard, the definition of “very relevant” past performance included benchmark examples of the types of grounds maintenance activities and numbers of maintained acres. These examples provided a guide to the various relevancy ratings, since the ratings were defined by reference to these activities and the amounts, and kinds, of acreage specified.

In assessing AMG under the past performance factor, the record reflects that the Air Force compared the types of activities performed and the number of acres involved under AMG’s referenced contracts, with the activities and acres identified in the RFP’s definitions of relevance. Specifically, the Air Force concluded that the Cannon AFB contract, which was a performance-based contract, was relevant based on the fact that it required similar grounds maintenance activities, such as maintaining 63 acres of enhanced grounds, 346.6 acres of improved grounds, 1,584 acres of semi-improved grounds, and 18.4 miles of perimeter fence grounds.[2] The Air Force indicated that the Cannon AFB contract was not considered “very relevant” because it lacked composting operations. AR, Tab 12b, Performance Confidence Assessment Group, at 4. Similarly, the FLETC contract was considered relevant where it involved maintaining 17 acres of enhanced grounds, 680.54 acres of improved grounds, and 456.35 acres of semi-improved grounds, amounts and kinds of acreage which were comparable to those required at Robins AFB. In rating this contract as relevant, as opposed to very relevant, the Air Force specifically considered the fact that the contract did not include airfield maintenance and composting operations. Id. at 5.

The Navy Mechanicsburg and Fort Jackson contracts were considered only “somewhat relevant,” which, as indicated above, was defined as involving some of the magnitude of effort and complexities required under the RFP. The Air Force’s evaluation of these contracts as only “somewhat relevant” was based in part on the fact that they did not involve airfield maintenance or composting operations. In addition, the number of acres maintained under the Navy Mechanicsburg contract was considered to be “significantly less than what is required by the solicitation,” and, while the acres maintained under the Fort Jackson contract were comparable to the Robins AFB contract, the effort did not include maintenance around runways, maintenance of irrigation systems, or perimeter fencing. Id. at 6.

Based on this record, which reflects the Air Force’s consideration of the activities, scope, and complexity of AMG’s past performance information, we have no basis to conclude that the Air Force’s evaluations regarding the relevance of AMG’s past performance references was unreasonable.

JSW also challenges the agency’s consideration of AMG’s past performance record in the aggregate when it assigned AMG a “confidence” past performance rating. Notwithstanding the fact that the solicitation expressly authorized the Air Force to consider an offeror’s past performance information in the aggregate for the purposes of making its overall confidence assessment, aggregation of AMG’s contracts was not appropriate, according to JSW, because the smaller values of its contracts do not reasonably lend themselves to aggregation. In JSW’s view, a long-term, high dollar value contract such as the Robins AFB contract is objectively different in terms of magnitude and complexity from AMG’s smaller contracts, and therefore presents fundamentally different challenges from the contracts which formed the basis of AMG’s past performance record. The agency, however, decided aggregation was appropriate given that AMG’s past performance record was based on contracts that were in many instances performed concurrently, and the obligations, when combined, exceeded the Robins requirements for maintaining improved grounds, semi-improved grounds, airfield grounds, and perimeter fencing. AR, Tab 13B, Source Selection Decision, at 6. While JSW may ultimately disagree with the agency’s aggregation assessment, its disagreement does not render the agency’s decision unreasonable.  (JSW Maintenance, Inc., B-400581.5, September 8, 2009)  (pdf)

As an initial matter, the protester’s arguments here are in large part predicated on its misunderstanding or mischaracterization of the solicitation’s terms. In this regard, and contrary to the protester’s arguments, the RFP did not state that a contract would be considered by the agency in its evaluation under the experience criterion only if the referenced contract was equivalent to the anticipated contract under all of the relevant experience criteria, that is, size, scope and complexity. Rather, the solicitation advised offerors that the experience information provided in their proposals would “be reviewed for relevancy,” and that “[r]elevance [would] be based on services similar in size, scope, and/or complexity to the work defined in the PWS.” RFP at 30 (emphasis added). As such, it was reasonable for the agency, as it did here, to consider the performance of contracts, under the experience criterion of the technical factor, that were similar to the work required here in terms of size, or scope, or complexity. Contracting Officer’s Statement at 3; see KIC Dev., LLC, B‑297425.2, Jan. 26, 2006, 2006 CPD para. 27 at 5; Roy F. Weston, Inc., B-274945, et al., Jan 15, 1997, 97-1 CPD para. 92 at 8-9.

We also note that the protester’s characterization of the term “similar” in the context of the solicitation’s statement that “[r]elevance [would] be based on services similar in size, scope, and/or complexity,” is too narrow. For example, the protester argues that two of Five Rivers’ contracts considered by the agency in evaluating Five Rivers’ proposal as “acceptable” did not involve “similar services” because, while they involved VI services, they “are, in reality, on the lower end of complexity in the overall spectrum of VI functions.” Protester’s Comments at 37, 46. The protester continues this argument by detailing each requirement of the RFP’s PWS that Five Rivers may not have had to perform in its past or current VI services contracts, and arguing that because of this, Five Rivers’ experience in performing VI services is not “similar” to the VI services to be performed here.

Although the protester may be correct in its assertion that Five Rivers has not performed a VI services contract as complex as that contemplated by the PWS here, that does not render the agency’s determination that Five Rivers has experience performing similar services unreasonable. As explained below, the record reflects that Five Rivers has performed or is performing VI services and other similar contracts, and that the agency reasonably determined that the scope of these contracts with regard to the complexity of the services, while not that same, are “similar” to those required by the RFP’s PWS.

In this regard, Five Rivers’ proposal provided information regarding past and current contracts at, among others, Patrick Air Force Base (AFB), Pope AFB, Cape Canaveral, and Fort Hood, and included a table that individually identified the RFP’s PWS requirements, and then provided an explanation as to why Five Rivers’ experience was relevant to the PWS requirements identified. AR, Tab 5, Five Rivers’ Proposal, at II-2 -- II-3. The proposal also included detailed descriptions of each of the contracts listed, providing, for example, each contract’s value, performance period, place of performance, and cognizant contracting officer contact information. Id. at III-2 -- III‑10.

The agency evaluators referenced Five Rivers’ experience in performing the contracts at Fort Hood, Pope AFB, and Patrick AFB in finding Five Rivers’ proposal “acceptable” under the technical factor. The record reflects that under the Fort Hood contract, valued at $12,836,408, Five Rivers provided, among other things, “a variety of training aids and devices, including the creation, design, fabrication; modification, installation, and reproduction of exhibits and scaled models of Army equipment and combat aids,” to the Fort Hood Training Support Center. AR, Tab 5, Five Rivers Proposal, at III-2. The agency found that while the services performed by Five Rivers here were for training aids and devices, rather than specifically for VI, the services involved functions, such as work control, property control, and system administration, similar to those necessary for the successful performance of the VI services at Fort Knox. AR at 6; Tab 8, Price Negotiation Memorandum, at 5. The agency also noted in evaluating Five Rivers’ proposal under the technical factor that the Pope AFB and Patrick AFB contracts, while lower in value (at $865,353 and $2,410,559, respectively) than the Fort Knox contract, involved the provision of VI services. AR at 6; Tab 5, Five Rivers Proposal, at III-6, III-8; Tab 8, Price Negotiation Memorandum, at 5. Specifically, the record reflects that under these contracts Five Rivers operates the Pope AFB and Patrick AFB VI service centers, and in doing so, provides products and services, such as “still electronic imaging, graphic services, to include web support, compact digital disk recording, and creating multimedia presentations.” AR, Tab 5, Five Rivers Proposal, at III-7, III-8. These services also include “still and video photography both on location and in-studio,” as well as “[g]raphic services includ[ing] computer graphic files, or actual graphic products such as illustrations, reproduction masters, viewgraphs, charts, signs, posters, displays, and short presentations.” Id.

Giving due deference to the agency’s broad discretion to determine whether a particular contract is relevant to the evaluation of an offeror’s experience, we believe that the agency’s consideration of the Fort Hood, Pope AFB and Patrick AFB contracts was reasonable, as was the agency’s determination that Five Rivers has performed or is performing contracts involving “services similar in size, scope, and/or complexity to the work defined in the PWS.” See RFP at 30.  (K-Mar Industries, Inc., B-400487, November 3, 2008)  (pdf)

With respect to the Navy's evaluation of JVSCC's proposal under the organizational and team experience factor, JVSCC's marginal rating reflected the TEB's judgment that four of the six contracts that the protester identified in its proposal were not relevant, and therefore were not considered in assessing JVCC's experience. AR, Tab 6, TEB Report, at 36. The Navy explains that, in evaluating the degree to which an offeror had completed relevant projects, it used the four relevance elements identified in the RFP, under which a project would not be considered relevant where an offeror failed to satisfy any two of the elements for that project. Supplemental AR, at 1-2. JVCC argues, however, that because the RFP did not specify that projects had to meet any minimum number of relevance elements to be considered relevant, the agency's evaluation methodology was arbitrary and inconsistent with the terms of the RFP. Moreover, the protester contends that the agency's relevance evaluation was inaccurate and unreasonable.

We find from our review of the record no basis to object to the agency's assignment of an overall marginal rating to JVSCC's proposal under the organizational and team experience factor. As described above, the RFP required that each of the six projects identified by the offeror in its proposal must have been performed by the offeror as a "prime contractor," and that, where, as here, the offeror was a joint venture, each joint venture partner must have performed at least one of the projects as a prime contractor or as part of a joint venture that was the prime contractor. See RFP amend. 1, at 35. One of the projects identified by JVSCC in its proposal--the only one identified as being performed by one of the JVSCC joint venture partners, CDM--was for the "construction of electrical services" for another firm, "Mirabella S.p.a.," for the construction of a "bowling centre" at a Navy support site. See JVSCC Technical Proposal at 2, 4. The Navy concluded that this project could not be considered relevant because CDM performed this work as a subcontractor to Mirabella and not as a prime contractor.

JVSCC argues, however, that because Mirabella is the property owner and the Navy only uses the facility, CDM actually performed the electrical work as a prime contractor. The Navy responds that because CDM did not bear overall responsibility for the job of building the bowling center, but instead performed only the electrical work, CDM cannot be considered the prime contractor for the project. Supplemental AR, at 3.

We find, from our review of the record, no basis to disagree with the Navy's judgment that CDM performed the work on this project as a subcontractor. The record shows that JVSCC's proposal simply stated that CDM performed the electrical work for another firm, Mirabella, for a bowling center being built for the Navy, which reasonably indicated that CDM's work on this project was performed as a subcontractor. It is an offeror's responsibility to submit a well-written proposal, with adequately detailed information which clearly demonstrates compliance with the solicitation and allows a meaningful review by the procuring agency. See CACI Techs., Inc., B-296946, Oct. 27, 2005, 2005 CPD para. 198 at 5. Moreover, JVSCC's arguments, here, do not demonstrate that the Navy's conclusions regarding CDM's performance of this project were unreasonable. In sum, the Navy reasonably found that CDM's work on this project (the only project provided for this joint venture partner) was not done as a prime contractor and therefore the Navy's marginal rating of JVSCC's proposal under this factor was consistent with the RFP.  (JVSCC, B-311303.2, May 13, 2009) (pdf)

The RFP provided for the award of a fixed-price contract for the construction of a new ambulatory care center and associated support systems and for minor demolition work at the VA Pittsburgh Health Care System, H. J. Heinz Division, Pittsburgh, Pennsylvania.

(sections deleted)

Burchick also complains that the VA’s assessment of Burchick’s “lack of corporate healthcare construction experience with projects of the Ambulatory Care Center’s type, size and complexity as well as the lack of experience in that area for the projected staff” is unreasonable because Burchick’s proposal identified numerous examples of such construction experience.

Our Office reviews challenges to an agency’s evaluation of proposals only to determine whether the agency acted reasonably and in accord with the solicitation’s evaluation criteria and applicable procurement statutes and regulations. Cherry Rd. Techs.; Elec. Data Sys. Corp., supra, at 6. A protester’s mere disagreement with the agency’s judgment is not sufficient to establish that an agency acted unreasonably. Entz Aerodyne, Inc., B-293531, Mar. 9, 2004, 2004 CPD para. 70 at 3.

Here, we find from our review of the record no basis to question the VA’s determination that Burchick had limited construction experience with projects of the type, size and complexity of the ambulatory care center sought by the RFP. Although Burchick identified six projects in its proposal’s discussion of the firm’s corporate project experience, only one of these six projects related to hospital and clinical construction, and that was for an “OB/GYN surgery center” with a stated cost of $16.8 million. The remaining projects identified in the protester’s proposal were for the construction of a federal office building, a business park, a U.S. Army training center, a multi-purpose academic complex, and a national cemetery. Burchick Technical Proposal, sect. A.1. In addition to these projects, Burchick described in its proposal other “Corporate Project Experience Relevant Healthcare Projects,” see id., which Burchick states consists of “different types of equipment upgrades and replacements in area hospitals,” including one project that was a multi-million dollar operating room suite that was “the most technically advanced suite within [the University of Pittsburgh Medical System].” Protester’s Comments at 5. The proposal identifies a number of “types of suites and/or equipment projects” that Burchick states it has completed over the last 5 years, which were collectively worth “in excess of $40 million.” Burchick Technical Proposal, sect. A.1, Corporate Project Experience Relevant Healthcare Projects, at 2. Although Burchick was informed during discussions that the VA found that the firm had limited clinic and hospital construction experience, Burchick identified no further experience in its revised proposal. See Burchick Response to Discussion Questions, Nov. 17, 2008, sect. A, at 9.

The record shows that the TEB considered Burchick’s identified experience and found that it demonstrated limited construction experience similar to the project solicited here. See AR, Tab 2, Declaration of TEB Chair, at 1-2. Given that Burchick identified only one healthcare construction project of similar size, scope and complexity (and that project was less than half of the value of ambulatory center sought here), and otherwise identified numerous other construction projects that were not of similar size, scope, and complexity, we find reasonable the contracting officer’s judgment that Burchick lacked corporate experience building hospital and clinical space that is similar in size, scope and complexity to the proposed facility here.  (Burchick Construction Company, B-400342.3, April 20, 2009)  (pdf)

Honeywell argues the agency's evaluation of ITT's past performance was unreasonable. The protester contends that NASA could not and/or should not have relied on ITT's MSP contract in its past performance evaluation, given both the instruction provisions of the RFP and the prior contract's lack of similarity in size. Honeywell also alleges the only other prior contract for ITT (prime), being found only "somewhat relevant" with good performance, does not support the agency's rating of the awardee's past performance as excellent.

Where a solicitation requires the evaluation of offerors' past performance, we will examine an agency's evaluation to ensure that it was reasonable and consistent with the solicitation's evaluation criteria. The MIL Corp., B-297508, B-297508.2, Jan. 26, 2006, 2006 CPD para. 34 at 10; Hanley Indus., Inc., B-295318, Feb. 2, 2005, 2005 CPD para. 20 at 4. The critical question is whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation's evaluation scheme. Clean Harbors Envtl. Servs., Inc., B-296176.2, Dec. 9, 2005, 2005 CPD para. 222 at 3. The agency's past performance evaluation of ITT here does not meet this standard.

As a preliminary matter, we do not think that the agency here was precluded from considering ITT's MSP contract for past performance evaluation purposes simply because its value was below the $50 million figure referenced in section L of the RFP. As noted above, while the RFP instructed offerors to submit past performance information on relevant contracts of at least $50 million, it also expressly permitted them to submit additional information if they considered it necessary to establish a record of relevant past performance. RFP sect. L at 00775-76.

Once having decided to consider ITT's MSP contract, however, the agency clearly was required to evaluate the relevance of that contract consistent with the evaluation criteria in the RFP, i.e., the degree of similarity in size, content and complexity between an offeror's past performance information and the RFP requirements. There is nothing in the contemporaneous record to suggest that NASA engaged in any such analysis concerning the relative size of ITT's MSP contract and the size of the RFP requirements. Rather, the SEB report indicates the evaluators' determination that ITT's MSP contract was "very relevant" was based entirely on the type of services involved in that contract. The extremely low dollar value (and staffing level) of the MSP contract relative to those of the SCNS requirements clearly raise a question as to the degree to which the MSP contract reasonably may be regarded as similar in size to the RFP requirements, such that it properly could be considered in evaluating ITT's past performance. See Continental RPVs, B-292768.2, B-292678.3, Dec. 11, 2003, 2004 CPD para. 56 at 8 (finding prior contracts no larger than 4 percent of the solicitation requirements were not similar or relevant); Si-Nor, Inc., B-292748.2 et al., Jan. 7, 2004, 2004 CPD para. 10 at 16-17 (finding in part a prior contract which represented less than 7 percent of the solicitation requirements was not similar in size, scope, and complexity). Quite simply, the record here lacks explanation as to why the SEB found the MSP contract to be "very relevant" notwithstanding its extremely small size relative to the RFP requirements. We fail to see, and the record fails to reflect, how NASA determined that a contract similar as to size but not as to content (i.e., ITT's JSC contract) was only "somewhat relevant," while, by contrast, a contract similar as to content but not as to size (i.e., ITT's MSP contract) was "very relevant."

We recognize that the agency's evaluation of ITT's past performance also included nine other contracts for its major subcontractors, many of which the SEB found to be "highly relevant" and having excellent performance. The record reflects, however, that ITT (prime) had only two contract references: the JSC contract which NASA found of such limited relevance that it admittedly did not consider it in the evaluation of the offeror's performance; and the MSP contract which, as detailed above, was significantly smaller in size than the RFP requirements. In this regard, ITT (prime) was to perform all the program management requirements, a large majority of the systems engineering requirements, and [DELETED] percent of the total SCNS contract. As a result, based on the current record, the agency's conclusion that ITT had "highly relevant" past performance lacks a reasonable basis, given that it is based in material part on consideration of the MSP contract.

Honeywell also argues that NASA's evaluation of Honeywell's own past performance was unreasonable because the evaluators failed to give proper credit to the past performance of its major subcontractor, [DELETED], in the area of systems engineering. The SEB found that [DELETED] had demonstrated both "highly relevant" and excellent performance in the area of systems engineering. The protester maintains the agency evaluators failed to give proper weight to that performance, however, on the mistaken ground that Honeywell (not [DELETED]) was proposed to lead and perform the majority of the systems engineering effort. Honeywell contends its proposal gave [DELETED] a leadership role with regard to systems engineering, as evidenced by the assignment of the SCNS [DELETED] position to [DELETED] and the fact that [DELETED] of [DELETED] engineers for SN sustaining engineering task are [DELETED] personnel. Protest, Oct. 20, 2008, at 33-35; Protest, Dec. 4, 2008, at 41-42.

Contrary to the protester's assertions, Honeywell's proposal indicated that it would lead and perform the majority of the systems engineering and development efforts. For example, Honeywell's organizational chart indicated its employees would serve in most engineering leadership roles (e.g., network operations division manager, network project division manager, systems engineering and hardware engineering department manager, functional leaders for the software engineering and hardware engineering departments). AR, Tab 49, Honeywell FPR, at 12671. Honeywell's proposal also indicated its [DELETED] position would be staffed part-time by [DELETED] and part-time by another proposed subcontractor, [DELETED]. Id. at 14601-02. Further, Honeywell's cost proposal indicated that it (not [DELETED]) would provide the majority of systems engineers for the core requirements, the ID/IQ tasks, and the RTO TIPs. In its evaluation of the protester's past performance, the SEB took into account the roles Honeywell and [DELETED] each would play in the performance of the SCNS contract when determining the relevance of their prior contracts. Id., Tab 80, Final SEB Report, at 23754.

We need not decide the exact percentage of systems engineering work to be performed each by [DELETED] and Honeywell to conclude the agency reasonably determined that Honeywell would lead and perform the majority of the SCNS systems engineering and development efforts as part of the evaluation of the offeror's past performance. Honeywell's proposal clearly indicated its employees would fill the majority of engineering leadership positions. The protester does not dispute that the [DELETED] position was to be split between [DELETED] and another subcontractor. Protest, Dec. 4, 2008, at 41-42. Moreover, even if [DELETED] of [DELETED] systems engineering positions for the SN sustaining engineering task are [DELETED] employees, that means that [DELETED] of [DELETED] (or 57 percent) of the positions here are not [DELETED] employees. In sum, the agency here properly considered the roles to be played by Honeywell and [DELETED] in the performance of the SCNS in making the past performance evaluation.

CONCLUSION AND RECOMMENDATION

The record shows that in evaluating ITT's past performance, the agency relied in material part on ITT's MSP contract, without explaining why, given its low dollar value, that contract reasonably may be regarded as similar in size to the effort under the contract to be awarded here, such that, under the terms of the RFP, it properly could be considered in the evaluation. As a result, we sustain the protest on this basis.  (Honeywell Technology Solutions, Inc., B-400771; B-400771.2, January 27, 2009) (pdf)

We find that the past performance requirements included in the RFP amendment are unobjectionable. Federal Acquisition Regulation (FAR) sect. 15.305(a)(2)(i), which is applicable to this procurement, provides that “[the] currency and relevance of the information” regarding past performance shall be considered in the past performance evaluation. Consistent with this FAR section, the Air Force explains that the solicitation language requesting past performance information from the contractors is standard language based on the Air Force Past Performance Evaluation Guide (PPEG) IG5315.305(a)(2)(at 7) that provides that 3 years is the standard time period to define “currency” of past performance to be considered in the evaluation of proposals. Moreover, consistent with FAR sect. 15.305(a)(2)(iii), the RFP amendment provides that offerors, which do not have current and relevant past performance, can reference “contracts/efforts demonstrating the present and past performance for each of their key personnel.” RFP amend. 1 at 3. Finally, the proposals of offerors, who have no current (within the past 3 years) or relevant past performance, as defined by the RFP, cannot be rejected as unacceptable because they lack current or relevant past performance, but “may not be evaluated favorably or unfavorably on past performance” and will be provided an “unknown confidence rating.” RFP amend. 1 at 4; FAR sect. 15.305(a)(2)(iv); Kalman & Co., Inc., B‑287442.2, Mar. 21, 2002, 2002 CPD para. 63 at 8 (agency reasonably evaluated a proposal under a past performance evaluation factor as “neutral” where it found that the offeror lacked current relevant past performance). Because the performance evaluation criteria included in the RFP amendment are consistent with applicable regulations and do not preclude Futurecom from submitting a proposal, we find no basis to object to this amendment.  (Futurecom, Inc., B-400730.2, February 23, 2009)  (pdf)

First, Aegis complains that it was downgraded by the Corps under the experience factor simply because its references were for contracts in Iraq, rather than Afghanistan. Aegis argues that the experience that it identified in Iraq is very similar to the services required under this contract, and that the Corps has made an unreasonable distinction between experience in Iraq and experience in Afghanistan.

The Corps responds that its evaluators concluded that the differences in the security situation between Iraq and Afghanistan are sufficiently different that it was reasonable to distinguish between them in evaluating experience and past performance.[7] The Corps explains that Iraq is outside the Afghanistan Engineer District, the two countries have different geography, and the people have different cultural practices, and different causes of unrest. Therefore, the Corps explained that even though Aegis had high-rated past performance in performing similar services in Iraq, given the different conditions in Afghanistan, the Corps had a reasonable basis to give Aegis a rating of “good,” rather than a rating of “excellent,” under the experience factor because the experience in Iraq was less relevant.

As relevant here, the RFP reasonably informed offerors that the Corps would consider the geographic location of experience in determining its relevance, and it follows that contracts performed in Afghanistan would be more relevant than contracts performed elsewhere. In our view, the Corps could treat contracts performed in areas outside the Afghanistan Engineer District (including Iraq) as slightly less relevant based on the Corps’s understanding of the differences in the operational environment in Afghanistan. Given offerors with similar experience, we also think the Corps could reasonably value more highly experience in Afghanistan than experience in Iraq. In short, we see nothing unreasonable about the Corps’s decision to rate Aegis as “good” under the experience factor.  (Aegis Defence Services Limited, B-400093.4; B-400093.5, October 16, 2008) (pdf)

Relevance

ARTS also challenges the agency’s evaluation of SP Systems’ past performance, arguing that the agency failed to evaluate the degree to which the size of SP Systems’ contracts was similar to the PAAC III requirement as required by the RFP. The PAAC III procurement was valued at nearly $200 million and is expected to involve more than 270 personnel. In its evaluation of SP Systems’ past performance, NASA considered six past performance contract references for SP Systems and its major subcontractor. AR, Tab 37, SEB Report, at 125-28. Of these, five were valued between $2 million and $3.5 million, with between 5 and 12 employees, and the sixth had a dollar value of approximately $30 million, with 67 employees. AR, Tab 23, SP Systems’ Past Performance Information, at 02168. In its evaluation of ARTS’ past performance, NASA considered the contracts performed by ARTS team, which included, among others, a contract with value of $600 million, one with a value of $250 million, one valued at more than $100 million, as well as a contract with a value of $43.6 million. AR, Tab 37, SEB Report, at 137-43. Most of these contracts involved performance with more than 200 employees. AR, Tab 14, ARTS’ Past Performance Information, at 01146. Both offerors’ proposals were rated as “excellent” under the past performance factor, which required a determination by NASA that their past performance information was “highly relevant.”

As a general matter, the evaluation of an offeror’s past performance is a matter within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. However, we will question an agency’s evaluation conclusions where they are unreasonable or undocumented. Clean Harbors Envtl. Servs., Inc., B-296176.2, Dec. 9, 2005, 2005 CPD para. 222 at 3; OSI Collection Servs., Inc., B-286597, B-286597.2, Jan. 17, 2001, 2001 CPD para. 18 at 6. The critical question is whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation’s evaluation scheme. Clean Harbors Envtl. Servs., Inc., supra. The agency’s past performance evaluation here did not meet this standard.

The record reflects that, in evaluating offerors’ past performance, NASA utilized, in essence, a “pass/fail” criterion with respect to its consideration of the relative size of offerors’ past performance references. In responding to ARTS’ allegations, NASA essentially argues that in considering relevance, offerors’ prior contracts were deemed to be relevant by the terms of the RFP if they met the $2 million minimum threshold established by the RFP. Agency Supplemental Report at 31.

The fundamental premise of NASA’s argument, however, is flawed. By the terms of the RFP, the evaluation of relevance with respect to size was not merely a “pass” or “fail” determination (either over or under the $2 million threshold). Rather, the solicitation specified that NASA would consider the “degree” to which the size of an offeror’s past performance references (in addition to scope and complexity) are similar to the size of the PAAC III requirements. Thus, consistent with the solicitation language, NASA had to consider the relative size of offerors’ past performance references in weighing their past performance ratings and assessing whether the references were highly relevant, very relevant, relevant, somewhat relevant, or simply not relevant at all. In this context, the $2 million minimum set forth in the solicitation cannot be considered anything more than a floor, establishing the minimum dollar amount that NASA would consider for the purpose of evaluating relevance. Since there is nothing in the record to indicate that NASA engaged in the type of analysis required by the solicitation, we conclude that its determination that SP Systems’ past performance was “highly relevant,” particularly given that SP Systems’ references were, in most respects, small fractions of the size of the contemplated PAAC III contract, was unreasonable. Sytronics, Inc., B-297346, Dec. 29, 2005, 2006 CPD para. 15 at 6-7. (ASRC Research & Technology Solutions, LLC, B-400217; B-400217.2, August 21, 2008) (pdf)

Past Performance - contacting reference

AES asserts that the evaluation of its past performance was flawed because the agency did not make a sufficient attempt to contact one of its references--a firm performing a contract at the VA Medical Center (VAMC) in Washington, D.C.

The evaluation was reasonable. The RFP requested at least three past performance references for offerors’ most recent and relevant contracts for biomedical waste services. RFP para. 5.1.5. AES’s proposal listed four references, with telephone numbers and brief descriptions of its work, including contracts for medical waste packaging services, supply of medical waste containers, removal and destruction of regulated medical waste and recycling of other waste, and destruction of U.S. currency waste. Proposal para. 5. It also identified three contracts with other VAMCs, but provided no contact information. Id.

The agency states that it had difficulty contacting any of AES’s references, including the Washington, D.C. VAMC reference. The agency was able to correctly deduce the contractor’s name, but it found that the person listed as a reference “did not exist at the contact number provided.” AR at 5. While AES states that the number provided was, and remains, the reference’s cell phone number, an agency is only required to use reasonable efforts to contact an offeror’s past performance references; it is not required to make multiple attempts to contact such references. See OSI Collection Servs., Inc.; C.B. Accounts, Inc., B‑286597.3 et al., June 12, 2001, 2001 CPD para. 103 at 9. AES has not shown that the agency’s efforts to contact the Washington, D.C. reference were unreasonable. Rather, it appears that it was AES’s providing a single cell phone number, and no other contact information (e.g., a phone number for the headquarters of the contractor) that led to the agency’s unsuccessful efforts. We conclude that there was no impropriety on the part of the agency, and therefore find no basis for questioning the past performance evaluation.  (Advant-EDGE Solutions, Inc., B-400367.2, November 12, 2008) (pdf)

We agree with the protester that the contracting officer’s explanation of his evaluation, in response to the protest, does not entirely track the contemporaneous record. For example, the protester is correct that there is nothing in the contemporaneous record supporting the contracting officer’s claim that he did not consider Ahntech-San Diego’s PTR contract in evaluating Ahntech-Korea’s experience because Ahntech-San Diego is not a Korean company. In fact, as discussed above, the record shows that the contracting officer did consider Ahntech‑San Diego’s PTR contract; he concluded that the proposal information concerning this contract was too “general in nature” to allow the contracting officer to find that Ahntech‑Korea met the 3-year experience requirement. However, notwithstanding the agency’s unsupported statements in response to the protest, it is clear from the contemporaneous record that it reasonably found that Ahntech‑Korea did not meet the 3-year experience requirement.

As discussed, Ahntech‑Korea’s proposal identified two contracts to show that it met the 3-year similar experience requirement--the firm’s own KTRAC contract and Ahntech-San Diego’s PTR contract. Regarding the KTRAC contract, as noted, the agency determined that it was not sufficiently similar to the effort under the RFP because it did not involve managing multiple bases. Moreover, as the agency notes in its report, Ahntech‑Korea had been performing under the KTRAC contract for fewer than the 3 years required under the solicitation. AR, Tab 1, Legal Memorandum, at 13. The protester does not question the agency’s findings, or its resultant conclusion that the KTRAC contract did not demonstrate the required 3 years of experience, and we thus have no basis to question the evaluation in this regard.

(See Ahntech-Korea Company, Ltd., B-400145.2, August 18, 2008 sections for discussion of "parent or affiliated company")

Since the agency reasonably determined that the KTRAC contract did not satisfy the 3-year experience requirement, and since the agency could not properly consider Ahntech-San Diego’s experience in the evaluation, it is clear that the protester did not meet the 3-year experience requirement. It follows that the contracting officer’s conclusion that Ahntech‑Korea’s proposal was technically unacceptable was unobjectionable. The protester’s other arguments--regarding, for example, the absence of any RFP requirement that warranted rejecting the protester’s proposal on the basis that the protester is not a Korean company--are irrelevant, since they have no bearing on the propriety of the agency’s rejection of the proposal as unacceptable under the experience requirement.  (Ahntech-Korea Company, Ltd., B-400145.2, August 18, 2008)  (pdf)

Under the experience factor, CWS was rated “above average.” CWS maintains that it could not have reasonably been rated less than “outstanding” since the agency considered the same information it used to rate CWS as “outstanding” under the past performance factor, and because the agency found its experience to be a strength and did not identify any weaknesses under this factor. CWS’s argument, however, fails to recognize that the experience and past performance factors reflected separate and distinct concepts. Under the experience factor, the agency examined the degree to which a vendor had experience performing similar projects; under the past performance factor, the agency considered the quality of a vendor’s performance history. Given the fundamentally different nature of the evaluations, a rating in one factor would not automatically result in the same rating under the other. In addition, it does not follow that the finding of a strength and the lack of weaknesses automatically entitled CWS to a rating of “outstanding.” Rather, such a rating was reserved for submissions determined to “well exceed[]” the requirements and containing “numerous significant outstanding features,” while the above average rating assigned to CWS under the experience factor applied where the submission was considered “good with some superior features.” AR, Tab 6, Source Selection Determination, at 3. We see nothing in the record, beyond CWS’s opinion of its own quotation, to support a conclusion that the agency acted unreasonably in rating CWS’s experience “above average.”  (Commercial Window Shield, B-400154, July 2, 2008)  (pdf)

In its protest, DRS argued that the Navy had improperly disregarded adverse past performance information regarding GD. The protester maintained that the agency evaluation report failed to recognize and take into account the MCS CPAR, even though it was relevant to every past performance subfactor. DRS argued that the Navy’s failure to take this adverse past performance information regarding the awardee into account constituted a departure from the stated evaluation criteria that was prejudicial to DRS. Protest, Dec. 31, 2007, at 53-58.  In its report to our Office, the Navy originally argued that the SSEB had reasonably disregarded the MCS CPAR as part of its evaluation of GD’s past performance. The agency contended that only two specific divisions of General Dynamics--[deleted]--would be involved in performing the CEDS work here, while the MCS CPAR involved another GD division--[deleted]. Because the past performance information involved a General Dynamics division that would not be performing work on the CEDS project, the agency argued, it was reasonable not to consider this information as relevant in the evaluation of the awardee’s past performance. AR, Jan. 11, 2008, at 34. In its comments to the agency report, DRS provided information to demonstrate that GD [deleted] was in fact [deleted]. Specifically, GD [deleted] had been merged by the parent company into [deleted] “with the integrated unit continuing to operate as [deleted].” DRS Comments, Jan. 18, 2008, at 20. Quite simply, DRS argued, the specific General Dynamics division mentioned in the adverse MCS CPAR was one of the two General Dynamics divisions that the agency acknowledged would be performing the CEDS work here. Thus, the protester maintained, the Navy’s stated factual basis for not considering the MCS CPAR was completely inaccurate. Id. at 19-20.

At the hearing conducted by our Office, the SSEB chairman originally testified that the agency evaluators did not see and did not consider the MCS CPAR as part of their evaluation of GD’s past performance. Tr. at 203-05. The Navy, however, subsequently introduced evidence that the MCS CPAR had in fact been considered by the SSEB in its evaluation of GD’s past performance insofar as the evaluation report included specific findings that could only be attributable to the MCS CPAR.[31] Id. at 366-70. The SSEB chairman stated, however, that he still had no recollection of ever having considered the MCS CPAR as part of the agency’s evaluation of GD’s past performance. Id. at 361, 368-69, 376. For example, the following exchange took place with the SSEB chairman:

Q: [C]orrect me if I’m wrong. You stated you don’t remember considering the GD CPARs on MCS, correct?
A: I believe I stated I don’t recall seeing it.
Q: Do you recall evaluating it?
A: If I didn’t see it, how can I actually evaluate it?
Q: You mentioned that you had a conversation with the deputy on the SSEB, is that correct?
A: Yes.
* * * * *

Q: And your recollection of that discussion with the deputy was that he also did not remember this CPARs?
A: That’s what he told me.
Q: If you don’t remember seeing it and the deputy doesn’t remember seeing it, how do you know that you gave it proper consideration in the agency’s past performance evaluation of GD?
A: I don’t know.

Id. at 407-08.

At the hearing conducted by our Office, the SSEB chairman also discussed how the evaluators considered the relevance of offerors’ past performance information. At one point the lead evaluator indicated that the determination of whether an offeror’s past performance was similar to the work to be performed was based on whether it involved the delivery of equipment: “We would look at the CPARs. We looked at the work. If it was similar in terms of they were producing a piece of equipment, we would count that as being similar.” Id. at 214. At another point, the following exchange occurred with the SSEB chairman:

Q: Did you give some references or some CPARs more weight than others because they were -- they were the same or similar, they were more relevant to the work here?
A: I believe we evaluated and gave credit for every CPARs we received.
* * * * *

Q: I’ve looked at the SSEB report. . . . I did not see in here the agency’s --the agency saying that some of the references were more relevant than others. Am I missing anything?
A: No. We treat[ed] them all equally.
Q: Regardless of relevance? And what if it was really good past performance, but it has nothing to do with the technology of CEDS. How much weight do you give that? Do you think that that should be weighed equally to something that is highly relevant and high quality?
A: No.

Id. at 211-13.

The SSEB chairman also indicated that at least one of the strengths identified in the agency’s report regarding GD’s past performance was inaccurate. As set forth above, the SSEB report considered as a major strength the fact that a majority (i.e., three out of four) of the CPARs for proposed subcontractor [deleted] rated its performance as either exceptional or very good. The SSEB chairman acknowledged that this finding was inaccurate, and that instead two of the four CPARs for [deleted] had rated its performance as either outstanding or very good. Id. at 404.


We conclude that the agency’s evaluation of GD’s past performance was not reasonable or consistent with the stated evaluation criteria. Of foremost concern, the record indicates that the Navy failed to give meaningful consideration to all the relevant past performance information that it possessed regarding GD. The evaluation report reflects that the SSEB was aware of, and apparently considered to some degree, the CPAR regarding the MCS contract. The agency cannot provide an explanation, however, as to why the contractor’s self-serving rebuttal (which the Navy reviewing official for the MCS CPAR did not accept) merited two major strengths, while the extremely adverse information and ratings regarding the contractor’s performance in the areas of technical, schedule, cost control, and management were completely ignored. Tr. at 378. Additionally, the SSEB chairman admits having no recollection that he ever saw or considered the MCS CPAR and, as a result, we cannot say that the Navy gave proper consideration to this adverse past performance information in its evaluation. We fail to see how the agency can properly evaluate an offeror’s past performance when its evaluators admittedly do not remember if all the past performance information was in fact considered.  The record also reflects that the Navy failed to adequately consider the relevance of GD’s past performance information as part of the evaluation. An agency is required to consider the similarity or relevance of an offeror’s past performance information as part of its evaluation of past performance. See FAR sect. 15.305(a)(2) (the relevance of past performance information shall be considered); United Paradyne Corp., B-297758, Mar. 10, 2006, 2006 CPD para. 47 at 5-6; Clean Harbors Envtl. Servs., Inc., supra.

The RFP here instructed offerors to provide past performance information that was “relevant and pertinent,” and later defined “relevant” as similar to the CEDS procurement in terms of technology, type of effort, contract scope, schedule, and risk. RFP amend. 1, Instructions to Offerors, at 60, 62. The record does not reflect that the agency adequately considered whether GD’s past performance information was in fact similar to the CEDS procurement in accordance with the RFP. The CPARs and questionnaires upon which the SSEB based their evaluation of GD’s past performance furnished adjectival ratings and narratives regarding the quality of an offeror’s performance in various areas. The contemporaneous evaluation report does not indicate that the agency went beyond considerations of quality and also considered the relevance of the offerors’ past performance references. The SSEB’s evaluation findings regarding GD concern the quality of the offeror’s prior performance and indicate equal consideration of the offeror’s past performance references without regard to relevance. Further, at the hearing conducted by our Office, the SSEB chairman’s statements were, at best, ambiguous as to the agency’s consideration of relevance. Specifically, the lead evaluator indicated that the SSEB gave equal consideration to all the offeror’s past performance references, irrespective of relevance, and that the determination of what past performance was deemed “similar” was based simply on whether the prior work involved producing a piece of equipment. As the RFP required the agency to determine whether an offeror’s past performance was similar to the CEDS procurement in terms of technology, type of effort, contract scope, schedule, and risk, we conclude that the agency did not properly consider the relevance of GD’s past performance in its evaluation.

The record also reflects various inaccuracies in the SSEB report regarding GD’s past performance. As detailed above, the SSEB chairman admits that one of the strengths given to GD--that a majority of the CPARs for [deleted] rated it as exceptional or very good--was factually inaccurate. Moreover, the two strengths given to GD related to its MCS CPAR are redundant, as well as based entirely on assertions by the contractor with which the Navy reviewing official there did not agree. In addition, GD received a major strength for certain CEDS document deliverables that provided insight into the contractor’s management plans and processes--a fact that has nothing to do with past performance. In sum, several of the SSEB’s specific findings regarding GD’s past performance are without factual justification. (DRSC3 Systems, LLC, B-310825; B-310825.2, February 26, 2008) (pdf)

KIC challenges the agency’s determination that the awardee had relevant past performance. Specifically, KIC asserts that the awardee’s four past performance contracts should not have been found to be relevant, since the value of each was not sufficiently similar to the work being solicited here. In support of this argument, the protester asserts that the solicitation established a $1 million relevance threshold, which the awardee’s past performance contracts failed to meet.  In reviewing a protest challenging an agency’s past performance evaluation, we will examine the record to determine whether the agency’s judgment was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations. Ostrom Painting & Sandblasting, Inc., B‑285244, July 18, 2000, 2000 CPD para. 132 at 4.  We find nothing unreasonable in the agency’s evaluation here. First, contrary to the protester’s assertion, the RFP did not establish a $1 million value as necessary for prior contracts to be considered relevant under the past performance factor. Rather, as noted above, the solicitation stated with regard to past performance only that the determination of what was “relevant past performance” would be made by the source selection authority; it established no specific requirements for a contract to be found relevant. RFP at 36. As also noted above, the rating forms for contract references did include a $1-$5 million range; however, this was solely in reference to the “relevant experience” subfactor, not the past performance factor. Thus, under this scheme, while contract value would be considered under the past performance evaluation through the relevant experience subfactor, the agency nevertheless reasonably could evaluate an offeror’s past performance as relevant even in the absence of similarly valued prior contracts.  (KIC Development, LLC, B-309869, September 26, 2007) (pdf)

Where a solicitation contemplates the evaluation of vendors’ past performance, the agency has the discretion to determine the scope of the performance history to be considered, provided all quotations are evaluated on the same basis and the evaluation is consistent with the terms of the solicitation. See Weidlinger Assocs., Inc., B-299433, B-299433.2, May 7, 2007, 2007 CPD para. 91 at 8. In this regard, an agency is generally not precluded from considering any relevant past performance information, regardless of its source. See, e.g., NVT Techs., Inc., B-297524, B-297524.2, Feb. 2, 2006, 2006 CPD para. 36 at 5. Regarding the relative merits of vendors’ past performance information, this matter is generally within the broad discretion of the contracting agency, and our Office will not substitute our judgment for that of the agency. See, e.g., Clean Harbors Envtl. Servs., Inc., B-296176.2, Dec. 9, 2005, 2005 CPD para. 222 at 3. A protester’s mere disagreement with the agency’s judgment does not establish that an evaluation was improper. Id. As a preliminary matter, the record here shows that DHS obtained the past performance reference for Paragon’s FPS contract for Alabama not from the former contracting officer, but from the COTR--see AR, Tab 9, Paragon’s Past Performance References, at 8-9--an individual who Paragon itself describes as being “in the best position to provide current and relevant information about Paragon’s performance.” Protest, June 4, 2007, at 7. With regard to Paragon’s FPS contract for Kentucky, while the record indicates that the agency did obtain the past performance reference from the former contracting officer, id. at 13-16, we find Paragon’s challenge on this ground to be without merit. As discussed above, an agency is generally not precluded from considering any relevant information, and is not limited to considering only the information provided within the “four corners” of vendor’s quotation when evaluating past performance. See FAR sect. 15.305(a)(2)(ii); Weidlinger Assocs., Inc., supra; Forest Regeneration Servs. LLC, B-290998, Oct. 30, 2002, 2002 CPD para. 187 at 6. Likewise, there exists no requirement mandating that an agency contact the specific individual designated by the vendor as the reference when seeking past performance information. Rather, the relevant inquiry as to who may furnish a past performance reference is whether the individual has a sufficient basis of knowledge to render an informed opinion regarding the vendor’s prior work efforts. Paragon does not argue that the former contracting officer for its FPS contracts did not have a sufficient basis of knowledge to render an informed opinion regarding Paragon’s performance. In fact, the protester admits that the former contracting officer was involved with Paragon’s FPS contracts for the past 3 years, while the current contracting officer had only been in that position for less than 1 month at the time the agency sought the past performance references here. Protest, June 4, 2007, at 7. Based on our review of the record we find nothing unreasonable in the agency’s evaluation of Paragon’s quotation with regard to past performance. As noted above, DHS obtained and considered the input furnished by individuals familiar with Paragon’s performance for each of the contract references that Paragon included in its quotation. The agency reasonably determined that Paragon’s references were all recent and relevant, and based on the information received, warranted an overall past performance rating of satisfactory. To the extent that Paragon argues that the reference comments were inaccurate and unfounded, see Comments, July 12, 2007, at 13, we conclude that this represents mere disagreement with the agency’s judgment. (Paragon Systems, Inc., B-299548.2, September 10, 2007) (pdf)

We find the agency’s assignment of a moderate risk rating to be problematic. As noted, the agency determined that only one of TVI’s key employees’ work was both relevant and recent within the meaning of the RFP’s definition. To the extent that the agency found any of TVI’s past performance information recent and relevant, it was limited to this one individual’s performance in connection with two contracts, only one of which was specifically identified as meeting the RFP’s $1 million threshold for relevance. In particular, the record shows that he worked to design a production line for [deleted], and also advised in the design and manufacture of C2A1 canisters by [deleted]. [deleted] manufactured C2A1 canisters under two prior contracts (only one of which is noted as meeting the $1 million relevance threshold), which were completed without any performance problems; at least one of the contracts was described as resulting in deliveries ahead of schedule, with excellent quality. AR, exh. 8, at 21. There does not appear to be anything negative in the information reviewed by the agency with respect to this individual’s work. The RFP specifically provided for the assignment of an unknown risk rating where the offeror was found to have “little or no” recent or relevant past performance upon which to base a meaningful performance prediction. While “no” past performance information is easily understood as a complete absence of past performance information, the question of what constitutes “little” past performance information is at issue here. As noted, the agency considered the past performance of only one of TVI’s key employees (out of 11 individuals whose resumes were included in the firm’s proposal), while rejecting the remaining information relating to its other key employees, all of its prior subcontractors, and its prime contracts as either not relevant or not recent. To the extent that this individual’s past performance information was reviewed, there is nothing in the record to show why the agency considered the information as predictive of a moderate risk of unsuccessful performance of the requirement by TVI; the agency simply did not articulate a nexus between the information reviewed with regard to this individual’s experience and its evaluation conclusion. It certainly is not clear how the positive past performance information found in connection with this individual could reasonably translate into the negative past performance rating assigned; while a limited quantity of positive information might not be sufficient to warrant assigning an offeror a positive, rather than neutral, past performance rating, absent some compelling justification, positive information should not result in a negative rating. Under the circumstances, we conclude that the agency should have assigned an unknown risk rating to TVI, since there was little information to consider, and the information considered apparently did not provide the basis upon which the agency made its performance prediction. We therefore find the agency’s assignment of a moderate risk rating unreasonable given the terms of the RFP.  (TVI Corporation, B-297849, April 19, 2006) (pdf)

Propper challenges the agency’s evaluation of its past performance as unreasonable; the firm contends that in light of its previous experience manufacturing and participating in the development of APECS items, its receipt of a large business DLA Vendor of the Year award and an agency certificate of appreciation in 2004, and its explanations during discussions regarding its late deliveries, the agency should have given the firm’s past performance a higher rating. While the protester does not refute the agency’s assertion that its proposal failed to provide detailed information as required by the RFP for evaluation of its past performance, it generally contends that the agency was required to conduct a more comprehensive investigation to obtain additional past performance references for the firm. Here, offerors were instructed to provide details of their past performance for evaluation. Our review of the record confirms the agency’s view that Propper failed to provide sufficient detail in its proposal to demonstrate the favorable performance it now claims for its APECS items and other relevant work; the firm did not, for instance, list contacts for all of its APECS work, and it did not elaborate in its proposal on the delivery and quality of the items it provided under prior contracts. Using the limited past performance information the firm did provide, the agency contacted at least one reference outside of the agency familiar with Propper’s delivery of APECS garments, and one reference within the agency who was also familiar with some of Propper’s prior APECS work. While the first contact reported that the customer was satisfied with the firm’s performance, the agency’s own experience with Propper’s past APECS contract was that the items were delivered significantly late due to the firm’s inability to meet an accelerated delivery schedule it had agreed to. To the extent the protester contends that the agency was required to conduct additional research to locate more knowledgeable, and possibly more favorable, references for the firm, Propper is incorrect. There is no legal requirement that an agency attempt to contact all past performance references that may be listed in a proposal or may be available for each contract performed by a contractor. See, e.g., Dragon Servs., Inc., B-255354, Feb. 25, 1994, 94-1 CPD para. 151 at 8. Here, the record shows that the agency reasonably considered past performance information it obtained from sources identified to it as knowledgeable about the firm’s prior contract work, as well as information close at hand regarding its own experience with the firm. Given these circumstances, including Propper’s failure to persuasively demonstrate its ability to meet all contract performance requirements by submitting the required detailed past performance information, we cannot find that the agency was obliged to investigate the firm’s performance beyond the information it considered. (Propper International, Inc., B-297950.3; B-297950.4; B-297950.5, March 19, 2007) (pdf)

UPC identified four contracts in the section of its proposal where it was to identify relevant contracts (the predecessor contract to the one at issue in the protest at Wright-Patterson and contracts performed at Edwards AFB, Vandenberg AFB, and the Kennedy Space Center); in addition, the protester identified one terminated contract (performed at Roosevelt Roads Naval Air Station in Puerto Rico). The agency assigned the Wright-Patterson contract a point score of 60 out of 60 possible points for relevance; the Edwards AFB contract a relevance score of 56; the Vandenberg AFB contract a score of 22; and the Kennedy Space Center contract a score of 0. It did not assign the Roosevelt Roads contract a relevance score. The agency then averaged the four scores for a relevance score of 34.5. The agency’s relevance matrix defined a score of 0-15 points as not relevant; a score of 16-30 as somewhat relevant; a score of 31-45 as relevant; and a score of 46-60 as highly relevant. Because the averaged score (34.5) fell within the point range for relevant performance, the Air Force assigned UPC’s proposal an overall contract relevance rating of relevant. The agency then integrated this rating with a past performance rating for UPC of exceptional/high confidence that it had arrived at by averaging the point scores on the past performance questionnaires furnished by UPC’s references. The result was an overall performance confidence rating of very good/significant confidence. Similarly, another offeror that had performed three contracts rated by the agency as highly relevant, three contracts rated by the agency as relevant, and five contracts rated by the agency as somewhat relevant was assigned an overall contract relevance rating of relevant because this was its average rating; as a result of this relevance rating, this offeror, whose past performance was, like UPC’s, rated outstanding/high confidence, was assigned an overall performance confidence rating of only very good/significant confidence. We think that the agency’s approach to evaluating past performance was unreasonable because it had the effect of penalizing offerors with relevant experience such as UPC and the other offeror noted above for their non-relevant experience. For example, using the agency’s methodology, an offeror that had performed four highly relevant contracts well would have received a higher performance confidence rating than an offeror that had performed four highly relevant and four somewhat or not relevant contracts equally well. Such a result is, in our view, clearly irrational. The agency’s methodology is further unreasonable in that it gave equal weight in the calculation of offerors’ past performance ratings to highly relevant and non-relevant performance. In UPC’s case, for example, the protester’s performance on the predecessor contract to the effort solicited here, for which it received a relevance score of 60 of 60, was given the same weight in the computation of its past performance score as its performance on the Kennedy Space Center contract (for which the protester received a relevance score of 0) and its performance on the Roosevelt Roads contract (for which, as discussed in greater detail below, the protester was given no relevance score). Agency Report, Tab 3b-4. Moreover, it was contrary to the terms of the RFP, which provided that the past performance evaluation would be accomplished by reviewing offerors’ “relevant present and recent past performance” (emphasis added), RFP at 46, for the agency to have considered non-relevant experience in its evaluation. In addition, the agency’s failure to assess the relevance of individual contracts in determining the weight to assign offerors’ performance of them was contrary to the direction in Federal Acquisition Regulation sect. 15.305(a)(2)(i) that “the currency and relevance” of the information should be considered in the evaluation of past performance. (United Paradyne Corporation, B-297758, March 10, 2006)  (pdf)

Computer Cite challenges the evaluation of the DSIS proposal under the technical capability factor, asserting that the prior contracts reviewed by the agency for evaluation of DSIS’s experience were not telecommunications services contracts similar in volume to the workload at Hickam AFB. Computer Cite, therefore, concludes that the technical evaluation was flawed and unsupported, and that the resulting award was improper. Protester’s Comments at 3-4; Protester’s Supplemental Comments at 2-6. The RFP did not set out clear criteria for determining technical acceptability with respect to assessing offerors’ experience. Instead, as quoted above, it indicated that the proposal would be considered to have met the standard if it included a list of “similar projects in telecommunications support services similar in volume as Hickam AFB’s workload.” While the protester contends that the agency had no reasonable basis for finding DSIS’s experience similar to the requirements of the current procurement, the fact is that the RFP did not establish objective criteria for defining “similar,” and the record demonstrates that the agency interpreted the word generously for the protester as well as for the awardee. Specifically, the record shows that the agency found Computer Cite’s experience acceptable, even though the [DELETED] projects the protester claimed were similar were [DELETED]. In other words, it appears from the record that to the extent the agency did not interpret “similar” in the strict way that the protester now advocates, nothing in the RFP required it to, and both DSIS and Computer Cite appear to have benefited from the agency’s interpretation. (Computer Cite, B-297291; B-297291.2, December 23, 2005) (pdf)

The protester’s list of 46 separate projects completed or underway for the U.S. government, provided both in its proposal and to our Office, displays a wide range of construction experience. Nonetheless, the evaluation of proposals in a given procurement must follow the stated evaluation scheme set forth in the solicitation. Tennier Indus., Inc., B‑286706.2, B-286706.3, Mar. 14, 2001, 2002 CPD para. 75 at 3. Here, the solicitation limited the review of an offeror’s experience to the five previously-performed contracts identified in the offeror’s proposal. These contracts formed the pool of information to be considered by the evaluators for determining whether the contractor has performed work that is “comparable to the types of work covered by this requirement.” RFP at 22. As a result, there was no basis for the agency’s evaluators to look to the protester’s performance of other U.S. government contracts to assess the proposal under the experience factor, and this evaluation is not, in any way, flawed because the evaluators limited their review in precisely the way the solicitation advised. See Tennier Indus., Inc., supra. (Hera Constructive S.A./Synthesis S.A., Joint Venture, B-297367, December 20, 2005) (pdf)

Here, as explained above, the record shows that the source selection decision was based upon a detailed evaluation of Airtronic’s past performance record. In the source selection document, the contracting officer clearly acknowledged that Airtronic had no relevant contracts as defined under the RFP and that, in accordance with the solicitation, Airtronic could have received an unknown risk rating. AR, Tab 4, Source Selection Decision, at 4, 9. Nonetheless, the contracting officer, consistent with the past performance evaluation factor, which permitted consideration of the offeror’s overall general past performance history, concluded that Airtronic’s past performance history of manufacturing military components indicated an acceptable level of performance risk that justified award to Airtronic at its lower price. Id. We see nothing improper in this conclusion.  (PHT Corporation, B-297313, December 8, 2005) (pdf)

Significantly, there is no indication--in the RSSD or elsewhere in the record--that the agency went beyond the questionnaires and considered the relevance of the offerors’ past performance references. This is problematic because, as noted, the RFP provided for consideration of the relevance of the past performance information received, RFP at 137, and the two references received for Clean Venture--from the Smithsonian Institution and the Washington Metropolitan Area Transit Authority--involved substantially smaller, less complex contracts than the current requirement. Clean Venture Proposal at 106-09; AR, exh. 7, at 2-14. There thus is reason to question the relevance of Clean Ventures’ past performance. At the same time, the protester, as the incumbent contractor for the NIH requirement here, possessed arguably the most relevant past performance information available. However, there is no indication that the agency ever considered the relevance of that contract, either; instead, the record shows only that the agency considered the questionnaires for two different contracts performed by Clean Harbors, one for a private drug company and the other for the Army. AR, exh. 7, at 2. Indeed, while the agency states in its submissions to our Office that it considered past performance information in the proposals, in addition to the questionnaires, there is no support in the contemporaneous record to show that it considered Clean Harbors’ performance as the incumbent at all, or for that matter, that it considered any other information included in the proposals or otherwise available to the agency. Rather, as discussed above, the record contains documentation showing only that the agency considered the scores derived from the questionnaire responses received for the two firms, and, thus, the agency’s arguments during the protest are simply not supported by the record. We conclude that the agency’s actions were inconsistent with the RFP, and otherwise unreasonable. We find as well that the agency’s failure to consider the comparative relevance of the offerors’ past performance could have affected its source selection decision; although both firms received the same past performance ratings, it appears, as noted, that the references relied on for Clean Venture’s rating were smaller, less complex contracts as compared to Clean Harbors’ incumbent contract for the very requirement being solicited. (In addition, to the extent that the other contracts referred to in the offers could have been evaluated and deemed more or less relevant, the record shows that the agency apparently did not consider that information in arriving at its ratings.) In view of the foregoing, we find that Clean Harbors was prejudiced by the agency’s failure to evaluate the comparative relevance of the offerors’ past performance, and sustain the protest on this basis.  (Clean Harbors Environmental Services, Inc., B-296176.2, December 9, 2005) (pdf)

UFC complains that the agency employed an "overly mechanical" evaluation of vendors' past performance. Protest at 7; Comments at 2. Specifically, UFC complains, citing our decision in American Dev. Corp. , B-251876.4, July 12, 1993, 93-2 CPD 49 at 10-11, that separately evaluating relevance and quality of past performance improperly favored incumbent contractors. In American Dev. Corp., we found that an agency's methodology for assessing offerors' past performance was unreasonable, where the methodology "rewarded offerors which had held at least one contract relevant to the work to be performed under the RFP without consideration of the quality of the work performed under that contract." American Dev. Corp. , supra , at 10. In that case, we found that although the agency reasonably assessed the relevance of offerors' past performance (even where the solicitation did not specifically identify the relevance of past performance as a evaluation factor), the agency could not make award to the incumbent contractor based upon its more directly relevant past performance without considering the quality of the incumbent's performance under that contract. We do not agree with the protester that Education's evaluation failed to properly evaluate the quality and relevance of the vendors' work. Although it is true that the SSEB separately assigned points and adjectival scores for relevance and quality, here, unlike in American Dev. Corp. , the vendors' past work that was evaluated for relevance was also evaluated for quality. In this regard, the SSA's source selection decision documents that the SSA considered both the relevance and quality of each vendors' past performance. See AR, Tab 69, Source Selection Decision. Thus, for example with respect to the proposal of one of the incumbent contractors, the SSA noted that although this vendor's past performance was "highly relevant," this vendor had not performed well in the last year of that contract; the SSA did not select this vendor's proposal to receive a task order. See Id. at 9. (emphasis added)  (Universal Fidelity Corporation, B-294797.2, February 7, 2005) (pdf)

BTC argues that the agency unreasonably downgraded BTC on the basis of a less favorable database rating for BTC's performance at Fort Jackson, rather than consider a more favorable (and more recent) survey response for that contract. Protester's Comments at 15. BTC then argues that when the agency evaluated Joppa, the agency discounted the less favorable database information in two instances--specifically that the agency used a survey response regarding a contract at Pope AFB, rather than less favorable information in the database and that the agency relied on the contracting officer's favorable personal assessment of Joppa regarding performance at Charleston AFB, rather than less favorable information contained in the database. Id. at 16. With respect to BTC, the contemporaneous evaluation record shows that the agency based its past performance evaluation of BTC on the more favorable survey response; while recognizing the lower database rating, the agency did not actually use that less favorable information to downgrade BTC's overall past performance. AR, Tab 8, Revised Past Performance Evaluation, at 3. With respect to Joppa, the contemporaneous evaluation record shows that the database information reflected a review of Joppa's performance at Charleston AFB from October 2002 through September 2003 and did not include any past performance information for the intervening period of nearly 2 years. AR, Tab 16, Contractor Past Performance Assessment Report for Charleston AFB, at 1. The contracting officer relied more on her own current evaluation of Joppa's performance at Charleston AFB than the less favorable database information. At Pope AFB, the database information again related to a period more than 2 years old. AR, Tab 16, Contractor Past Performance Assessment Report for Pope AFB, at 1. The contracting officer also relied more on a current reference obtained from Pope AFB. The record thus shows that the contracting officer treated both offerors fairly and equally; that is, she relied on the more favorable assessment in each case (in favor of both offerors). (BTC Contract Services, Inc., B-295877, May 11, 2005) (pdf)

While the language in section L of this solicitation may have caused the protester to anticipate that the agency would distinguish between degrees of relevance in evaluating past performance, we agree with the agency that there is nothing in the RFP that requires it to do so. Simply put, information provided in section L of an RFP is not the same as evaluation criteria in section M; rather than establishing minimum evaluation standards, section L generally provides guidance to assist offerors in preparing and organizing their proposals. All Phase Envt'l, Inc. , B-292919.2 et al. , Feb. 4, 2004, 2004 CPD 62 at 4. In addition, information required by section L does not have to correspond to the evaluation criteria in section M. Cascade Gen'l, Inc. , B-283872, Jan. 18, 2000, 2000 CPD 14 at 10. Thus, we see nothing in the requirement that offerors provide information about contracts "that are similar in nature to the solicitation work scope," RFP at 62, or in the other section L provisions quoted above, that dictates that the agency must weight differently--within its assessment of each offeror's collective experience--the ratings given each company. (University Research Company, LLC, B-294358.6; B-294358.7, April 20, 2005) (pdf)

The essence of Ben-Mar's protest is that a "company [ i.e. , TSI] with no experience in meeting the Coast Guard's strict requirements for fitting and alterations of working and dress uniforms for recruits was selected over an incumbent with a flawless service record over a twenty-year period." Protester's Comments at 1. Ben-Mar continues that the agency unreasonably failed to distinguish between the past performance of it and TSI, based on the "relevance and quality of all available past performance information." Id. at 15. In reviewing a protest against an agency's proposal evaluation, we will consider whether the evaluation was reasonable and consistent with the terms of the solicitation and applicable statutes and regulations. Kira, Inc.; All Star Maint., Inc. , B-291507, B-291507.2, Jan. 7, 2003, 2003 CPD 22 at 5. Mere disagreement with an agency's evaluation is not sufficient to render the evaluation unreasonable. Bevilacqua Research Corp. , B-293051, Jan. 12, 2004, 2004 CPD 15 at 8 n.8. Contrary to Ben-Mar's suggestion, the RFP did not restrict this competition to firms with experience in altering and tailoring Coast Guard uniforms. Rather, the RFP required an offeror to provide "relevant" past performance information for evaluation in the areas of product quality, timelines, cost control, and customer satisfaction. In requiring an offeror to provide "relevant" past performance information, the RFP did not define "relevant" in terms of an offeror having past performance that was identical to the requirements described in the RFP. In other words, the RFP did not require an offeror to have past performance in providing fitting, alteration/tailoring, and garment pressing services for the Coast Guard in order to be eligible to compete in this procurement. Here, as detailed above, the record shows that the agency recognized Ben-Mar's successful performance over the past 20 years as the incumbent contractor. The record also shows that the agency considered TSI's past performance at Lackland AFB, in terms of fitting and altering garments for Air Force trainees, to be "relevant" to the Coast Guard requirements as described in the RFP. To the extent the agency had concerns, based on TSI's initial technical proposal, with the work the firm did at Lackland AFB, TSI responded to these concerns, as raised by the agency during discussions, in its final revised proposal. Other than disagreeing with the agency's assessment that TSI's Lackland AFB past performance was relevant to the CoastGuard requirements, Ben-Mar has provided no meaningful basis for our Office to question the reasonableness of the agency's assessment that TSI had a record of past performance that was "relevant" to the Coast Guard requirements. (Ben-Mar Enterprises, Inc., B-295781, April 7, 2005) (pdf)

Regarding the agency's first argument, the RFP did provide that for experience to be considered relevant, a project needed to be "similar in magnitude (euro amount)," which, we think, can only reasonably be interpreted as meaning that the project needed to be similar in magnitude to the project(s) here. It is not clear, however, whether the solicitation here comprised a single project (the overall work effort), with a value of 20 to 30 million euros, or two projects (construction of a personnel alert holding area and construction of a heavy drop rigging facility), with a combined value of 20 to 30 million euros. Either interpretation is reasonable in our view, given that the RFP itself refers to the work both as a project (in the specification table of contents) and as projects (on the RFP cover page, SF 1442). Accordingly, we do not think that the RFP can be said to have clearly placed offerors on notice that only projects with values of 20 to 30 million euros would be considered relevant; rather, we think that it may reasonably be interpreted as providing for consideration of projects similar in value to one of the phases as relevant. Regarding the agency's argument that it was reasonable for the evaluators to distinguish between experience in performing multiple projects under a single contract with an overall value of 20 to 30 million euros and multiple projects under multiple contracts with combined values of 20 to 30 million euros because supervising and administering a 20 to 30 million euro contract is a much bigger job than supervising and administering a 10 million euro contract, the issue is not whether administering and supervising a larger contract is more difficult than administering and supervising a smaller one; the issue is whether administering and supervising a larger contract is more difficult than administering two smaller ones with an equivalent overall value concurrently. The agency has offered no persuasive argument as to why such is the case, whereas the protester has offered two reasonable arguments as to why concurrent administration of multiple contracts is in fact more difficult: (1) under multiple contracts, the contractor is required to deal with multiple government contract managers, each of whom may interpret and apply government procedures differently, while under a single contract, the contractor deals with only one government contract manager, and (2) increasing the number of contracts increases the number of submittals since use of the same material at multiple sites under multiple contracts requires a separate submittal for approval of the material under each, whereas use of the same material at multiple sites under a single contract does not. To the extent that the agency argues that it can assume that a company with experience with a 20 million euro contract will staff management positions under this contract with qualified individuals, but that it cannot make the same assumption for companies that have performed combined efforts of 20 million euros, see id. , we do not think that the agency can reasonably make assumptions about personnel qualifications without instructing offerors to submit information pertaining thereto and evaluating such information. In our view, the agency's evaluation of CMR's projects under the "similarity in magnitude" (or project value) criterion was unreasonable because it failed to take into consideration CMR's experience in concurrently performing smaller projects with combined values in the range of the estimated value of the contract here. We think that it was unreasonable for the evaluators not to consider concurrent performance under multiple contracts at multiple sites as relevant experience with regard to the effort to be performed here, given that concurrent performance at multiple sites is precisely what the RFP here requires. We also think that the agency's determination that none of CMR's projects demonstrated sufficient similarity with regard to project complexity to be rated as relevant under that criterion was unreasonable. In our view, it was not reasonable for the agency to downgrade the relevance ratings of CMR's projects on the basis that each project, on an individual basis, failed to involve multiple sites, given that the projects, as a group, demonstrated abundant experience with multiple sites. It simply makes little sense that if an offeror presented two projects, each involving security issues and multiple sites, both would be determined relevant with regard to project complexity, whereas if an offeror presented two projects, each involving security issues, that were performed at different sites at the same time, neither would be determined relevant with regard to project complexity.

In our view, the record fails to demonstrate that the agency had a reasonable basis for its determination that CMR could only be considered minimally qualified with regard to experience and that awarding to the firm would constitute some risk to the government, and thus that it should be rated as merely satisfactory with regard to organizational experience. (Cooperativa Muratori Riuniti, B-294980; B-294980.2, January 21, 2005) (pdf)

With respect to the plaintiff’s allegation that it was improper for the agency to request past performance evaluations from only those of Arora’s listed references involving medical specialties which the agency deemed relevant to the acquisition, plaintiff has not demonstrated that the DHHS abused its discretion by electing not to contact three of Arora’s six proffered references.9 The court notes that the contracting officer also selectively contacted the references submitted by CasePro, choosing to contact one of the four references provided for CasePro and both of the two references provided for CasePro’s subcontractor, PPDG. The solicitation informed offerors that “[t]he Government will focus on information that demonstrates quality of performance relative to the acquisition under consideration. . . . [and] is not required to contact all references provided by the Offeror.” Also, the forms to be used by offerors to identify reference contacts as part of their initial proposal submission requested that offerors “[e]xplain why you consider the services similar to the services required by this solicitation,” giving offerors notice that the similarity of services would play a role in the contracting officer’s review. (emphasis added). Finally, the evaluation scheme in the solicitation provided, “[w]hen assessing performance risks, the Government will focus on the past performance of the Offeror as it relates to all acquisition requirements . . . .” (emphasis added). Thus, the contracting officer’s selection of references based on an assessment that the references were, or were not, related to “all acquisition requirements” was well within the announced evaluation criteria included in the solicitation.  (The Arora Group, Inc., v. U. S. and CasePro, Inc., No. 04-366C, October 12, 2004) (pdf)

The evaluation here was unobjectionable. As noted above, the Westover contract was referenced in the Overall Risk Assessment Spreadsheet, a summary of the contracts reviewed for each competitive range offeror that generally includes the rating, relevance, and value of each contract. While the Spreadsheet indicates that the agency rated the Westover contract as highly relevant, it included no assigned evaluation rating for the contract. The agency explains that this is because, notwithstanding that the contract administrator and flight chief for that contract were very happy with SWRs performance to date, AR, Tab 12, Integrated Assessment Best Value Decision, at 11, the contract administrator believed there was too little data for a meaningful evaluation. AR, Tab 2, Contracting Officers Statement, at 2. In this regard, the Air Force reports that the Westover contract calls for 300 aircraft washings per year and that, at the time the agency evaluated SWRs past performance, SWR had performed only 9washings. As a result, the contracting officer explains, the Westover contract was not assigned a performance rating and was not weighted as significantly as other SWR contracts. AR, Tab 2, Contracting Officers March, 2004 Statement, at 3. We find nothing unreasonable in the agencys judgment. We think the agency reasonably could determine that, given SWRs brief performance on the Westover contract at the time of the pp/pr evaluation, the fact that the firm so far was performing well was not sufficient to offset the concerns raised by SWRs performance of the Cherry Point contract. We conclude that both the agencys evaluation of the Westover contract and its overall rating for the protester were reasonable. (SWR, Inc., B-292896.3, June 7, 2004) (pdf)

Where a solicitation advises offerors that experience is to be evaluated, an agency may properly consider an offeror’s specific experience in the area that is the subject of the procurement. In this regard, experience as an incumbent may offer genuine benefits to an agency and may reasonably distinguish the incumbent’s proposal. IBP, Inc., B-289296, Feb. 7, 2002, 2002 CPD ¶ 39 at 5. As mentioned previously, the solicitation specifically listed corporate experience as an evaluation criterion under the technical evaluation factor. In this regard, the RFP stated that “[t]he offeror shall demonstrate experience in performing similar type work, size, volume and complexity for the last five years with a value of $8,000,000 or more per project annually.” The RFP added here that, among other things, it was “the Offeror[’]s responsibility to clearly explain and demonstrate to the Government how their work experience in each referenced contract is relevant to the contract requirements in this solicitation.” RFP amend. 1, at L-9. In considering the impact of incumbency, the SSB noted that Jones’s status as “the incumbent contractor currently performing most of the services for the same customers in the same remote location,” and the firm’s intent to “roll their existing management team over from the current contract to the new one . . . provides the government a high degree of confidence and low risk in the successful performance . . . on a follow on contract.” AR, Tab 17, SSB Report, Sept. 24, 2002, at 9-10. Although, as noted by the protester, both Jones’s and B&R’s proposals received ratings of “good” under the corporate experience criterion, the agency could consider Jones’s incumbency as a discriminator in the source selection decision because this criterion was part of the evaluation scheme.6 IBP, Inc., supra, at 7. (Burns and Roe Services Corporation, B-291530, January 23, 2003) (pdf)

In its review of Griffon’s prior contracts, the PRAG reasonably found that Griffon’s sub-scale spacecraft contract involved work similar to the design, development, and testing efforts required by the solicitation here; however, the agency found no similarities to many other areas specified in the RFP, including RPVT production and operational services. With regard to Griffon’s cryotank contract, the PRAG found no similarities between it and the RFP requirements here, yet nonetheless deemed this past performance relevant and supportive of its performance risk assessment in that Griffon “met technical, cost and schedule requirements,” and “consistently found way[s] to keep complex integration jobs on schedule, resolved unanticipated problems and developed recovery plans for items that fell behind.” AR, Tab R, PRAG Report at 11. We find the agency’s analysis unconvincing, inasmuch as almost any contract effort would be relevant by this standard. Lastly, the similarities found by the PRAG between Griffon’s MRI composite table contract and the RFP requirements here were limited to “the intricate RPVT airframe specifications” and “innovative testing and composite production techniques.” We note that Griffon’s cryotank and MRI table contracts, like its sub-scale spacecraft contract, involved the design and development of single items and related engineering services. By contrast, the efforts required by the RFP here were not limited to design and engineering services, but also included the production of an estimated 2,000 RPVTs and extensive operational services.[9] We find, therefore, that the record lacks any basis upon which the agency could reasonably have concluded that Griffon’s prior contracts either individually or collectively demonstrated past performance similar in scope to the efforts required by the RFP. (Continental RPVs, B-292768.2; B-292768.3, December 11, 2003) (pdf)

As an initial matter, there is nothing in the record to suggest that the agency engaged in any contemporaneous analysis concerning the relative value of the RFP’s indefinite-quantity requirements and the value of the Trammel Crow contract. More importantly, however, the RFP was not limited to the indefinite-quantity portion of the RFP. Therefore, in order for a reference to be relevant for purposes of determining a contractor’s experience in completing contracts of similar “size, scope, and complexity,” offerors had to submit references that were similar to the RFP’s requirements as a whole. Given the relatively low dollar value of the Trammel Crow contract when compared with the total value of the RFP’s requirements, as well as the agency’s admission that the Trammel Crow contract was only relevant to a limited portion of the RFP’s requirements, it was unreasonable for the agency to regard the Trammel Crow contract as similar “in size, scope, and complexity” to the RFP requirements such that it properly could be considered in evaluating IRRI’s experience. As a result, based on the current record, the agency’s conclusion that IRRI had [deleted] experience with relevant contracts lacks a reasonable basis, given that it is based in part on consideration of the Trammel Crow contract.  (Si-Nor, Inc., B-292748.2; B-292748.3; B-292748.4, January 7, 2004) (pdf)

Cortez raises several arguments, questioning, for example, NASA’s relying on the mostly good ratings for the firm’s incumbent contract at MSFC, rather than on the reports for other contracts, including one at GSFC, which included excellent and excellent plus ratings. However, there is nothing unreasonable in an agency’s placing particular emphasis upon a firm’s performance as the incumbent contractor; such performance reasonably may be viewed as a more accurate indication of likely future performance than performance on other contracts. See D.M. Potts Corp., B‑247403, B-247403.2, Aug. 3, 1992, 92-2 CPD ¶ 65 at 4; Inlingua Schools of Languages, B-229784, Apr. 5, 1988, 88-1 CPD ¶ 340 at 5 (prior performance on incumbent contract for the same services was the most relevant). In any case, EG&G also received high performance ratings on a number of contracts. For example, EG&G’s performance was rated as excellent plus/excellent for the center operations support contract at MSFC, excellent plus/excellent for a support services contract at the Department of Energy’s National Energy Technology Laboratory, exceptional for a contract for operation and maintenance of the Air Force Radar Cross Section Test Facility, and either excellent or excellent/good for a very large classified logistics services contract. EG&G Past Performance Documentation. Unlike Cortez, however, EG&G had not been found deficient in performance of the incumbent contract. (Cortez, Inc., B-292178; B-292178.2; B-292178.3, July 17, 2003) (pdf)

While KTI had better performance ratings in the majority of its task orders, and their combined value outweighed that of the remaining task orders, its poor and unsatisfactory ratings were not insignificant, and for the 11 task orders only four respondents gave an unqualified “yes” in response to the question whether they would contract with KTI again. Three additional respondents indicated that they would contract with KTI again, but stated that this was dependent upon KTI's hiring of a good subcontractor or on the type of work required. AR, Tab 7, at 668. The respondents for the other four orders answered “no.” In our view, KTI's mixed past performance on these directly relevant task orders reasonably supports the agency's conclusion that its past performance was not better than acceptable. KTI's position to the contrary constitutes mere disagreement with the agency's judgments, which does not establish that the evaluation was unreasonable. UNICCO Gov't Servs., Inc., B‑277658, Nov. 7, 1997, 97-2 CPD ¶ 134 at 7.  (Kathpal Technologies, Inc., B-291637.2, April 10, 2003  )  (pdf)

The record shows that the VA did attempt to telephone the three non-VA references cited in Prime's proposal, but two of the numbers were no longer in service and the third reference failed to return the evaluator's telephone call. There is no legal requirement that all past performance references be included in a valid review of past performance. Advanced Data Concepts, Inc., B-277801.4, June 1, 1998, 98-1 CPD P: 145 at 10. An agency is only required to make a reasonable effort to contact an offeror's references, and, where that effort proves unsuccessful, it is unobjectionable for the agency to evaluate an offeror's past performance based on fewer than the maximum possible number of references the agency could have received. Universal Bldg. Maint., B-282456, July 15, 1999, 99-2 CPD P: 32 at 8 n.1. This is particularly true where, as here, the contracting officer actually contacted the most current and relevant references for the work at hand: the four facilities currently served by Prime under its incumbent contract for these services.  (Prime Environmental Services Company, B-291148.3, March 4, 2003)  (pdf)  (txt version)

Here, the solicitation specifically contemplated evaluation of past contract performance within a specified 3-year period. It is clear that, when PSI made late deliveries within the 3-year period, the firm was still engaging in contract "performance" at the time the late deliveries were made. In our view, the fact that the originally scheduled delivery date was prior to the beginning of the 3-year period does not provide a reasonable basis for excluding that contract performance from consideration--particularly in the context of evaluating "on-time delivery."  Further, where PSI's evaluated price was [deleted] than MEI's, and the only non-price discriminator between the two proposals was PSI's [deleted] rating for on-time delivery, we cannot find reasonable the agency's selection of PSI's [deleted] proposal when that decision incorporates the agency's unreasonable failure to consider delinquent deliveries by PSI.  Accordingly, we sustain the protest on this basis.   (Martin Electronics, Inc., B-290846.3; B-290846.4, December 23, 2002.)  (txt version)

The solicitation explicitly asked offerors to identify the cost of the projects identified, as well as the square footage and other criteria. In our view, it was reasonable and consistent with the evaluation criteria for the evaluators to view cost as a factor in determining whether the projects identified by Knightsbridge were comparable in size to the work contemplated under the proposed contract. Cf. Marathon Constr. Corp., B-284816, May 22, 2000, 2000 ¶ 94 at 5 (noting that language similar to that used by the VA here‑‑requiring experience with “projects of the same or similar size, scope and complexity”‑‑could reasonably include consideration of whether the projects were comparable in value to the project being awarded). Nothing in the RFP at issue here limited the assessment of size to square footage, and the request that offerors identify the dollar value of their past projects put Knightsbridge on notice that those dollar values, and their comparability to the anticipated value to the current project, would be evaluated. Accordingly, while the projects Knightsbridge listed were similar in square footage to the current project, we do not find unreasonable the agency's conclusion that those projects, when measured by the dollar values that Knightsbridge listed, did not meet the RFP experience requirements, and that Knightsbridge's proposal was therefore unacceptable. As previously stated, the dollar value of Knightsbridge's projects was significantly less than the dollar value of the current project. (Knightsbridge Construction Corporation, B-291475.2, January 10, 2003.) (pdf)  (txt Version)

As set forth above, the CPARS for the performance of the current Sigonella contract rated the contractor's as “satisfactory” to “exceptional,” and the response to the questionnaire for this contract rated the contractor's performance as ranging from “better” to “outstanding.” Although we agree with SA that the past performance assessments considered by the agency evidence continual improvement in the contractor's performance on the current contract, there is no requirement that such improvement result in an overall rating of “outstanding,” particularly where, as here, there are CPARS that were prepared within the past 3 years that rate the contractor's performance as only “satisfactory” to “very good.” Sterling Servs., Inc., B-286326, Dec. 11, 2000, 2000 CPD ¶ 208 at 3. In short, we cannot find unreasonable the agency's view that “[w]hile the trend upward was good, there was still some weak past performance that needed to be weighed in the process.” AR at 14.  (Servizi Aeroportuali, Srl, B-290863, October 15, 2002)

In awarding Maranatha a perfect past performance score, UNICOR relied upon a contract that had not been "completed"; indeed, the firm had yet to pass first article testing or deliver any handsets under the contract (although the Army reference was very satisfied with Maranatha's contract performance to date). [2] However, the RFQ contemplated that "completed" contracts would be the basis for the past performance evaluation. Moreover, we question how Maranatha's performance on a contract where it had made no deliveries and had not yet passed first article testing could reasonably be found to justify a perfect score. In this regard, the questions quoted above that the contracting officer said she asked the references primarily pertained to contracts where products were delivered. Furthermore, even though the RFQ requested "at least" three "completed" contracts, Maranatha's quote listed only one completed contract. [3] Thus, Maranatha's perfect past performance rating is neither reasonable nor consistent with the RFQ's evaluation scheme.  (Sonetronics, Inc., B-289459.2, March 18, 2002)

Given that Eagle's experience was more similar to the current requirement than Yardney's, but not recent, and that Yardney's was less similar but more recent, the agency's conclusion that the offerors merited the same rating for the subfactor was reasonable.  (Eagle-Picher Technologies, LLC, B-289093; B-289093.2, December 27, 2001)

With regard to past performance, the RFP provided that the evaluation would consider the quality of the offeror's experience, judged by its recency, relevance, and similarity in scope and magnitude to the RFP project, as well as the quality of the references received on the offeror's listed projects. The Corps was not required to consider other factors besides project type and cost, such as those suggested by the protester, to determine relevance, nor could it consider Lawrence's projects performed more than 3 years ago. In this regard, the RFP specifically states that relevant contracts would be defined in terms of dollar amount ($5 to $7 million) and building type (office/administrative, instructional, and light commercial) and thus it was appropriate for the agency to consider only these factors. Also, the RFP instructions expressly stated that only projects performed in the last 3 years would be considered.  (C. Lawrence Construction Company, Inc., B-287066, March 30, 2001)

The contracting officer sought to identify the most relevant contracts for purposes of assessing PCI’s past performance and, in doing so, considered a relevant contract not listed in PCI’s quotation and disregarded two listed contracts which were not similar to the current requirement. This was reasonable, and well within the agency’s discretion.  (Power Connector, Inc., B-286875; B-286875.2, February 14, 2001)

We find nothing objectionable in the agency's evaluation of Amtech's past performance. The e-mail and letter Millar references do show that the agency was having a problem with Amtech's accelerating its performance to coordinate with the other lobby modernization contract. However, Millar has neither asserted nor shown that Amtech's problem extended beyond the lobby work, and the record shows that Amtech corrected the problem by providing an additional manager for the lobby work. AR at 11. Further, this one instance of negative past performance would have been viewed together with the fact that Amtech's proposal provided 18 references, and that all those contacted by GSA rated Amtech's performance as excellent. Supplemental Technical Evaluation and Consensus Report (CR) at 9. In this regard, the August 2 letter shows that Amtech recently very successfully completed a similar high profile GSA contract (the Byrne/Green Elevator Project). In light of these considerations, there is no basis to conclude that the problem Amtech experienced in performing the lobby work would have affected its past performance rating.  (Millar Elevator Service Company, B-284870.5; B-284870.6, January 31, 2001)

Here, in view of the clear evidence in the record showing that TLT has had ample opportunity to comment on its unsatisfactory performance, we think that the CO reasonably could exercise her discretion in deciding not to communicate further with TLT regarding the alleged negative past performance information in the CCASS database. Given the permissive language of FAR sect. 15.306(a)(2), the fact that TLT may wish to rebut or provide further comments on the information in the database does not give rise to a requirement that the CO give TLT an opportunity to do so. [6] See A.G. Cullen Constr., Inc., B-284049.2, Feb. 22, 2000, 2000 CPD para. 45 at 5-6.  (TLT Construction Corporation, B-286226, November 7, 2000)

In these circumstances, where the solicitation involves a very complex and somewhat unique requirement and the protester has not been the prime contractor on a single contract that involves the same requirements or can be shown to be extremely similar, we find reasonable the agency's conclusion that "the evaluated contracts only give a picture of what [Jones] can do for contracts with less complex requirements than what is required" for this effort. Agency Report II, Tab 10, Source Selection Decision (June 21, 2000), at 5. In sum, the agency reasonably concluded that Raytheon offered better performance on substantially more relevant contracts than Jones.  (J. A. Jones Management Services, Inc., B-284909.5, October 2, 2000)

In our view, it was unreasonable for MTMC to compare the absolute number of negative performance actions an offeror received, without considering that number in the context of the number of shipments the offeror had made over the relevant time period.  (Green Valley Transportation, Inc., B-285283, August 9, 2000)

In light of the evaluation scheme, it was reasonable for the Navy to give a more favorable risk rating to Nova than to Marathon, based on its having successfully performed more relevant contracts. See, e.g., Browning Ferris Indus. of Hawaii, Inc., B-281285, Jan. 21, 1999, 99-1 CPD para. 35 at 6; Ogden Support Servs., Inc., B-270012.4, Oct. 3, 1996, 96-2 CPD para. 137 at 3. While Marathon may have performed smaller projects that encompassed the elements of the larger project involved here, its arguments ignore the fact that combining numerous elements into a single large project may introduce performance challenges and risks not present in smaller scale projects; the RFP's focus on the size, scope and complexity of past projects reflects the agency's concern in this regard.  (Marathon Construction Corporation, B-284816, May 22, 2000)

Since the COR had not provided a final rating by the time the SEB was completing its past performance evaluation, and FDC provided a response in which it denied that it was responsible for the difficult transition, it was reasonable not to automatically attribute the problems to FDC. We think the SEB therefore reasonably could conclude that the information available did not support a finding of deficient past performance, and thus did not warrant downgrading FDC. See The Communities Group, supra.  (Dynacs Engineering Company, Inc., B-284234; B-284234.2; B-284234.3, March 17, 2000)

With regard specifically to clarifications concerning adverse past performance information to which the offeror has not previously had an opportunity to respond, we think that, for the exercise of discretion to be reasonable, the contracting officer must give the offeror an opportunity to respond where there clearly is a reason to question the validity of the past performance information, for example, where there are obvious inconsistencies between a reference's narrative comments and the actual ratings the reference gives the offeror. In the absence of such a clear basis to question the past performance information, we think that, short of acting in bad faith, the contracting officer reasonably may decide not to ask for clarifications.  (A. G. Cullen Construction, Inc., B-284049.2, February 22, 2000)

Since the RFP indicated that proposals would be qualitatively evaluated for quality of performance relative to the size and complexity of the JOC procurement under consideration, it follows that a proposal reflecting successful past performance on contracts closer in size and complexity to the procurement under consideration should be rated higher than a proposal reflecting successful performance on less similar contracts. See Ogden Support Servs., Inc., B-270012.4, Oct. 3, 1996, 96-2 CPD para. 137 at 3; Chem-Services of Indiana, Inc., B-253905, Oct. 28, 1993, 93-2 CPD para. 262 at 3-4. Here, the evaluation did not so qualitatively evaluate past performance.  (Beneco Enterprises, Inc., B-283512, December 3, 1999)

While it is appropriate, in evaluating past performance, to consider a contractor's "combative" attitude, we have recognized that absent some evidence of abuse of the contract disputes process, contracting agencies should not lower an offeror's past performance evaluation based solely on it having filed claims; firms should not be prejudiced in competing for other contracts because of their reasonable pursuit of such remedies in the past.  (OneSource Energy Services, Inc., B-283445, November 19, 1999)

Absent some evidence of abuse of the contract disputes process, contracting agencies should not lower an offeror's past performance evaluation based solely on its having filed claims. AmClyde Engineered Prods. Co., Inc., B-282271, B-282271.2, June 21, 1999, 99-2 CPD para. 5 at 6 n.5. Contract claims, like bid protests, constitute remedies established by statute and regulation, and firms should not be prejudiced in competing for other contracts because of their reasonable pursuit of such remedies in the past.  (Nova Group, Inc., B-282947, September 15, 1999)

The agency's reading of FAR sect. 42.1503(e) is, in our view, the more reasonable reading of the provision and is consistent with the provision's regulatory history. Specifically, the regulatory history of FAR sect. 42.1503(e) reflects that the amount of time that past performance information could be retained and considered was lengthened from 3 years to, ultimately, "three years after completion of contract performance" in light of the belief that "the retention period should exceed the length of the contract." 58 Fed. Reg. 3575 (1993). Accordingly, we agree with the agency that a contractor's past performance information may be considered for up to 3 years after the completion of contract performance as a whole, rather than for only 3 years after each incident of performance under the contract, as argued by the protester.  (D. F. Zee's Fire Fighter Catering, B-280767.4, September 10, 1999)

Where we have charged an agency with responsibility for considering such outside information, the record has demonstrated that the information in question was "simply too close at hand to require offerors to shoulder the inequities that spring from an agency's failure to obtain, and consider, the information."  (TRW, Inc., B-282162; B-282162.2, June 9, 1999)

Under the circumstances, the agency could not reasonably ignore personally known information about GTSD's prior experience on the PHNS contract merely because the firm did not submit a Contractor Past Performance Data Sheet for that contract. See Safeguard Maintenance Corp., B-260983.3, Oct. 13, 1995, 96-2 CPD para. 116 at 12. While there is no legal requirement that all past performance references be included in a valid review of past performance, some information is simply too close at hand to ignore. See International Bus. Sys., Inc., B-275554, Mar. 3, 1997, 97-1 CPD para. 114 at 5.  (GTS Duratek, Inc., B-280511.2; B-280511.3, October 19, 1998)

IBSI's challenge to the past performance evaluation is that the agency could not reasonably ignore IBSI's past performance on the Brockton/West Roxbury contract when that contract involved the same agency, the same CO, and virtually the same services as here. IBSI further argues that this result is untenable when other evidence--i.e., the CO's letter to the SBA--demonstrates the CO's first-hand knowledge of IBSI's past performance of this work. Thus, IBSI argues that the inclusion of the Brockton/West Roxbury experience would have enhanced its standing in the area of past performance, and would have made less likely a finding that the two offerors were equal in this area. We agree. We start our review with the evaluation approach outlined in the agency report. As described above, the conclusion that IBSI and DNA were essentially equal under the past performance factor, leading to a selection decision based on price, was based on an evaluation of one relevant reference for IBSI and no relevant references for DNA. Thus, the contracting officer compared a rating of "good" with a rating of "neutral" to reach her conclusion that the offerors were essentially equal. In the abstract, we have no basis to disagree with this conclusion.

Where an RFP identifies past performance and price as the evaluation factors and indicates that an offeror with a better past performance record than that of another offeror can expect a higher past performance rating, proposals must be evaluated on that basis. The selection official, however, has the discretion to decide the appropriate trade-off between past performance and price in determining which proposal represents the best value to the government. Excalibur Sys., Inc., B-272017, July 12, 1996, 96-2 CPD ¶ 13 at 3. Such a trade off is not precluded under an evaluation scheme specifying a "neutral" rating for vendors with no past performance record. Engineering and Computation, Inc., B-275180.2, Jan. 29, 1997, 97-1 CPD ¶ 47 at 4-5; Excalibur Sys., Inc., supra.

Our disagreement with the agency springs from its overly mechanical application of its procedures for evaluating past performance. While the VA is correct in its view that there is no legal requirement that all past performance references be included in a valid review of past performance, Dragon Servs., Inc., B-255354, Feb. 25, 1994, 94-1 CPD ¶ 151 at 8; Questech, Inc., B-236028, Nov. 1, 1989, 89-2 CPD ¶ 407 at 3, some information is simply too close at hand to require offerors to shoulder the inequities that spring from an agency's failure to obtain, and consider, the information. See G. Marine Diesel, 68 Comp. Gen. 577 (1989), 89-2 CPD ¶ 101 at 5-6; New Hampshire-Vermont Health Serv., 57 Comp. Gen. 347 (1978), 78-1 CPD ¶ 202 at 12-13; Continental Maritime of San Diego, Inc., B-249858.2; B-249858.3, Feb. 11, 1993, 93-1 CPD ¶ 230 at 6-8; G. Marine Diesel; Phillyship, B-232619; B-232619.2, Jan. 27, 1989, 89-1 CPD ¶ 90 at 4-5; Inlingua Schools of Languages, B-229784, Apr. 5, 1988, 88-1 CPD ¶ 340 at 5.

Here, the record shows that IBSI's proposal clearly identified a recent contract involving the same agency, the same services, and the same contracting officer, and asked that its performance of this contract be considered as part of its evaluation, as the solicitation anticipated and required. The record also shows that the contracting officer was aware of IBSI's performance of this contract and had termed it "exemplary" in a letter to the SBA written barely 4 months before the award decision here. Under these circumstances, we conclude that the agency unreasonably failed to consider IBSI's performance on its earlier contract simply because an individual in the agency did not complete the assessment required. See G. Marine Diesel; Phillyship, supra (protest sustained where Navy elected not to consider unsatisfactory past performance of awardee involving similar services and the same command because awardee did not include the controversial contract on its list of references for the past performance review).  (International Business Systems, Inc., B-275554, March 3, 1997)  (pdf)