By Linda Koone on Wednesday, April 25, 2001 - 12:17 pm:

I'm looking for some opinions.

Does the following statement contained in a source selection plan satisfy FAR 15.305(a)(2)(iv), which states that an offeror without a record of past performance may not be evaluated favorably or unfavorably on past performance?

'If an offeror lacks past performance information, they shall receive a neutral evaluation on this subfactor. When offers are ranked in this subfactor, offers with good past performance will be ranked ahead of offers with neutral past performance and offers with neutral past performance will be ranked ahead of offers with poor past performance.'

Thanks.

By Anonymous on Wednesday, April 25, 2001 - 12:52 pm:

Linda,

My personal preference is not stating it in this way. What you end up as ranking can be can be nebulous. Rather I like equating the factor in terms of risk. Therefore I might state "offers with favorable past performance data constitute a higher probability of successful performance than a neutral rating... and offers with neutral data constitute less of a risk than offers with unfavorable past performance data." Hope this helps

By Loki on Thursday, April 26, 2001 - 09:56 am:

Linda, it's a way.

Anon 4/25 - 12:52, are you working up to using a LOCAR system?

By Linda Koone on Thursday, April 26, 2001 - 10:00 am:

Anonymous/Loki:

Thanks for your responses. I agree that assessing past performance in terms of risk is better than 'ranking' past performance. I guess my real question is whether a company with no past performance always represents a better risk or deserves a better ranking than one with unfavorable past performance.

Let's say you are on a journey. The road you are on branches out into three separate toll roads, each promising to take you to your final destination. You ask the toll collector about each road. The toll collector tells you the following:

'Road A' is, by far, the highest priced route. It's been traveled often and will most likely get you to your destination on time and in good condition. It's a scenic route, has call boxes every ½ mile, and is well maintained. 'Road B' is less expensive than 'Road A'. It's also been well traveled. You will probably get to your destination, maybe not as quickly, and you may suffer some bumps or encounter some surprises along the way. Call boxes are placed sporadically and the view is very mundane. 'Road C', is the least expensive route, but it has just recently opened and has never been traveled. The sign says that this road will take you where you want to go, but there's no proof of this. It's a totally unknown route. It may or may not have call boxes, the atmosphere is unknown, as are the road conditions.

Given this scenario, does the lack of 'past performance' information on 'Road C' automatically make it a better risk than 'Road B', i.e., is unknown always better than unfavorable?

By Anonymous on Thursday, April 26, 2001 - 11:09 am:

Linda:

You have loaded your scenario!

With regard to Road B you say, "You will probably get to your destination... ." Given that supposition and the information you have about Road C, Road B probably is the better choice.

But what if Road B has been impassable under certain conditions and Road C is known to be well-constructed even though no one has used it yet?

Everything depends on the information you have.

By Linda Koone on Thursday, April 26, 2001 - 01:05 pm:

Anonymous:

You're right, and you're helping me make my point.

Until you actually receive proposals and learn about each company's capabilities, how can you say what your reaction to a lack of past performance will be?

I suppose I get nervous when the language included in my initial post gets put in solicitations. I'm not sure that you can be that definite.

By Mike on Thursday, April 26, 2001 - 01:52 pm:

OFPP's "BEST PRACTICES for COLLECTING AND USING
CURRENT AND PAST PERFORMANCE INFORMATION" states the following:
"This requirement is implemented at FAR Part 15.305(a)(2)(iv): "In the case of an offeror without a record of relevant past performance or for whom information on past performance is not available, the offeror may not be evaluated favorably or unfavorably on past performance." We expect this will happen very rarely. In this case, the best practice is to give the offeror the middle rating in the rating system specified (e.g., 50 on a 100-point scale, 3 on a 5-point scale, satisfactory on an adjective scale, or the middle color on a color rating system)"

I personally disagree with this suggestion. I don't understand how OFPP thinks that giving someone 50/100 points isn't treating them favorably, especially compared to someone getting below 50 points who actually has past performance. In the event that someone has no past performance (an event I've yet to actually encounter) my office's solicitations have been stating that they will not be evaluated on that factor.

Mike

By Anonymous on Thursday, April 26, 2001 - 01:57 pm:

Linda/Loki,

I'm Anon 4/25 12:52. Yes, Linda. You are right on with the last comment. Loki seemed to hit what I advocated right on the head with the LOCAR question. You can't look at risk/past performance in isolation. In effect, you examine what each offer is promising to do. You look at the benefits that derive from each one. Then you go to past performance and assess the risk with each in delivering on their promises to perform. Finally, you see how price/costs compare and decide which offers the value you seek.

By Charlie Dan on Thursday, April 26, 2001 - 02:37 pm:

Linda:

I believe if contracting officers request and use all the past performance information available to them, it should be extremely rare that they will ever face a proposal from an offeror without a record of past performance. FAR 15.305(a)(2)(iii) says: "The evaluation should take into account past performance information regarding predecessor companies, key personnel who have relevant experience, or subcontractors that will perform major or critical aspects of the requirement when such information is relevant to the instant acquisition."

If you receive a proposal from an offeror that has no key personnel with relevant experience, no experience as a predecessor company, and no major subcontractors with relevant experience, you have an offeror that cannot be determined responsible. Even though FAR 9.104-1(c) indicates that a lack of relevant performance history, in and of itself, cannot be used to determine that an offeror is not responsible, subparagraph (f) goes on to require "the necessary organization, experience, accounting and operational controls, and technical skills, or the ability to obtain them" in order to determine an offeror responsible.

By joel hoffman on Friday, April 27, 2001 - 10:25 am:

Linda - If you are concerned about tying your hands ahead of time, I suggest not using language in the solicitation which ties you to a direct comparison between the "neutral" risk rating and someone with a poor track record or a great track record.

However, I believe I agree with the Anon 4/25 12:52 post.

I would expand the evaluation criteria. Recommend specifically stating something to the effect that the offeror can or should help establish some "past performance" consideration through the past performance of relevant experience by personnel who will play a key, relevant role on this project. Just my opinion. Happy Sails! Joel

By Loki on Monday, April 30, 2001 - 12:18 pm:

Linda - yes, you're going to need to pre-determine the standards for evaluating past performance up front.

You will be able to assess the RELATIVE merits/risk/etc..between the offers after a best judgment rating of individual offers against the established standards for each factor.

The relative (to other offers) evaluation is usually as touched on above, and there are thick books on this stuff. The relative process is typically described as a series of paired comparisons in which the differences in nonprice factors are traded off against the differences in price (the process varies, depending upon your construct & decision rule.)

By Monty on Monday, April 30, 2001 - 11:33 pm:

How do you define "relevant" past performance?

Suppose you are buying systems development services for an Army transportation-management computer application, what is relevant?

Experience developing complex computer applications?

Experience developing complex computer applications for the Army?

Experience developing complex computer transportation-management applications for the Army?

Experience developing complex computer transportation-management applications for the Army's [insert name of specific program here] program?

I've seen all of these used as past performance criteria. Which should be the cut-line for saying that a potential offeror lacks relevant past performance?

By Anonymous on Tuesday, May 01, 2001 - 09:10 am:

I am beginning to think that past performance is being used in an unintended way....less as a method of risk management and more of a reward scheme for certain contractors. Its the impression I have after reading numerous evaluation criteria which ,in many instances, tend to leave only one contractor standing. If this were true it would be a disturbing trend. Any thoughts?

By bob antonio on Tuesday, May 01, 2001 - 09:18 am:

Each of the factors below is potentially restrictive of competition and I would need to see support showing that it is a valid agency requirement. I have highlighted my areas of concern.

Experience developing complex computer transportation-management applications for the Army's [insert name of specific program here] program?

Experience developing complex computer applications for the Army?

Experience developing complex computer transportation-management applications for the Army?

By Mongoose on Tuesday, May 01, 2001 - 09:24 am:

Anonymous:

What's "disturbing" about using past performance as a way to reward contractors that did a good job by giving them a competitive advantage? The assessment of contractor performance and the use of that information during source selection is the ultimate performance incentive.

By Anonymous on Tuesday, May 01, 2001 - 09:33 am:

The how to do it must "simply" be reasonable.

I'm sure the following is not new to most, I keep such things handy:

In evaluating past performance under solicitation for quantity of leather, agency reasonably disregarded non-leather supply contracts in its evaluation of the protester, and limited its evaluation to leather contracts, on the basis that leather contracts were the most relevant.

In evaluating past performance, an agency has discretion to determine the scope of the vendors' performance history to be considered, provided that it evaluates all submissions on the same basis and consistent with the solicitation. OMV Medical, Inc.; Saratoga Medical Ctr., Inc., B-281387 et al., Feb. 3, 1999, 99-1 CPD 52 at 4. An agency properly may base its evaluation on contracts it believes are most relevant to the solicitation, USATREX Int'l, Inc., supra, at 4; Braswell Servs. Group, Inc., B-278921.2, June 17, 1998, 98-2 CPD 10 at 6; it has discretion to consider information other than that provided by the vendors, TEAM Support Servs., Inc., B-279379.2, June 22, 1998, 98-1 CPD 167 at 6, and need not consider all references a vendor submits. Advanced Data Concepts, Inc., B-277801.4, June 1, 1998, 98-1 CPD 145 at 10.

We find nothing unfair or otherwise improper in the contracting officer's actions. Her different approach to evaluating PCI and the other vendor reflected those firms' different circumstances, not improper disparate treatment. In evaluating both firms, the contracting officer considered similar contracts to the extent possible, consistent with the RFQ reference to "similar" contracts. Similar (leather supply) contracts, including the one she discovered in the agency's records, were available for PCI, so she considered those in PCI's evaluation. Since the other vendor had not performed leather supply contracts, she could not consider similar contracts to evaluate its past performance. Thus, she considered the firm's dissimilar contracts, and then penalized it in the evaluation by finding that the lack of past similar contracts made the performance risk too great to permit award to the firm. In our view, nothing in this record indicates that the treatment of PCI was unreasonable.

+++

In reviewing protests challenging the evaluation of proposals, we do not conduct a new evaluation or substitute our judgment for that of the agency but examine the record to determine whether the agency's judgment was reasonable and in accord with the RFP evaluation criteria. Abt Assocs., Inc., B-237060.2, Feb. 26, 1990, 90-1 CPD 223 at 4. Judgments are by their nature often subjective; nevertheless, the exercise of judgment in the evaluation of proposals must be reasonable and bear a rational relationship to the announced criteria upon which competing offers are to be selected. Southwest Marine, Inc.; American Sys. Eng'g Corp., B-265865.3, B-265865.4, Jan. 23, 1996, 96-1 CPD 56 at 10.

In order for us to review an agency's evaluation judgment, an agency must have adequate documentation to support its judgment. Id. While point scores are useful as guides to decision-making, they must be supported by documentation of the relative differences between proposals, their strengths, deficiencies, weaknesses and risks. Federal Acquisition Regulation § 15.305(a); Century Envtl. Hygiene, Inc., B-279378, June 5, 1998, 98-1 CPD 164 at 4. While an agency is not required to retain every document or worksheet generated during its evaluation of proposals, the agency's evaluation must be sufficiently documented to allow review of the merits of a protest. KMS Fusion, Inc., B--242529, May 8, 1991, 91-1 CPD 447. Where an agency fails to document or retain evaluation materials, it bears the risk that there may not be adequate supporting rationale in the record for us to conclude that the agency had a reasonable basis for the source selection decision. Southwest Marine, Inc.; American Sys. Eng'g Corp., supra, at 10.

The evaluation of technical proposals is a matter primarily within the discretion of the contracting agency; in reviewing challenges to an agency's evaluation, our Office does not reevaluate technical proposals but, instead, considers whether the evaluation was reasonable and consistent with the solicitation's evaluation criteria. Dual, Inc., B-279295, June 1, 1998, 98-1 CPD 146 at 3.

+++

TITLE: Clean Venture, Inc., B-284176, March 6, 2000
BNUMBER: B-284176
DATE: March 6, 2000
**********************************************************************
Clean Venture, Inc., B-284176, March 6, 2000

Decision

Matter of: Clean Venture, Inc.

File: B-284176

Date: March 6, 2000

Michael Persico and Michael Lancos for the protester.

William E. Hughes III, Esq., Whyte, Hirschboeck, Dudek, for Safety-Kleen, Inc., an intervenor.

Reba M. Harrington, Esq., Defense Logistics Agency, for the agency.

Christine Davis, Esq., and James Spangenberg, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Agency reasonably evaluated protester's past performance as "fair" under a solicitation for a hazardous waste disposal contract based on the protester's "good" performance on its prior contracts that did not require the handling of the large quantities and the numerous varieties of waste at a large number of performance locations as required by the solicitation, as compared to the awardee's "good" past performance on contracts very similar in size and scope to the solicited work. The RFP provided for award based on a tradeoff between past performance, price and other factors. [1] RFP sect. M.13(a). The solicitation stated that, as between past performance and price, price was a less important, but still significant factor. RFP sect. M.13(c). With regard to the past performance evaluation, the RFP provided that the government would evaluate offerors' "[p]ast performance on references that are of a similar nature to the subject solicitation." RFP sect. M.13(d)(1)(i). For purposes of this evaluation,
the RFP asked each offeror to provide information about its performance under contracts "for the same or similar services" during the past 2 years. RFP sect. L.15(a). For each project submitted as a past performance reference in its proposal, the offeror was to list the waste streams serviced, the corresponding quantity removed, any related support services provided (such as roll-off services), and the annual cost of the work, and was to discuss its qualifications and experience relative to each project. [2] RFP sect. L.15(a)(1), (b); Agency Report, Tab 9, Protester's Past Performance Proposal, DRMS Form 1989.

It is not the function of our Office to evaluate past performance information de novo. Rather, we will examine an agency's evaluation only to ensure that it was reasonable and consistent with the stated evaluation criteria and applicable statutes and regulations, since determining the relative merits of offerors' past performance is primarily a matter within the contracting agency's discretion. Pacific Ship Repair and Fabrication, Inc., B-279793, July 23, 1998, 98-2 CPD para. 29 at 3-4.

As noted above, the RFP informed offerors that the agency would evaluate information provided by each offeror concerning contracts for "the same or similar services" performed within the last 2 years. Consistent with this advice, the agency evaluated the degree of similarity between the offerors' prior contracts and the RFP requirements. Based on this analysis, the agency
found that the protester's contracts were smaller and less difficult than the RFP work, but that the protester's successful performance of these contracts gave it a fair chance of successfully meeting the greater demands of the RFP. Our review of the record, including the hearing testimony, shows that the agency had a reasonable basis for its findings in this regard. [6]

+++

BNUMBER: B-278921.2
DATE: June 17, 1998
TITLE: Braswell Services Group, Inc., B-278921.2, June 17, 1998**********************************************************************

DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release.

Matter of:Braswell Services Group, Inc.

File:B-278921.2

Date:June 17, 1998

Patricia H. Wittie, Esq., and Karla J. Letsche, Esq., Wittie & Letsche, for the protester.
Sharon Hershkowitz, Esq., Jannika E. Cannon, Esq., Lisa L. Hare, Esq., and Catherine A. D'Andrea, Esq., Department of the Navy, for the agency.
Linda C. Glass, Esq., and Paul I. Lieberman, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision.

DIGEST

1. Agency reasonably evaluated protester's past performance as unsatisfactory where record shows that the protester's most relevant contract performance was reasonably perceived as inadequate by the agency, notwithstanding protester's allegations that the agency overstated the scope and significance of the performance deficiencies.

2. Agency reasonably evaluated awardee's past performance record as satisfactory, which was the rating closest to a neutral assessment, where the agency did not have available on file past performance rating information for the awardee which it viewed as sufficient to permit a full evaluation; satisfactory rating did not overstate the record, since agency had orally received a favorable overall assessment of the awardee's relevant performance by a cognizant contracting official. 

3. Agency selection of slightly higher-priced proposal with a satisfactory past performance rating instead of the lower-priced
proposal with an unsatisfactory rating was reasonable and consistent with the solicitation's evaluation scheme, which weighted past performance as slightly more important than price. 

We will review an evaluation of an offeror's performance risk to ensure that it was reasonable and consistent with the stated
evaluation criteria. Dragon Servs., Inc., B-255354, Feb. 25, 1994, 94-1 CPD para. 151 at 6. An agency's evaluation of past performance may be based upon the procuring agency's reasonable perception of inadequate prior performance, even where the contractor disputes the agency's interpretation of the facts. Pannesma Co. Ltd., B-251688, Apr. 19, 1993, 93-1 CPD para. 333 at 6.

With respect to Braswell's contention that its list of other contracts was not properly considered in the past performance evaluation, there is no legal requirement that all references listed in a proposal be checked.

By Mongoose on Tuesday, May 01, 2001 - 09:34 am:

Bob:

See: Systems Application & Technologies, Inc., B-270672, April 8, 1996, in which the GAO denied a protest that the Naval Air Warfare Center's experience requirements were unduly restrictive. The RFP required offerors' key personnel to have "Naval Air" experience. The protester complained that only the incumbent contractor could meet the requirement.

The GAO said that the experience requirement was "unobjectionable" and said, "The Navy's explanation for the requirements generally establishes that familiarity with Naval Air systems and procedures will facilitate successful performance... The fact that [the protester] has had difficulty finding personnel who can meet the Navy's qualifications, while unfortunate, is not a basis for challenging requirements such as these, which are reasonable related to the agency's needs."

By bob antonio on Tuesday, May 01, 2001 - 10:02 am:

In this case, the Comptroller General concluded that "familiarity with the Naval Air systems and procedures" would facilitate successful performance. Accomplishing the mission is more important than obtaining competition.

An agency's requirement is just that. Requiring organizations know best what their needs are. As long as they have a reasonable basis for a requirement, the requirements stands.

By Anonymous on Tuesday, May 01, 2001 - 10:49 am:

Momgoose and Bob
I inherently agree however ,at a recent DoD class a speaker emphasized that risk management was not the same as risk avoidance and what I see frequently is the latter not the former. Perhaps this issue may be too difficult to fully explain but one knows it when one sees it,I guess. Bob s 9:18 is very much what I was thinking about. What I find "disturbing" is similiar language in procurements dealing with very simple requirements. I was just thinking out loud.

By Loki on Tuesday, May 01, 2001 - 11:48 am:

A procuring agency is required to specify its needs and solicit offers in a manner designed to achieve full and open competition, and may include restrictive provisions or conditions in a solicitation only to the extent necessary to satisfy the agency's needs. 41 U.S.C. §§ 253a(a)(1)(A), (a)(2)(B) (1994). The determination of the government's needs and the best methods for accommodating those needs are generally the responsibility of the contracting agency; we will review such determinations to confirm that they are reasonably based. HG Properties A, L.P., B-284170 et seq. Mar. 3, 2000, 2000 CPD 36 at 10.

+++

The determination of the government's needs and the best method of accommodating them is primarily the responsibility of the procuring agency, since its contracting officials are most familiar with the conditions under which supplies, equipment, and services have been employed in the past and will be utilized in the future. DGS Contract Servs., Inc., B-249845.2, Dec. 23, 1992, 92-2 CPD 435 at 2; Westbrook Indus. Inc., B-248854, Sept. 28, 1992, 92-2 CPD 213 at 2. Where a protester challenges a specification as unduly restrictive, it is the agency's responsibility to establish that the specification is reasonably necessary to meet its needs. CardioMetrix, B-259736, Apr. 28, 1995, 95-1 CPD 223 at 3. The adequacy of the agency's justification is ascertained through examining whether the agency's explanation is reasonable, that is, whether it can withstand logical scrutiny. Keeson, Inc.; Ingram Demolition, Inc., B-245625, B-245655, Jan. 24, 1992, 92-1 CPD 108 at 4.

By Mongoose on Tuesday, May 01, 2001 - 11:54 am:

Anonymous:

As a KO, I think that the evaluation of past performance in a source selection is about risk avoidance (or risk minimization), not risk management. I think that it makes sense to avoid a risk unless the potential payoff makes the risk worth taking.

If I'm looking at two companies and one has a better record of performance than the other (whose record may not be BAD, just not as good), I'm going with the company with the better record every single time UNLESS the other one offers me the prospect of some offsetting benefit. That benefit MIGHT be lower price, but the promise of a lower price (that's all it is -- a promise) might not be enough, depending on the nature of the acquisition.

So as a KO I'm into risk avoidance or minimization. I consider risk management to be the joint responsibility of the Government and contractor project managers.

By formerfed on Tuesday, May 01, 2001 - 11:56 am:

Something interesting is showing up, and that is most people seem to like making past performance very specific in relevancy. I'm not sure that works best. Personally I like tieing specific experience and qualifications into a factor. Then I like assessing risk using past performance. In most cases, a company with excellent past performance will succeed on most anything they attempt. The elements that make the best that way (committed management, employees with pride on doing a good job, focus on meeting/exceeding customer expectations, etc.) manage to succeed on all it does.

By bob antonio on Tuesday, May 01, 2001 - 12:17 pm:

Loki:

And here are a few quotes on incumbency from the Comptroller General.

"An agency is not required to equalize competition for a particular procurement by considering the competitive advantage accruing to an offeror due to its incumbent status provided that such advantage is not the result of unfair government action or favoritism." (85-2 CPD 309)

On the other hand,

"it is clear from the record that the Navy at all times contemplated a sole-source award to (incumbent) and drafted the disputed requirements with only (incumbent) in mind. The Navy has not shown convincingly, however, that its needs can be satisfied only by the two employees of the incumbent on whose backgrounds the requirements are based." (86-1 CPD 166)

By Anonymous on Tuesday, May 01, 2001 - 02:02 pm:

FF & BOB
FOR WHATEVER ITS WORTH I AGREE WITH YOU BOTH