|
By
PWG on Wednesday, November 06, 2002
- 10:31 am:
I am interested to hear what people think about this
(reference FAR 15.304). Under what circumstances is it most
appropriate to give significantly more weight to the
Technical/non-price evaluation factors than to cost or price?
When conducting a best value procurement and having the two
factors approximately equal, the Contracting Officer may still
determine than the technical benefit is worth the extra cost and
can make award to a higher cost/price proposal. Doesn't the
definition and process of a best value procurement provide the
Government with sufficient latitude for awarding to a higher
priced offeror?
By
Vern Edwards on Wednesday, November
06, 2002 - 11:00 am:
You can award to a higher priced offeror even when
price/cost is the most important factor.
Make technical/non-price factors more important than price or
cost when technical/non-price factors are more important to you
than price or cost. When you tell offerors that
technical/non-price factors are more important, you are
essentially saying the following:
Offerors, if, when preparing your proposal, you face a choice
between giving us better non-price promises or giving us a lower
price, keep in mind that we are more interested in the better
non-price promises than we are in getting a lower price.
However, keep in mind that you are in a competition and that you
don't know what the other offerors are promising, so don't go
overboard.
That's it. Not much help, is it? But that's the challenge of
competing in a marketplace in which you have limited info about
what your competitors are doing. In an open marketplace (e.g.,
the local supermarket) firms know exactly what their competitors
are offering and at exactly what price, and can adjust
accordingly. Government procurements are not conducted in an
open marketplace; the government generally cannot and does not
disclose the non-price contents of the competing offers. So,
when a competitor has to price its offer, it generally does not
know what it's up against and has to make its decisions on the
basis of very limited information.
By
joel hoffman on Wednesday, November
06, 2002 - 11:13 am:
In my experience, if the contract will be a cost
reimbursement type with a not entirely defined scope, sometimes
the pressure of competition causes some (many?) offerors to
"understate" (low-ball) the price proposal.
If you state that price and technical are essentially equal, you
can encourage low-balling. The owner may not be in a position to
refute the offered prices, if it can't adjust upward on the
basis of "price realism" analysis. You could face a real problem
by not selecting the low-baller, if there aren't very
significant differences in the technical qualifications or
performance solution.
Another situation may be where your primary consideration is the
qualifications of the contractor (probably including past
performance in controlling cost and schedule growth). This might
be for unique or state-of-the art type work.
You can state that price is less important than technical, but
also include a cost ceiling - depending upon the type of
acquisition.
Just some ideas. happy sails! joel hoffman
By
joel hoffman on Wednesday, November
06, 2002 - 11:46 am:
clarification - the above comments were directed to
cost reimbursement contracts...
happy sails! joel hoffman
By
PWG on Thursday, November 07, 2002
- 07:20 am:
Thanks for the insight. This gives me a much better
perspective. This was a topic that I thought I understood until
I tried to explain it to someone last week. My explanation was
so weak it didn't even make sense to me.
I am still thinking about Vern's comment that award can be made
to a higher priced offeror even when cost/price is the most
important factor. I am having a difficult time imagining a
scenario where that would apply. It seems like that would be
very difficult to justify. My Contracting Office treats those
like "lowest cost, technically acceptable" evaluations.
By
Vern Edwards on Thursday, November
07, 2002 - 09:21 am:
PWG:
See, Day & Zimmermann/IMR L.L.C., B-280568, B-280569,
Oct. 19, 1998, in which the GAO said:
"Even under a solicitation which states that proposed price is
more important than technical evaluation factors, the
contracting agency is not required to make award to the firm
offering the lowest price; the agency retains the discretion to
select a higher-priced, technically higher-rated proposal, if
doing so is in the government's best interest and is consistent
with the solicitation's stated evaluation and source selection
scheme. See University of Kansas Medical Ctr., B-278400,
Jan. 26, 1998, 98-1 CPD ¶ 120 at 6; Nomura Enter., Inc.,
B-277768, Nov. 19, 1997, 97-2 CPD ¶ 148 at 2-5."
For an older case, see: CORVAC, Inc., B-244766, Nov. 13,
1991.
By
Vern Edwards on Thursday, November
07, 2002 - 10:08 am:
PWG:
An additional note:
In Environmental Tectonics Corp., B-280573.2, Dec. 1,
1998, a commercial items acquisition under the simplified
acquisition test program to procure acrllic replacement windows
for hypobaric low pressure chambers, the RFQ stated the
evaluation factors to be as follows, in descending order of
importance: price, delivery schedule, risk, and past
performance. The agency's evaluation of non-price factors for
the protester and the winner were as follows:
Protester: deliver schedule - acceptable; risk -
acceptable; past performance - high risk.
Winner: delivery schedule - acceptable; risk -
acceptable; past performance - low risk.
According to the GAO, the protester's price was about 46
percent lower than the winner's, leading the protester to
complain that the price/technical tradeoff was unsound. The GAO
denied the protest, saying:
"In a best value procurement, price is not necessarily
controlling in determining the quotation that represents the
best value to the government. Rather, that determination is made
on the basis of whatever evaluation factors are set forth in the
solicitation, with the source selection official often required
to make a price/technical tradeoff to determine if one
quotation's technical superiority is worth the higher price that
may be associated with the quotation. In this regard, price/past
performance tradeoffs are permitted when such tradeoffs are
consistent with the solicitation's evaluation scheme. See
Rotair Indus., Inc., B-247435.2, July 15, 1997, 97-2 CPD ¶
17 at 3. In this case, where the RFQ does not expressly specify
that price will be the determinative factor for award, the
agency retains the discretion to select a vendor with a
higher-priced quotation and higher past performance rating, if
doing so is in the government's best interest and is consistent
with the solicitation's stated evaluation and source selection
scheme. See University of Kansas Med. Ctr., B-278400,
Jan. 26, 1998, 98-1 CPD ¶ 120 at 6. While [the protester's]
quotation was approximately 46 percent lower than the quotation
of [the winner], we conclude that the contracting officer
reasonably determined that the performance risk associated with
[the protester], based on past performance information,
outweighed the value to the agency of the firm's low price."
You said that your office treats procurements in which price is
the most important factor like
lowest-price-technically-acceptable (LPTA) procurements. In LPTA
procurements, price is the determinative factor for award. It's
a mistake to treat tradeoff process ("best value") procurements
like LPTA procurements, no matter what the relative importance
of the factors -- a mistake which could lead to a sustained
protest under the right circumstances. Treat all tradeoff
process procurements like tradeoff process procurements, no
matter what the relative importance of the factors.
By
Eric Ottinger on Thursday, November
07, 2002 - 09:44 pm:
Three comments –
To understand the Comp. Gen. it helps to remember that this is
the COMPTROLLER General. Their fundamental charter is to assure
that Uncle Sam gets good value for the dollar. In my view, the
1102 community is typically more “legalistic” than the lawyers
who decide the protests, and the lawyers at GAO are less
legalistic, more pragmatic and practical than the 1102 community
expects. This makes for no end of misunderstanding.
It appears that there are a number of veteran 1102’s who
continue to operate in a Low Price Technically Acceptable mode
because that is the way they were trained and low cost appears
to be the safest, least controversial choice in all
circumstances. They need to pay more attention to protests. In a
situation where the cost difference is very small and the
technical difference is significant, the PCO is taking a big
chance on a sustained protest if he/she picks the low cost
offer.
Up to the Part 15 Rewrite, LPTA and best value were understood
to be polar opposites. For some reason the rewrite team decided
to confuse us all by making LPTA part of the “best value”
continuum. I guess this was well intentioned, but it makes the
term “best value” meaningless except as a generalized wish that
we should buy better value rather than worse value.
For lack of a better term, I will sometimes use the term “best
value” in the pre-rewrite sense, where it meant an explicit
willingness to spend more money to obtain something better than
minimally acceptable.
If a selection factor doesn’t have the potential to swing a
source selection when “all other things are equal” it shouldn’t
be a selection factor. If you can’t imagine a scenario where a
factor will decide the selection, get rid of the factor.
Eric
By
PWG on Friday, November 08, 2002 -
07:41 am:
Thanks all:
Great information here. I should clarify though; my Contracting
Office only treats procurements where "cost is significantly
more important" like LPTA. They do permit technical/cost
trade-off decisions with a best value procurement where cost and
technical are approximately equal. The difficulty we face with
the best value decisions is that our CO usually requires that we
use some type of mathematical formula to compute a definitive
measureable value associated with a technical advantage. That is
always difficult and sometimes impossible.
By
joel hoffman on Friday, November
08, 2002 - 10:13 am:
I feel sorry for you, because your CO obviously
doesn't understand "best value". It's not practical to equate a
dollar value with or designate a tangible dollar value against
every benefit, advantage or disadvantage. I can also understand
why that the CO treats a trade-off with price the most important
factor the same as an LPTA approach. happy sails! joel
By
Vern Edwards on Friday, November
08, 2002 - 10:35 am:
PWG:
You said: "The difficulty we face with the best value decisions
is that our CO usually requires that we use some type of
mathematical formula to compute a definitive measureable value
associated with a technical advantage. That is always difficult
and sometimes impossible."
Tell your contracting officer to read the last sentence of FAR §
15.308, which says: "Although the rationale for the selection
decision must be documented, that documentation need not
quantify the tradeoffs that led to the decision."
By
Anonymous on Saturday, November 09,
2002 - 10:04 am:
Use of some type of mathematical formula to compute a
definitive measurable value is not required. A mathematical
proof is sometimes not practical or even possible as noted. That
being said, I'm afraid too many use those points as excused for
not using such analysis at all.
Applying mathematical analysis with full recognition it may not
result in "a definitive measureable value" is useful as a sanity
check. I have seen it result in reconsideration of whether a
presumed improvement is really worth what we will pay. That is
particularly true when savings in one presumed, but actually low
yield, benefit can be applied to a more cost effective one.
One resource selection teams are characteristically lacking is
such mathematical support. Many agencies have the technical
resources. They just aren't on the panel. I'd propose that such
internal experts be assigned a support function. They would not
be on the selection team. They would be available to run the
math on specific issues and report to the team. All the
confidentiality requirements can be imposed without the full
time commitment being an obligation.
By
Anon on Wednesday, November 13,
2002 - 01:10 pm:
PWG, From a GAO decision posted today at the GAO
website (B-291092):
"...to be evaluated in terms of Offerors understanding of the
desired services, appropriate methodology, qualifications, and
experience with similar projects, cost proposal and any other
factors within the sole discretion of the FTC. Cost will be a
significant, but not necessarily the only, determining factor in
the decision. The FTC is not obligated to award a contract based
on the lowest cost proposed.
Solicitation at 15."
Even the CompGen doesn't support your KOs decision to treat a
"price is a significant evaluation factor" as presuming the
selection will be based on LPTA.
By
Eric Ottinger on Thursday, November
14, 2002 - 12:29 pm:
Anon,
The GAO also noted that this solicitation was identified as an
RFQ when it really was an RFP, and further, the solicitation was
“inartfully” worded.
Actually, this case is a good illustration of the reason why
protested source selections are a poor place to look for good
practices. The agency won, but it is clear that the agency could
have avoided this mess by writing with greater clarity.
In any case, the fact that GAO quotes something out of a
solicitation doesn’t mean that GAO intends to give the wording
an endorsement.
I’ve never used a LPTA or written a solicitation where price was
significantly more important than the technical factors. It
won’t do for us to hire low rent rocket scientists.
However, in fairness to PWG’s contracting office, I would think
that there should only be a subtle shade of difference between
an LPTA and an evaluation scheme where price is significantly
more important than the technical factors. (In the second case,
it might be possible to select an offer with a minor deficiency.
LPTA doesn’t allow this.)
Eric
By
Anon on Thursday, November 14, 2002
- 01:00 pm:
Well Eric, not everyone is buying R&D services or
dealing with cost reimbursement, in the past I've acquired R&D
and have done cost contracts, in those cases, the technical
approach was normally paramount. My point in citing that GAO is
that when price is more important than the nonprice factors, it
doesn't necessarily follow that one must award to the low priced
technically acceptable offer, which it seems PWG's KO believes.
With respect to a poorly written solicitation, in 24+ years I've
seen some real doozies and I lay the blame directly at the
supervisory contracting officer's feet, they should be doing a
thorough review of the specialist's work AND training their
specialists in the day-to-day activities that acquisition
entails. Where I'm at (and this peeves me to a degree, but it
does give me a level of job security) the supervisory
contracting officers pass that job to the office procurement
analyst for an independent review (that's me).
As for citing a GAO statement, I have found it very useful in my
career/current position when answering an agency level protest
to quote the Comptroller. That has almost always averted the
protest going to the next level.
By
Eric Ottinger on Thursday, November
14, 2002 - 01:32 pm:
Anon,
I think you are missing my point. The Comp. Gen. quoted the
evaluation scheme in the solicitation. The Comp. Gen. did not
endorse the evaluation scheme.
This is similar to the way people quote Shakespeare when they
are really quoting Richard III, Hotspur or Polonius. (For
instance, brevity is the soul of wit, but Polonius was a windbag
who should have taken his own good advice.) It is perfectly good
to quote the Comp. Gen., but we shouldn't assume that everything
quoted in a case is endorsed by the Comp. Gen.
If price is significantly more important than technical, I would
think it would be very difficult to justify a high cost/ high
technical selection. However, there would be a little more
flexibility than LPTA allows.
Eric
By
Anon on Thursday, November 14, 2002
- 01:35 pm:
Reread the part of the decision that I quoted in my
previous, The CompGen was not quoting the solicitation language,
read my extract in the full context of that paragraph.
By
Vern Edwards on Thursday, November
14, 2002 - 01:39 pm:
Eric:
You said" "[I]t might be possible to select an offer with a
minor deficiency. LPTA doesn't allow this.)"
I wonder whether a contracting officer could select an offer
with a deficiency of any kind, as that term is used in FAR.
FAR § 15.001 defines deficiency as follows:
"Deficiency is a material failure of a proposal to meet a
Government requirement or a combination of significant
weaknesses in a proposal that increases the risk of unsuccessful
contract performance to an unacceptable level."
Note that FAR does not distinguish levels of deficiencies, as it
does in the case of weaknesses. A deficiency is a deficiency.
There are two kinds of deficiencies: (1) a material failures to
meet a Government requirement and (2) an "unacceptable" level of
risk. Given the definition, I don't know on what bases we could
distinguish between "minor" deficiencies and other types.
I searched the GAO decisions and found 111 cases in which the
phrase "minor deficiency" or "minor deficiencies" occurs, but
only three of them were decided based on the post-Rewrite
version of FAR Part 15, in which the new definition of
deficiency took effect, and none of them seem pertinent to this
discussion.
The GAO has long and consistently held that failure to meet a
material requirement of a solicitation renders a proposal
unacceptable and its offeror ineligible for award. Since the FAR
definition says "material" failure, I think that all
deficiencies of the first type would render a proposal
unacceptable.
The rule with regard to deficiencies of the second type is
harder to determine. However, since the FAR definition uses the
phrase "to an unacceptable level," it very well may be that any
such deficiency will also make a proposal unacceptable and its
offeror ineligible for award.
I only point this out because I'm interested in how formal
definition and field usage differ and how those differences can
lead to problems. I realize that people use the term deficiency
to describe a variety of conditions and that they do not always
use it as defined by FAR. I'm not saying that you are wrong; but
I think your phrase raises an interesting question about usage
and effect.
Vern
By
Eric Ottinger on Thursday, November
14, 2002 - 02:11 pm:
Vern,
Many years ago I remember watching an experienced PCO expound on
the exact meaning of the term “deficiency.” I don’t remember
what he said, but I remember thinking that we are group that
makes a large investment in petty theological arguments over the
exact meaning of terms.
Looking at Part 15, I would say that one person might say "minor
deficiency" and another person might say "significant weakness"
and they might both be right. Part 15 doesn't draw a clear line
between weakness and deficiency.
I would say that you have a deficiency is you know that the
offeror is not going to meet your requirement 100%. You have a
“weakness” if you think there is a probability that the offeror
won’t meet the requirement 100%. Deficiency should be objective
and factual. However, several weaknesses may add up to a
deficiency.
You are correct regarding material requirements. However,
selection factors allow for more flexibility.
Let’s say for a hypothetical moving contract, one of my factors
is “Caliber of Personnel.” Let’s say Offeror A employs only
Eagle Scout, alter boys. Let’s say Offeror B employs only
paroled felons, recently out of drug rehab. Let’s say Offer B is
the only offer that fits within my budget and I must get the
furniture moved.
My point is that an imperative requirement matched with a
severely limited budget may be a perfectly legitimate reason to
select a proposal with a weakness.
Eric
By
formerfed2 on Thursday, November
14, 2002 - 02:36 pm:
Eric,
It doesn't matter if you call it deficiency or weakness. When
using Lowest Price Technically Acceptable, if the technical
evaluation can't make the proposal pass the minimum requirements
you shouldn't even be considering the price.
By
Eric Ottinger on Thursday, November
14, 2002 - 02:42 pm:
Anon,
I believe we are reading the following sentence differently:
“The FTC is not obligated to award a contract based on the
lowest cost proposed.”
Apparently, you are reading this as a conclusion reached by the
Comp. Gen.
I read this as simply a precis of the FTC’s language in the
solicitation. It was the FTC which stated that the government
would not be obligated to award a contract to the lowest cost
[acceptable proposal] proposed.
I don’t doubt that the Comp. Gen. agrees. But the proposition
that the agency might award to other than the low cost proposal
was never an issue in this protest.
Eric
By
Eric Ottinger on Thursday, November
14, 2002 - 02:57 pm:
Formerfed2,
I don't see a disagreement. Minimally acceptable sounds like
"minor weakness" to me. It isn't quite what you want, but you
can live with it.
Anything identified as a deficiency (albeit minor) would not be
something that you could select under LPTA.
Vern,
That's the reason why I chose the phrase "minor deficiency"
rather than "weakness." A weakness might be trivial. A
deficiency is not trivial, but you might be willing to live with
it if money is tight. Or you may be able to find a workaround.
Eric
By
Anon on Thursday, November 14, 2002
- 03:21 pm:
Vern, I didn't read it as a conclusion, rather I
simply saw it as an affirmation that in a best value acquisition
where price is considered more significant than nonprive
evaluation factors, it does not automatically construe that one
must award to the lowest priced acceptable offer.
By
Anon on Thursday, November 14, 2002
- 03:25 pm:
My oops, previous comment was directed to Eric, not
Vern
By
Eric Ottinger on Thursday, November
14, 2002 - 03:35 pm:
Anon,
I don't disagree with your affirmation. I am sure the Comp. Gen.
agrees. However, I wouldn't quote the Comp.Gen.'s authority. The
Comp. Gen. is merely quoting the language in FTC's solicitation.
Eric
By
Anon on Thursday, November 14, 2002
- 03:54 pm:
??? and in an agency protest you wouldn't cite GAO's
precedent cases in order to further support your (the agency's)
acquisition positions???
What is GAO doing in their bid protest decisions when they cite
previous bid protest decisions?
By
Eric Ottinger on Thursday, November
14, 2002 - 04:35 pm:
Anon,
Without naming names, you are going to win my second place prize
for carrying on an argument when there is no good reason.
Comp. Gen. decisions follow a consistent and rather stereotyped
pattern. First the Comp. Gen. provides some backgound including
a brief description of the requirement and key events leading up
to the solicitation.
Then the Comp. Gen. describes the solicitation with particular
attention to the selection factors.
Further in the sequence, the Comp. Gen describes the offers
submitted, the evaluation and the issues raised in the protest.
Generally, with regard to Court cases there is a distinction
between the really authoritative part of the decision which
addresses the issues raised in the case and “dicta” which is
usually conversational in character and off on a tangent. The
part that you cite doesn’t even rise to the level of dicta. It
is merely background.
When you do an independent review, you address all aspects of
the document that you are reviewing. Presumably, if you don’t
find a problem with some part of the document, the reader can
conclude that you consider this part of the document to be
minimally acceptable.
The Comp. Gen. addresses the issues raised by the protester. The
Comp. Gen. does not attempt to provide a comprehensive quality
check of the agency’s work. (However, the Comp. Gen. may address
other infelicities in passing. For instance, the Comp. Gen.
notes that this RFQ should have been an RFP.)
To be exact, I would expect the Comp. Gen. to address a really
fatal flaw, even if the protester doesn’t see the issue.
However, much that is obtuse, illogical or otherwise less than
excellent will be passed over without comment.
Once more – I don’t disagree with your affirmation.
Eric
By
Vern Edwards on Thursday, November
14, 2002 - 05:09 pm:
Eric:
When a regulation stipulates a definition of a term, I don't
consider it to be a "petty theological argument" to inquire into
the exact meaning of that term. Consider, for example, cost
or pricing data. There has been nothing either very petty or
very theological about all of the litigation about that term.
It seems to me that if you consider a "minor deficiency" to be
some type of weakness, then you are using the term deficiency
in a manner inconsistent with the definition in FAR, which says
that a deficiency is either (1) a "material failure" to meet a
government requirement or (2) a "combination of significant
weaknesses" that has the very particular effect of raising risk
to an "unacceptable level."
A couple of other responses to things you said today at 2:11
p.m.:
1. "Deficiency should be objective and factual." That is
inconsistent with the second type of deficiency -- a combination
of significant weaknesses that increase risk to an unacceptable
level -- because the level of risk is an inherently subjective
determination.
2. With regard to your hypothetical moving contract, for which
you selected a factor that you called "caliber of personnel,"
such a factor allows more flexibility because it is vague. Who
knows what it means, beyond that you are concerned about some
unspecified qualities of each offeror's proposed personnel?
"Caliber" tells us nothing about what qualities you are looking
for, how you will measure them, and what is their relative
importance.
3. "An imperative requirement matched with a severely limited
budget may be a perfectly legitimate reason to select a proposal
with a weakness." Well, I don't disagree, but I didn't ask you
about weaknesses, I asked about deficiencies. If you decide that
you are willing to accept a proposal that includes a "minor
deficiency," and if you are using the word deficiency as
it is defined in FAR, do you have to notify the other offerors
and give them a chance to propose on the same basis?
The regulation is either clear or it's not; we are using terms
either as defined by regulation or we aren't. I didn't ask you
the question in order to attack you. I asked because I am
generally interested in the significance, if any, of formal
(i.e., regulatory) definitions, and the potential consequences,
if any, of field usage that departs from formal definition. I
think that inquiry can make for an intellectually rewarding
discussion. However, I won't push you any further into any petty
theological arguments.
By
Eric Ottinger on Thursday, November
14, 2002 - 06:03 pm:
Vern,
I said, “Deficiency should be objective and factual. However,
several weaknesses may add up to a deficiency.” If that is
inconsistent with your last post, there is only a shade of
difference. And I don’t see any operational significance.
However, if you wish to correct that, to say that one definition
of deficiency in Part 15 should be objective and factual, while
the second definition allows more room for judgement, I would
not object.
A single small weakness by itself may be charaterized as an
anomaly. Several weaknesses are a pattern, and the pattern has
some objective significance independent of the individual
weaknesses.
(Just personal opinion folks. You won’t find this in the FAR.)
For instance, if an offeror has just one negative past
performance evaluation and the remainder are all excellent, I am
willing to bet the problem is the sorehead customer. If there
are several negative evaluations, I would reach a different
conclusion.
My opinion is just an opinion. Solicit a hundred opinions and
you have a survey. Thus, something judgmental becomes something
more or less objective and factual.
I said "deficiency" to make it clear that this was something we
couldn't select in an LPTA and "minor" to suggest that this is
something which we might be willing to live with, albeit with
some pain.
It struck me that we were getting into the whole vexed issue of
“standards,” which are neither material requirements nor
selection factors per se. And that is a long discussion.
Eric
By
Vern Edwards on Friday, November
15, 2002 - 12:17 pm:
Eric:
When you mention "standards," are you referring to the old Air
Force source selection term for the minimum performance or
compliance required to earn an acceptable (green) rating?
Vern
By
Eric Ottinger on Friday, November
15, 2002 - 12:39 pm:
Vern,
Yes, but my working definition is a little different. Standards
should be a common yardstick which all of the evaluators agree
to, which allows the team to produce a consistent evaluation
without impairing the independent judgment of the individual
evaluators.
For instance, a 5.0 for one skating judge should be more or less
the same as a 5.0 for another skating judge, because they have
explicitly or implicitly agreed on the same standards.
The old AF definition had a number of pitfalls. For one thing,
no one really intends to buy minimum.
Quantitative standards are nice but real world standards are
often more conceptual.
Eric |
