Eric Ottinger on
Thursday, June 8, 2000 - 07:37 pm:
Apologies for any misunderstanding. I wasn't thinking of you
when I wrote "we."
It was clear that you were comfortable with the DoD Guide.
By joel hoffman on Thursday,
June 8, 2000 - 06:17 pm:
Let me add that I don't have any
problems with the DOD Guide and its approaches to collecting and
evaluating past performance data. See also page seven, where
distinctions between "extent" of experience and "past
performance" are very clearly pointed out. Happy Sails!
Eric Ottinger on
Thursday, June 8, 2000 - 06:02 pm:
Here is the site for the DoD Guide. The answer to your question
is on Page 9.
To answer a few question which you may or may not have intended
Q: Can this evaluation of the relevancy be judgmental?
A: Yes. Note that the Guide provides a broad rather than a
restrictive definition of “relevancy.” But you should have some
reasonable basis for your evaluation.
Q: Is there any way to select a small contractor who does good
work, but has not done exactly the same work before?
A: Yes. The offeror may give you good reasons to think that the
performance risk is low. The offeror may hire key personnel (or
a subcontractor) with solid experience doing exactly the work
that you have in mind.
(It isn’t unheard of for the incumbent’s key staff to jump ship
and go with the challenger. In a case like this, who really has
the relevant experience? The corporate entity or the team of key
Q: Do you have to have separate (sub)factors for “Experience”
A: That is my personal preference, but you can evaluate both
under the same factor if you are being pressed to use the
minimum number of selection factors.
I doubt we will be having the same arguments because we have at
least ten times as much case law at this point. I expect we will
find that the case law supports the advice in the DoD Guide.
bob antonio on
Thursday, June 8, 2000 - 11:51 am:
I agree. No harm was done to the protester. The offeror with the
relevant past experience received credit for it.
Ramon Jackson on Thursday, June 8, 2000 - 10:36 am:
Interesting reference. It confirms again that a reasonable, well
defined basis for evaluating relevance and nature of past
I also found it interesting Navy took the option not to rate the
sum of the parts as relevant and poor, but rated it not relevant
and neutral. That seems a sane course of action that avoids a
great deal of bickering over qualitative aspects of something of
questionable relevance. Anyone involved in very large, complex
projects knows the sum of the parts is not the whole as many
fail on integration at both technical and management levels.
It seems to me that part of Bob M's problem, along with anyone
else planning such an exercise, is to carefully scope the
relevance envelope so that there is clarity in instructions,
procedure and finally in justification for placing an offeror
into the no relevant performance and neutral rating category.
Lack of thinking upon realistic relevance boundaries, rooted in
project requirements and needs, and defendable in the end result
tends to lead to much bickering, protest and muddy waters upon
review. When selectors try to defend what is really irrelevant
performance as "poor" it not only muddies those waters. It
creates a more offensive outcome and pours fuel upon protest
bob antonio on
Thursday, June 8, 2000 - 09:29 am:
A recent decision dealing with
on Thursday, June 8, 2000 - 09:18 am:
What makes past performance relevant to future performance is
the nature and scope of the work. In your case it seems likely
that there would be significant differences in fabrication and
assembly methods. The differences between manufacturing
aluminium antenna towers and steel trash cans are likely to be
so great that I can only wonder why you doubt that you could
You are simply wrong to think that you cannot consider
differences in relevancy when evaluating past performance. It is
well-established in GAO decisions that you can.
Suppose that you are going to choose the contractor on the basis
of only past performance and price and that you are comparing
your trash can maker to another vendor who has extensive
experience making aluminium antenna towers. Suppose further that
both have "excellent" reputations with their customers. Finally,
suppose that the trash can vendor has proposed a lower price for
the towers than the experienced antenna maker. You could
consider the aluminium tower maker's excellent past performance
more favorably than the trash can maker's excellent past
performance because it is more relevant to your procurment, and
you could award to the higher priced offeror, assuming that you
consider its price to be fair and reasonable and the difference
in price to be worth paying in light of the difference in past
There wouldn't be anything controversial about such a decision.
hoffman on Wednesday, June 7, 2000 - 05:21 pm:
At the risk of repetition, we
have covered this subject previously and there will be the same
arguments again from some persons on this Web Site. The
following is my opinion and is not contradicted in the FAR or
the "DOD Guide for Collection and Use of Past Performance
Information". I don't know of any Protests, sustained for
following my practice.
"Past Performance", as defined in the FAR, concerns the QUALITY
aspect of a contractor's recent experience. How well did the
Contractor perform on recent, relevant projects. The law and the
FAR state that we must rate an offeror without any relevant PAST
PERFORMANCE record "nuetrally" that is an unknown risk - neither
negative or positive.
The OTHER, distinct aspect of "recent, relevant experience" is
the "EXTENT" of such recent experience, which can be related to
your project through corporate experience or experience by the
key personnel involved in your project. You CAN make a separate
assessment of "extent of recent, relevant experience" and rate
the firm, accordingly. AMEN. Happy Sails! Joel
Ramon Jackson on Wednesday, June 7, 2000 - 04:02 pm:
Whatever the convention, I think
it is important to separate past performance into logical
components. Not doing so can lead to muddy and nearly useless
First, contract performance should be separated from technical
performance, whether building something or cleaning floors. They
are two entirely different things and should not be confused.
Take your towers. You may have the people who know more about
such towers than anyone in the world. They would do an excellent
job. One problem, they have an absolutely horrible business
staff and can't ever seem to do a job on time in cost. Schedules
mean nothing, the guys will do it when they get back from
fishing, they forget to order things and stand around waiting --
but one of these days you will sure get a quality tower. In one
bin and with one set of questions rate how well the offerer
tends to perform on these business matters.
You also don't want to find you have the most professional, even
clairvoyant, contract management team who can predict schedule
and costs to an astounding degree while having a reputation for
meeting cost, schedule and all those other administrative things
with some sloppy and corner cutting work.
When you lump these things you may tend to obscure real problems
in either. Construct your PP evaluation and analysis to drive
out each so that not only can you pretty much eliminate someone
truly failing on either, but can also drive out what you might
expect to be the bumps in the winner's performance and be
forewarned. Then you'll be in the position to address those
potential bumps on day one, not after the bounce.
I wholly agree with Kennedy on the difference in trash cans and
tower work. Aside from materials used the safety factors are
completely different. I believe your technical staff needs to
take more than a few moments to determine the envelope of what
is relevant work and also be open to a provision for letting an
offeror defend relevance. In some fields there can be surprising
similarities that are not obvious to even those experienced in
Kennedy How on
Wednesday, June 7, 2000 - 01:47 pm:
Actually, relevancy IS
important; in your case, there is a big difference in welding an
aluminum antenna tower and fabricating a 50gal trash can.
Speaking from personal experience, welding aluminum takes a
different skill than welding steel.
My feeling happens to be exactly the opposite than yours is, in
this area. Without actually researching the FAR/Regulations, my
feeling is that the farther away from the kind of work you are
contracting for, the lower you can rate him. I might even
suggest using the SIC codes as a potential guide to relevance.
(Note I said potential, it's only to give you some idea as to
classifying relevant experience.)
By BobM on Wednesday, June 7,
2000 - 01:29 pm:
I got such good comprehensive
answers to my last question, please indulge me by answering one
more (by the way, I will ask for missing info in future).
When asking for past performance data from a vendor we ask for
"relevant" past performance. In my mind, if I am contracting for
antenna towers fabricated and welded using aluminum, I should be
able to rate lower (or not even review) an offerer who's past
performance is only in the fabrication of small trash cans and
other small-sized implements out of carbon steel metal.
However, something tells me I am incorrect. If the vendor who
made the trash cans always performed very well in that, then I
can neither screen him out nor rate him lower because he has
never worked with and welded large aluminum structures. As long
as his quality is good and customers love him, he's in as far as
past performance goes.
If that assumption is true, then is there a way that I can rate
the relevancy of his work in order to place him properly? All
other things being equal he should rate lower than an antenna
tower manufacturer. I have a feeling there should be another
factor called experience.
Bear in mind that most of my contracts are from 50K to 2M,
commercial equipment, using SAP and the FAR 13.5 provision. I
usually do not use points