Has anyone done a source selection in which they asked the offerors to make an oral presentation about a sample task scenario, but didn't give the offerors the sample task scenario until the day of the oral presentation?

A member of our team says the rules say you have to put the sample task scenario in the RFP. But other members say that would just let the offerors hire consultants and rehearse their answers.

By Eric Ottinger on Tuesday, November 28, 2000 - 11:35 am:

Anon,

Let me ask a more general question.

Q: Why do so many of us have the idea that we are required to provide discussion questions to the offerors in advance?

A: I don’t know why. I think many of us are afraid of anything that is real time, spontaneous or unscripted.

If your sample task is complex and requires some research and thinking, you may not get the best responses if you don’t give the offerors some time.

On the other hand, if it makes sense to do your sample task as a kind of pop quiz, why don’t you. I presume you want to see how the offeror’s key people respond in real time.

As for this consultant bugaboo, I always thought that the offeror, who demonstrates that he knows which consultant to call, has demonstrated something important.

It is of course, a mercy and a cost savings all the way around, if you don’t give your offerors too much time to suffer.

In short, I believe the rules are to treat everyone consistently, adapt to the circumstances and do what makes sense.

Any particular source selection team will have several people who can tell you what the “rules” are, based on what they have done or what they have been told in the past. You need to tell them, politely but very firmly, that only the PCO is responsible for the “rules.”

Eric

By Anonymous on Tuesday, November 28, 2000 - 11:51 am:

Eric,

Thanks. I agree with you. But I've got somebody saying that we can't do it and I've been asked to produce an example of an agency that has done it.

I have searched the GAO decisions but cannot find an example of a pop quiz, etc.

So if anyone has done a scenario type oral presentation in which they waited to give the offerors the scenario on the day of the presentation I would appreciate hearing from them.

By Eric Ottinger on Tuesday, November 28, 2000 - 01:02 pm:

Anon,

I found this in the Deskbook under Ask A Professor:

"Oral Presentations
Posted to on 2/9/98 by Peter Zawoiski
The Scenario

The Question
1. Are pop quiz questions asked during oral presentations rated? Is so, how are they rated? 2. How is company "A's" proprietary rights protected when company "B" requests a video copy of company "A's" oral presentation under FOIA?
Peter: This is in reply to your questions concerning pop quizes during orals and release of video tape presentations to third parties. Regarding pop quizes, there are no prohibitions against using pop quizes (or responding to sample tasks or problems). You should identify this requirement in section L of the solicitation and the evaluation criteria should be part of your Source Selection Plan and reflected in Section M. Regarding release of video tapes to third parties, the tapes should be considered "source selection sensitive" and not releasable under FOIA (exception four). The "rating" system used for evaluating proposals, including pop quizzes, is up to you. As always, the important issue is the strengths, weaknesses and risks that support the evaluation, not whether you use colors, numbers, etc."

Personal Opinion: Notwithstanding the direction to "identify ... in Section L," I would not have any qualms about asking a few hypothetical "What would you do if..." questions in discussions, (as long as everyone is treated equally, etc.).

Eric

By joel hoffman on Tuesday, November 28, 2000 - 04:13 pm:

Anon, I always get a kick out of those who strongly argue that "you can't do that", without being able to provide an authoritative source. In something like this scenario, I'd make them justify "why not?" in order to stop you. Happy Sails! joel

By Eric Ottinger on Tuesday, November 28, 2000 - 05:10 pm:

Anon,

A couple of afterthoughts.

You need to have a good idea how your pop quiz “sample tasks” are going to tie into your selection factors. Free floating observations which don’t really fit in the evaluation scheme are obviously not a good idea.

You are probably asking more than one question, addressing more than one selection factor. This is going to drive some purists nuts. Specifically, you want to see how well the offeror UNDERSTANDS THE PROBLEM and you want to see how well the offeror’s KEY PERSONS handle the problem. (Plus, you want to see how well the team members, including key managers, prime contractor and subcontractors, work together.)

I would probably allow the offeror as much time as he wants to caucus and get his answer together. (And I would make a note if it took too long.)

None of this should stop you from doing what makes sense. Just make sure that you have thought these issues through.

I think the Comp. Gen. has come to identify a “Sample Task” with a sample task order. They see a “sample task” as a little contract, and they want to see it priced and they want you to consider the sample task price in the cost evaluation. To avoid confusion, I might want to use a different terminology. I might say “sample problem” or “question for discussion.”

The choice between a “pop quiz” verbal response and a more formal written response is not an either/or. I might ask for an impromptu verbal response in discussions, then ask the offeror to follow-up with a written task plan.

Eric

By Anonymous on Wednesday, November 29, 2000 - 09:07 pm:

Eric,

Thanks for the Ask A Professor ref. Very helpful.