Lauri:

My recent response to you was just to answer your question about when you could receive oral presentations. I kept my opinions to myself, but since you've asked:

I don't like the idea of receiving oral presentations after the determination of the competitive range, either separate from or combined with discussions. In my opinion, the best use of oral presentations is to evaluate the offerors' understanding of the work that will be required under the prospective contract. I think that this should be done before deciding which offerors are in the competitive range.

However, agencies rightly worry about having to sit through too many oral presentations, and want to eliminate some first. In my opinion, the best way to do that is through a phased evaluation. For example, after receipt of proposals you compare the offerors based on price and eliminate any that are obviously not competitive. If there are still too many offerors, then you compare them on price and experience eliminate some more. If there are still too many, then you compare them on price and experience and past performance, and so on and so on. You work from the easiest factors to consider to the hardest.

In order for you to evaluate proposals in phases your RFP must describe the evaluation process and the factors that will apply to each phase.

If you would like, contact me at my e-mail address and I will send you some sample RFP language that I have. You'll have to be a little patient with me--I'm at a remote location right now and won't be able to answer you again until this weekend, but I will be glad to do so.

Vern

By Lauri Newkirk-Paggi on Thursday, May 4, 2000 - 11:03 am:

Mr. Edwards - Thank you so much for responding. This is the first time I've used Open Forum and am thrilled to have found it. Your answer was helpful but in reading other oral presentation discussions you've had in the past, has raised more questions. In your 'water-cooler' discussion dated 20 April 1998, you stated that "oral presentations is to judge the offeror's knowledge of the work - a test of the offeror know-how." I agree. You later said that "if the purpose of the oral presentation is to test the offeror know-how, then combining orals with discussions would require that you establish a competitive range and choose firms for negotiation BEFORE you've determined how well they understand the work. Why would you want to do that?" You went on to say, "If you don't need the information in the oral presentations in order to establish a competitive range, then perhaps you don't need it at all." I also agree with thoses statements. So, here is my question, in the response you gave me earlier you explained what I would need to explain in the solicitation if I decided to have orals after the competitive range determination. No mention of the statements above. It seems I'm getting two answers. If I had orals after the competitive range determination, what would my factor be when my understanding of the process is that we are testing the offeror's know-how. Could I please get you to explain this further to me? Thank-you.

By Vern Edwards on Monday, May 1, 2000 - 05:19 pm:

FAR 15.102(a) clearly states that oral presentations "may occur at any time in the acquisition process... ." If you read GAO decisions, several describe procurements in which agencies received oral presentations after the establishment of the competitive range. In one case an agency received oral presentations after the submission of written best and final offers.

If you decide to receive oral presentations after the establishment of the competitive range, your RFP must explain to prospective offerors that: (a) offerors in the competitive range must make oral presentations, and (b) you will evaluate offerors in the competitive range on the basis of an additional evaluation factor that will be introduced after the initial evaluation of proposals, e.g., understanding of the work as demonstrated during the oral presentation.

By Lauri Newkirk-Paggi on Monday, May 1, 2000 - 04:50 pm:

My question is when should I have Oral Presentations? I thought I'd read that you can have them any time...including after determination of the competitive range. Is that true and if so, wouldn't you need to evaluate that portion of the proposal in order to determine the competitive range? I have a very large list of interested offerors and I'd like to use oral presentations, but, with such a large interest, I'm afraid we'd be having oral presentations for a month! Please advise.

By Chuck Solloway on Wednesday, January 26, 2000 - 10:04 pm:

Raymond,

I would like to respond to your inquiry. However, I would first have to know:

a. What general areas are to be covered in the orals?
b. Do you plan to make award w/o discussions?
c. Are the orals addressing material not covered in a written proposal or they being used to further explain written proposals, or both?
d. If a competitive range is going to be established, will orals be held before or after the establishment of the competitive range?

The answers to these questions are necessary to address how you can keep "clarifications" (or communications) from becoming discussions. Or, perhaps, to address why it may not make any difference.

In the event you are not aware of it, there is a wealth of information on oral presentations already posted on the "water cooler". A lot of it is very interesting.

You may also want to contact the Army Materiel Command as a part of your research. I understand that they have received complaints from contractors regarding the expense of oral presentations and they are collecting data to determine if the complaints are valid.

By Fran Cass on Wednesday, January 19, 2000 - 04:34 pm:

Raymond:

Since your RFP is on the street, this advice may come too late -- but to ensure our clarifications don't become discussions, I structure the presentation agenda to include a rather large break (45 - 60 min) prior to the Q&A period. During the break I ask the tech eval to write down the text of the "clarifications" they need. We work on it together to make sure it doesn't accidently elicit a change to the proposal. I coach the evaluators to identify the specific slide which prompted the question or the portion of the RFP. Then we strive to construct a question which asks the offeror to "elaborate on x, y or z".

One issue that routinely causes us a lot of grief is scheduling the order of the presentations. All of the offerors want to go last -- and of course they can't. The fairest process we have is a "lot" drawing within our office (offeror's aren't invited because we don't want to reveal the source of their competition.)

Regarding your request for recommendations on worksheets -- since you stated you have a complex procurement, I'd make sure there was lots of room for note taking. The first oral presentation I processed was for a highly complex telecommunications procurement. It became very clear to me why our counsel so strongly recommended against orals -- it was difficult to come to conclusions. The eval team had too many questions. We ultimately opened discussions.

Regarding video taping -- we have done as Vern recommended. We used a stationary camera in the corner of the room positioned to capture the whole front of the presentation area (so there wasn't a lot of fiddling.) The presenting offerors did not appear very intimidated by it. I recommend the video tape as a much better record than the audio cassette tape. It is difficult to identify speakers in our audio cassette tape records -- particularly for the Q&A portions. [To be fair -- it was partly because the microphone was inadequate.]

With regard to your premise that oral presentations reduce processing time -- we find processing times are really not a function of the oral/written process. In our field contracting environment the speed of the process is far more dependent upon how much time per week the TEB can commit to the evaluation process (as opposed to the day-to-day demands of their "real" job) and also the workload demands on the negotiator. The experience level of the tech evaluators can be important -- folks that have participated in previous technical evaluations are usually able to more quickly through the process. And as I said -- we are in the field contracting system, so we generally don't have an experienced pool of tech evaluators.

By Vern Edwards on Wednesday, January 12, 2000 - 09:59 pm:

Since your RFP is already on the street our suggestions will be of limited usefulness; however, here are some thoughts:

When an agency asks for oral presentations it is asking for submission of proposal information in two phases. The first phase is the written submission of information; the second phase is the oral submission of information.

Oral presentations are initial submissions of proposal information. They are not exchanges about proposal information; they are not clarifications, communications, or discussions.

In order to avoid the inadvertent conduct of clarifications, communications, or discussions, do not address information that was previously submitted in writing. Do not ask for clarification of information that was submitted previously in writing. Do not bargain with offerors or let them change the terms of their offers as previously submitted in writing. Conduct those kinds of exchanges separately, as clarifications, communications, or discussions in accordance with FAR 15.306. Hopefully, the proposal preparation instructions in your RFP make the distinction between oral presentations and exchanges after the receipt of proposals clear to all prospective offerors.

If you are going to videotape the presentations, do so as unobtrusively as possible. Don't make a major production out of it. Set a video camera in a corner of the room so that it can record the presenters, not the government personnel. You want to make a record of what the presenters see and hear, not a comprehensive documentary about the session. Do a test before the first presentation to make sure that existing lighting is adequate and that you are able to capture all that is said.

By Vern Edwards on Wednesday, January 12, 2000 - 06:30 pm:

I can tell you that the first part of your premise -- that "a majority " of contract specialists use oral presentations -- is almost certainly untrue.

Based on FPDS data, there were about 20,000 source selections during FY98. Although no one knows for sure, a casual review of RFPs and other information suggests that most source selections do not include oral presentations.

I would like to think that the second part of your premise is true -- that contract specialists who use oral presentations are able to shorten the acquisition cycle. However, you will have a hard time getting data which confirm or contradict your assertion. Agencies don't seem to have good data about the relative cycle times of procurements that use oral presentations compared to those that do not.

The third part of your premise -- that oral presentations do not create any (serious) problems -- appears to be true. I base that claim on the relative lack of protests about oral presentations.

If you still claim that a majority of contract specialists use oral presentations, you are going to have a difficult time proving it. The data simply are not available.

By Raymond W. Bouford on Wednesday, January 12, 2000 - 05:34 pm:

I am a contract specialist at the Dept of State and will have to control oral presentations on a large fairly complex contract. The RFP is out to the vendors and it outlines the process we are to use well. I am looking for some practical, in the room experience. What type of score sheets work best, how can you ensure the clarifications don't become discussions, best way to organize video taping, etc. Your advice is appreciated.

In addition, I am doing orginal research on the subject for my grad project. My premise is: "A majority of contract specialists use oral presentations to shorten the aquisition cycle and have few problems doing it".

I would appreciate your opinions on this statement. I would also like your e-mail address so I may send you a very short research survey.