This question would pertain to DOD solicitations during the evaluation phase.

When calling upon cited references to give information on someone they have done business with, is it appropriate to ask them the typical past performance survey questions via email?

I looked around before asking this question; I could find no guidance.

By Dave Barnett on Thursday, January 24, 2002 - 02:17 pm:

I see nothing prohibiting one from using e-mail as the mode of communication, what is the particular concern...security? Mail can get lost, misdirected, etc. Someone could answer the phone and take a message, leave it on someone's desk where passersby could see it. Faxes can get picked up by other than the addressee.

Offhand I see nothing wrong with using e-mail, why has this issue arisen, what am I missing?

By formerfed on Thursday, January 24, 2002 - 03:43 pm:

I've frequently received email questionnaires to fill out on a given company. I've also seen lots more. There is nothing inheritantly wrong with this. The big problem is you have turned a poentially very beneficial source selection tool into a mechnical, follow the numbers, process.

Obtaining valuable past performation information is much like conducting a job interview. The true value is not so much in the words used for responding, it's the other aspecst of communications that provide the clues for what the next question might be. Someone need to listen, digest what's said, think out a strategy to get at the persons's true thoughts and feelings, ask the probing questions, and really get at the maet of how they think the contractor performed. None of this happens with an email.

By anon2 on Thursday, January 24, 2002 - 06:50 pm:

Formerfed, the same really applies to regular mail. To get at that real information it is almost always necessary to conduct a follow-up interview of some sort. I agree it is valuable to get that extra sense of things.

The main disadvantage of e-mail is the tendency to sit down and zap something out. Even an old e-mail hand, as I am, will tend to give more attention to something hard copy that is going into an envelope.

Even more interesting to me personally is how I will often find an error on a printed copy of something I've done electronically with considerable care. That may be a generational thing since I grew up with and used paper most of my life.

For most things I never go to paper. Before sending by e-mail or direct computer to FAX I'll run important stuff through spell check and do an extra screen proofing. For really important things I still generally print a final and do a proof read on paper.

I've seen some dashed off examples and they weren't worth the paper or time we took to send them out. Those say little for the individual or the agency responding so poorly (One might put that in a type of past performance file!). For a past performance I personally would do a paper copy even if I were eventually sending it by e-mail for those reasons. Personally, I think past performance, the requesting agency and the companies (good or bad) deserve the time and care that takes.

By joel hoffman on Friday, January 25, 2002 - 08:38 am:

Anonymous, in lieu of relying on a written questionaire, have you ever thought about using a tool which seems to be obsolete or feared by many Government evaluators? It's called "the telephone". Or have you considered combining the e-mail with a follow-up phone interview? I often call the receiver, as soon as I receive back an e-mail receipt acknowledgement. That way, I can at least introduce the e-mail topic to the respondent, which is an effective way to reduce the chance that the reader will simply ignore the e-mail.

I found oral interview techniques to be much more effective than written responses. Respondents generally will be honest and say more in an interview than they do in writing - if they respond at all. I simply take notes on my questionaire and submit them back to the rest of the selection board.

Have you ever tried oral interviews? If so, were they useful?

happy sails! joel

By Dave Barnett on Friday, January 25, 2002 - 10:19 am:

Joel, I agree, the telephone is what I've used when doing past performance inquiries. It affords flexibility and you can pick up nuances with the interviewee. However, I've received questionaires by mail from some DoD contracting offices, so evidently, some contracting offices feel written documents are needed (what, they don't trust their contract specialists/officers to properly document their phonecons?)

When using the phone, I still have a list of questions/topics to be addressed so it's fairly easy to annotate the list and therefore have a phone call record.

Speaking of the mailed questionaires, I received one that was so extensive and (for me) labor intensive, I round filed the dang thing. I would therefore advise those who use mailed questionaires to remember the acronym KISS - Keep It Simple, Stupid

By anon2 on Friday, January 25, 2002 - 12:13 pm:

Not only KISS, don't make it stupid. If you provide check boxes and a comment line you will likely get checks and little comment. Ask a leading question and invite a short essay question response.

Q: "Did the company manage start-up staffing effectively?"

A: "Yes"

You've learned they did with little in the way of discrimination factors to separate offerors. If everybody is "Yes" you really know very little on this issue.

Q: "How well did the company manage start-up staffing and overcome initial staffing problems?"

A: You might get "Very well" but you also might get a short description revealing a company with good policy, real thinking and management or one blundering about that will be of real value in making a decision.

By anonymous on Friday, January 25, 2002 - 12:32 pm:

Thanks to all for your input, it is most helpful.

However, I dare say most of you were a little hasty in presuming I am unfamiliar with, or have never used the telephone as an interviewing technique.

I too agree that the telephone is a most accomodating method of obtaining the sought after information. However, on a recent proposal evaluation I had a devil of a time reaching someone by telephone due to their work situation. This was someone who I was most eager to obtain an opinion from (as revealed by the nature of their title as submitted by the offeror). I happened to be a little anxious about the email method, as I realize it may reside on one or more servers forever, whereas a phone conversation may be recorded in a locked up file.

Last anonymous:

Was the fellow you wanted to contact on the move and using a Personal Digital Assistant that could track e-mail?

By anon2 on Friday, January 25, 2002 - 01:01 pm:

Ah yes! Those of us proposing phone interviews are obsolete fossils! We forget that government phones are no longer a means of communication. They are a means of "putting off" callers. During my last days in government I was amazed at how calling someone supposedly the POC was an adventure into the world of voice black holes! Of course I saw some of this in my own area. Some people, not engaged in anything urgent, would simply let voice mail answer all calls.

What was worse, the virtual eliminatin of real secretaries and phone operators in even fairly large organizations, led to unanswered calls to the person's organizational phone to inquire if they still lived -- much less try to stimulate a response. Call the Division or Department and what? Voice mail again! What happened to the individual in Congress that was pushing for a legal requirement that at least organizational phones be answered by a human?

Here is an amusing little story from a recent experience with a commercial firm I've done business with for half a decade. The 4464 extension was given me by the corporate President's assistant as a direct line to the person who can solve this problem.

"You have been forwarded to a voice mail system. However the person at this number, 4464, does not subscribe to this service. Good bye."

Needless to say I am trying to terminate my relationship, but that itself seems difficult as my messages aren't returned and phones only give voice mail. One of these days I'll perhaps get an answer to my snail or e-mail. I'm not holding my breath.

By Dave Barnett on Friday, January 25, 2002 - 01:14 pm:

Anon2, the questionaire I received was 23 pages long for Chist sakes! Sorry, but I work for my agency not that particular DoD contracting office. It would have taken me a day or two to pull files and interview the varied COTRs to provide the specifics they were looking for.

I have other priorities/customers that are more important.

By anonymous on Friday, January 25, 2002 - 02:32 pm:

In reply to Bob Antonio's;

"Was the fellow you wanted to contact on the move and using a Personal Digital Assistant that could track e-mail?"

I have no idea, but I was hoping he would enter his email and respond at a time that I was unavailable too. You see, he resides in Hawaii, and is a college professor.

By joel hoffman on Friday, January 25, 2002 - 03:44 pm:

Original Anon, you're correct that I probably concluded that you might not use phone interviews. Sorry.

I agree with you that using e-mail can be useful when you have problems reaching someone on the phone. I, as a fossil, would prefer the oral interview with e-mail as a back-up. Actually, I'd prefer sending the e-mail questionaire with a request to confirm when I could call over simply sending the e-mail with a request to return the e-mail. happy sails! joel

Anyway, I guess your original question was answered. YEP, e-mail is okay, not formally restricted.

By anon2 on Friday, January 25, 2002 - 05:26 pm:

Dave, someone that has to make a 23 page past performance questionnaire probably hasn't a clue on how to prioritize. I'd also guess they had not examined their own "contractor performance" requirements to distill discriminators needed to make a judgment. You remember the old story about the amazing farm hand? The one about the hand that did any manual job in record time, who broke down on something like sorting apples to big, medium and small. It was the decisions that were killing him.

I expect your 23 page people had similar problems. They probably scored past performance too. I wouldn't be surprised to see numerics at two decimal places with final spread at one decimal place. In the end they probably threw a dart at a board. Can't you see it? They more or less read all 23 pages and distilled the "scores" to a final range of ±.4 points. Then they picked a "winner" and went on to further foolishness. Do they ever wonder why there is no net improvement in contractor choice? Probably.

I guess I'm just ranting. The lack of thoughtful and effective procedure just irritates me. It is not as if an R&D program were needed to find an effective solution either! The subject has been beaten to death right here and in the old Water Cooler. It is covered in various publications and advisory sites. Some is just plain common sense.