I would like to thank everyone for their responses and information. This was my first time participating in the Open Forum and I found it to be very helpful. Just a few points to add to the discussion:

1. Vern, I understand that I used the term best value rather loosely and I guess technically it may be incorrect. But here's my problem with the language. "Competitive Negotiation," which by law may be the more technically correct title for the broad catetory of non-low-bid projects, can be a misnomer since many of these solicitations do not involve any "negotiation" whatsoever. It seems that there are a lot of contracts awarded under a best value/trade-off metod that never involve any negotiations.

2. Also, isn't true that some agencies acutally refer to "trade-off" solicitations as "best value." I think using the best value title has benefits, since it is a term of art, at least to some degree and, to some degree, it is known in the industry. A number of state procurement systems, for example, specifically use the term "Best Value" meaning an alternative to sealed bidding, where other issues, such as qualifications, performance, warranties and schedule, are considered in the decision- making process.

3. I guess a lot of this is semantics, but the different terms do create confusion. I also understand that we have to live with the law now, and that because of the FAR language in place, Vern is correct. At any rate, if the FAR ever gets another re-write, it would be helpful to clear this up. By the way, Vern, you should know that you are very highly regarded by many contracting officers across the country, your name seems to be known everywhere.

4. I understand that the trade off or BVC negotiated process takes longer, requires more resources and could present real problems when not properly used. But I also think we may be heading toward a system where low-bid is used more infrequently, mostly for small and non-complex projects, and I think this is a good thing. Even if performance and qualifications count for only 10% of the overall score and price is at 90, at least there is some serious performance incentives built in.

5. From working with and talking to a number of agencies, it seems that there has been a lot of dissatisfaction with the low-bid system, not just with federal, but with state and local contracting agencies as well.

6. One big problem is that agencies seem very reluctant to disqualify a contractor as non-responsible, even though they have substantial discretion to do so for good cause. As a result, performance evaluations become essentially meaningless. I've seen this with both federal and state agencies. With best value, or trade-off procurement, these evaluations become critical.

7. So, to me, it seems agencies should be striving to build the most efficient, most fair BVC systems possible. On smaller and mid size projcts, for example, could evaluation factors be more often limited to a few factors, and proposals be evaluated in a few weeks time. Interestingly, state governments are becoming interested in BVC mentods, both for construction and non- construction, and federal and state could assist each other by sharing information in this area. Does anyone know of any forums for such information-sharing?

8. A related question I have on this subject is whether most of the trade-off solicitations, apply a type of rating formula, where price is scored, then non-price factors are scored, and the offeror with the highest overall score wins. I've seen this in some contracting programs and I am curious as to whether it is very common,sometimes used, fairly common, or rare?

I'd also like to say to thanks to Vern, Joel Hoffman, Bob Antonio, JET, Brian Fisher and Eric Ottinger for all of the leads and information sources. I'm going to try to follow up on these and I'll be glad to share the results if you are interested. It seems that the data showing that a majority of federal construction is going out under the trade-off/best value, negotiated approach will not be difficult to document through the Federal Procurement Data center.

I'm somewhat suprised that there is more data on the cost-effectiveness issue, because while it is hard to measure, it is a critical issue. I just saw one OFPP report that mentioned a $10 million saving produced by using BVC methods in one contracting program, but I don't know how well this is backed-up. I can forward this if anyone is interested; I'd like to see any other such reports or claims.

By joel hoffman on Thursday, June 29, 2000 - 11:03 am:

Bob, as usual, your research and source material are excellent - thanks!

Vern, regarding your comment concerning Gerry's point #21 (construction contract performance evaluations), I mostly agree with you. Prior to the widespread use of competiviely negotiated procurements and prior to the current attempts to effectively use past performance information, the COE performance evaluations were almost a joke. I can't speak for other agencies or services.

The database was mostly used by CO's in making responsibility determinations. Contracting (rightly so, perhaps) often did not make the fruitless effort to declare the low bidder non-responsive. Why? They usually were unsuccessful. SBA routinely issued a certificate of competency, unless the bidder had previously defaulted

The evaluation forms were too broad - there were only 4 or 5 broad performance areas to evaluate; it took a superhuman effort to rate a contractor unsuccessful with no apparent benefit to the evaluating office. The field office still ended up with the same contractor - who now was ticked off because the COE attempted to declare them non-responsive - and whom was jubilant because they "beat the system" to get the contract; etc. As one should see, it was a vicious cycle.

The newer, DD Form 2626 has 33 subareas. These suareas can be more easily, subjectively rated as "outstanding", "above average", "satisfactory", "marginal" and "unsatisfactory". The evaluator can now fairly evalaute the subareas to show where a contractor is strong or weak, without having to resort to an overall "unsatisfactory" rating.

Source selection boards can better use these evalauations in their COMPARATIVE analysis of past performance in making tradeoff decisions.

I believe that field offices now realize that their evaluations are being taken seriously. I also believe that the quality and accuracy of the evaluations are improving.

I have been stressing this to my Design-Build classes each year, since 1996. Hopefully the message will get out. We have 200-250 students per year receiving the message.

Happy Sails!

By Vern Edwards on Wednesday, June 28, 2000 - 12:50 pm:

Gerry:

Before I respond to your questions you should know that the meaning of "best value" has changed. Before September 30, 1997, it was merely a popular name for the approach to conducting competitive negotiations in which the Government reserved the right to award the contract to other than the offeror who submitted the lowest-priced, technically-acceptable offer. Today, that process is called the "tradeoff process." See FAR 15.101-1. The term "best value" is now defined in FAR 2.101 and no longer refers to a specific contracting procedure. As presently defined, an agency can use sealed bidding to obtain "best value."

Now in answer to your questions:

(a) To the best of my knowledge, no one knows the percentage of contract actions or dollars that were awarded using the tradeoff process ("best value"). The Government keeps statistics on how many actions and dollars are awarded using sealed bidding and competitive negotiation, but the competitive negotiation statistics include both tradeoff process actions and lowest-price-technically-acceptable process actions.

(b) I know of no reliable study based on empirical evidence that demonstrates that the tradeoff process ("best value") has produced better contract performance outcomes than sealed bidding or lowest-priced-technically-acceptable competitive negotiations. Personally, I believe that it has and does, but I cannot prove it.

Be wary of Government agency claims to the effect that "best value" has produced superior results. Such claims are usually based on anecdotal evidence and are often made to prove the wisdom of some course of action. Such claims may be true, but they often are not verifiably true.

In response to some of your comments:

The shift from sealed bid to negotiated construction contracts began long before 1997 and in one period during the late 1980s and early 1990s it was quite controversial. The U.S. Postal Service was a leader in using competitive negotiation to buy construction, something that it undertook as part of the rewrite of its contracting rules in the mid-1980s. The use of competitive negotiation for construction seems to be less controversial today. The Association of General Contractors may be able to provide you with more information. As I recall, they were a leading opponent of the change.

Comment 6: I was a construction contracting officer and I agree that switching to competitive negotiation from sealed bidding has often been a good thing. However, competitive negotiation is usually procedurally more costly and time-consuming than sealed bidding, and sealed bidding still works fine for many jobs.

One important strength of tradeoff process approach to competitive negotiation is that it enables agencies to elude the Small Business Administration's certificate of competency program, which enabled some incompetent contractors to win sealed bid contracts over the objections of agency personnel.

Comment 9: Using competitive negotiation techniques does not automatically preclude award to below-cost bidders. The GAO has limited agency discretion to deny a fixed-price contract to an offeror on the grounds that its price is too low. You must do some research with regard to GAO decisions about "price realism."

Comment 10: The source of my statistics was the Federal Procurement Data System, which you can find at www.gsa.gov. I think Bob Antonio gave you the complete website information in his response to you.

Comment 14: While I agree that competitive negotiation offers advantages over sealed bidding in many cases, don't forget that the Government has used sealed bidding to hire construction contractors for many, many decades. While sealed bidding has shortcomings, so does competitive negotiation. In the wrong hands, competitive negotiation can become a real mess.

Comment 15: Generally, I would say that competitive negotiation is more labor intensive for agencies than sealed bidding, so I don't think that the trend toward greater use of competitive negotiation was the result of budget cutbacks. If anything, it grew out of the quality movement of the 1980s, and was facilitated by the passage of the Competition in Contracting Act of 1984, which eased the restrictions against the use of competitive negotiation to award contracts.

Comments 16 - 20: While competitive negotiation should make it easier to avoid the performance problems brought about by tight and below-cost bids, it is not clear that all agencies have taken advantage of its effectiveness in that regard. Much depends on the mindset of the individual contracting officer.

Comment 21: You are right about performance evaluations. Keep in mind, however, that the Government required agencies to prepare performance evaluations for construction contractors long before OFPP promulgated its policy about past performance and long before the enactment of the Federal Acquisition Streamlining Act's past performance provisions. See FAR 36.201. That program has not been particularly successful in terms of the quality of the evaluations, and "best value" contracting, in and of itself, will not change that.

By bob antonio on Wednesday, June 28, 2000 - 08:14 am:

Joel, Gerry, etc.:

Here is the basic information that I provided Gerry last night. It is a listing of the input forms and the organizations that produce reports from that data source.

Here is the SF 350. Check Block C8.

http://web1.whs.osd.mil/peidhome/guide/dd350_92.pdf

This page provides huge files (16 MB) of SF 350 information.

http://web1.whs.osd.mil/peidhome/guide/procoper.htm

I do not know what those files look like because they are too large to download quickly. You need T1 or T3 service. This information is provided by DIOR. They are here

http://web1.whs.osd.mil/DIORHOME.htm

The Federal Form is the SF 279. It is here

http://www.gsa.gov/forms/pdf_files/sf279.pdf

The information is in block 26. The various supply and service information is further up the form.

This information is at the Federal Procurement Data System. That is below.

http://fpds.gsa.gov/Fpds/customer.htm

A skilled user can make these systems sing-for a hefty price if ordered outside of government. FPDS also provides a free annual report that has some basics.

I also mentioned the design-build issue and gave him the site for the law offices in Chicago that are provided in another thread.

By joel hoffman on Wednesday, June 28, 2000 - 12:41 am:

Gerry, I don't know if there any reliable, official statistics for overall numbers of Best Value vs. IFB in Corps of Engineers construction programs. However, I'm familiar with two of the COE Districts' workloads and much of the general Corps of Engineer's programs.

One District reported to me that 35-40% of their overall construction program, last year, was Design-build (D-B is awarded, using negotiated, best value acquisition method).

The other District reports about 40-50% of their overall military construction program uses design-build - virtually all of Air Force Materiel Command program is design-build and I'm sure ALL of their large Air Force projects are at least negotiated BV. Virtually all of Special Operations Command work is D-B or negotiated. Some AF Commands are reluctant to use Design-Build, particularly those in the Pacific and in Europe. Some Army commands are reluctant to use D-B. Most Civil Works Districts are not using much D-B but are starting to try it.

Generally, all foreign based US military construction is awarded Best Value or at least using negotiated procurement. Central and South American work is increasingly using D-B methods.

The Construction Industry Institute recently sponsored a Penn State Study on cost and time growth for D-B vs. CM, vs. design-bid-build acquisition methods. Mark Konkar (spelling?) headed this study.

You might contact "Mark.Grammer@usace.army.mil" for Corps of Engineers' statistics, if available.

Happy Sails! Joel Hoffman

This is dated cut and paste information from a previous open forum discussion by Vern Edwards,but it is the best source I know of w/o doing extensive research.

By Vern Edwards on Monday, May 15, 2000 - 10:47 am:

Bob:

I don't know if this is the kind of statistic that you're looking for, but according to data for FY98, DOD obligated $58.2 billion on prime contract award actions that were in excess of $25,000 for which it sought full and open competition under CICA. The partial distribution of those dollars by solicitation procedure was as follows:

$40.1 billion by competitive negotiations
$ 6.0 billion by sealed bidding
$ 0.8 billion by a combination(two-step?)
$ 1.2 billion by A-E selection
$ 1.0 billion by broad agency announcement
$ 0.4 billion by multiple award schedule

Although there is no information about how the remaining partof the $58.2 billion were awarded, the numbers give you an idea about the relative significance of sealed bidding. $6 billion is not a small amount.

By Brian Fisher on Tuesday, June 27, 2000 - 04:52 pm:

Gerry,

I have not been around as long as some of the other ARNet "regulars" (involved with defense contracting only since 1982), but I also have seen the trend away from what I call "check the box" contracting (where the COs were hamstrung by rigid rules) and more toward business-oriented contracting (where the CO uses his/her discretion to fashion a deal that best suits the uniqueness of a particular contract action).

Going to your specific questions -- "(a) What is the percentage breakdown of BVC vs. Sealed Bidding on a government-wide basis for all contracts, and for construction in particular?; and (b) Is there any data or studies proving that BVC produces superior results in terms of quality, cost-efficiency, timeliness of delivery and accountability? -- I would suggest that you contact OFPP.

If you call OFPP, I might suggest that your first point of contact be Dave Muzio (202-395-6805). I don't recall whether or not the office studied those issues during my tenure; David could provide that information. If he cannot help you directly, I am sure that he can give you some guidance (or another P.O.C.).

By Eric Ottinger on Tuesday, June 27, 2000 - 03:13 pm:

Gerry,

We have been moving to toward “best value” contracting over the course of my career (which started in 1974). You are talking about negotiation vs. sealed bidding. (Since the Part 15 Rewrite, anything we do in negotiated procurement is “best value.”)

Based on some casual conversations a few years back, I would say that the buying offices are trying to use the process of negotiation to put more of the responsibility on the contractor. Without the negotiation process, the contractor usually “gets well” on change orders.

Eric

By GerryWaites on Tuesday, June 27, 2000 - 02:46 pm:

After following what appears to be a distinct shift in federal sector contracting over the last few years toward Best Value Contracting ("BVC"), I was wondering if anyone is tracking statistics and results? In particular, I have two questions:

(a) What is the percentage breakdown of BVC vs. Sealed Bidding on a government-wide basis for all contracts, and for construction in particular?

(b) Is there any data or studies proving that BVC produces superior results in terms of quality, cost-efficiency, timeliness of delivery and accountability?

As a private sector attorney, working mostly in construction, I find that most of my clients prefer the BVC approach -- provided there are sufficient checks and balances to ensure fair treatment and accurate evaluation of contractor capabilities and performance.

We noticed what appeared to be a beginning of a shift and major trend in federal construction and maintenance contracting toward BVC around 1997. Some of the key points we found in this regard are as follows:

1. Broadly speaking, BVC includes any contracts that are not sealed bid and permit evaluation of non-price factors. (If there is a downside to such an expansive definition, I'd appreciate hearing any views on it.)

2. Tracking solicitations for construction and maintenance contracts for 1997, there was nearly $2 billion worth of projects awarded under the BVC approach. I'm not sure what percentage this is of federal construction/maintenance ("C/M") contracts this was, but it is significant, particularly since most federal contracts were previously sealed bid.

3. Since 1997, the use of BVC appears to have substantially increased for C/M projects and may even be close to or at 50% by now (dollar-wise), especially when BVC/design-build projects are included. It's my understanding, for example, that the Navy uses BVC for all of its construction projects. Numerous other federal agencies across the country are also using this method for all sorts of C/M projects, from simple low-dollar projects as low as $80,000 (e.g. West Point) to highly complex, mega projects, as high as nearly $1 billion (Pentagon Renovation).

4. I'm not aware of any extensive or systematic comparative studies between BVC and sealed bidding, in terms of the extent of use, or end-results, but I would really like to know if any are out there. In terms of results, the limited information I did find favors BVC. For example, there was one short comparative analysis in a report printed in Cost Engineering by two form Army Corps engineers that showed, from the government's perspective, BVC won hands-down over sealed bidding in terms of cost/quality/schedule results and overall agency satisfaction. The trend toward BVC would also suggest agencies are getting better results.

5. To follow up on this, we did a type of focus group survey of contracting officers and, using cold calls, contacted COs around the country who were running BVC solicitations. The results were uniformly positive and interesting because it was like having the same interview over and over.

6. All COs said the results were good and better than sealed bidding and that they got to reward good contractors and avoid non-performers. Virtually all emphasized that BVC allowed them to avoid the "change order" artists, who intentionally low-balled the job, only to look for problems with the plans and specs once they got on site, and would then find ways to try to gouge the agency. Some said Best Value was the best thing that happened in their contracting office in twenty years. (I'll be glad to share any of the above information if anyone is interested).

7. From what I've personally seen so far, I'm a believer in the BVC system. It seems to produce better results for the government and taxpayer and, on balance, is better for the contracting community, although checks and balances can always be improved.

8. In fact, I know of many quality-oriented contractors who ceased bidding under sealed bidding because of the low-ball syndrome, but now are coming back and willing to compete on BVC projects.

9. This means the quality of the competition can be enhanced by using BVC, which is also good for the taxpayers. Too many low-bid contracts end up costing 30 or 50 or 100 percent more than they should, producing what the FAR explicitly recognizes as the "false economy" of the low-bid system.

10. I recently read with great interest the earlier discussion among Vern Edwards and others as to whether sealed bidding should be terminated in federal contracting. I would say it should not, but that its usefulness is very limited, e.g., if you're building a simple parking lot, or other types of acquisitions where quality, schedule and other factors are not that important. (I should note that in terms of BVC usage, Vern pointed out that DOD used competitive negotiation for $40 billion and sealed bidding for $6 billion; Vern, I'd greatly appreciate your source for this and to find out, if you know, what types of contracts the $6 billion was for?)

11. Legally, I don't think there is much of an issue. BVC can be used as long as the agency determines that the contract award will not and should not be made just "on the basis of price and other price-related factors." FAR 6.401(a)

12. In construction contracts, as with many other types of contracts, schedule, quality and numerous other "non-price" factors need to be considered in most instances, if you want to achieve the most advantageous deal for the government. Thus, it is unlikely that the use of BVC could be successfully challenged for most construction contracts and many other types of contracts, a point backed up by literally solicitations.

13. Significantly, the overarching goal of the FAR can be read to require agencies to use of the procurement method that will give the government the best results for the taxpayer, i.e., the best overall results in terms of quality/timeliness/ cost of the goods and services procured. See FAR 1.102(a)(1), 1.102-2(b).

14. And, while sealed bidding was the method of choice for most construction contracts in the past, it appears that this approach did not yield very good results on many projects, a fact that I suspect could be verified by an honest, objective review of project records for virtually any construction contracting program.

15. Too often there have been excessive cost overruns, project delays, poor quality, excessive claims or all of the above. Query: Did the trend toward BVC result, in part, from the Balanced Budget Amendments and government cut-backs since agencies now are forced to do more with less?

16. Further, it seems that the poor results from low-bid contracts flow from inherent flaws and limitations in the system itself. As one industry observer recently stated: it drives the least-cost interpretation of the bidding documents. This is a fact of life on low-bid contracts and tends to breed disputes, claims and excessive change orders.

17. A related problem is that on low-bid obs, there is no incentive to perform once a contractor wins the job. Minimal performance is usually what is delivered and basically what is expected. The problem is that, in terms of final costs and overall value, there is a big difference and a long way between minimal performance/minimal specification compliance and a high quality, successful project. This is true for construction, as well as other areas.

18. Plus, under low bid, the only recourse agencies have for poor performers is to bird-dog the specs, fight over every discrepancy and issue, dispute change requests and perhaps eventually consider the ultimate threats of default, debarment, or future non-responsibility determinations - triggers agencies seem extremely reluctant to use.

20. BVC seems to turn everything around and effectively address these problems. Contractors have every incentive to excel on every job because performance on one may very well determine the winner on the next. Innovation and partnering replace claims and disputes. (And, this is fine, provided agencies don't deny contractors "legitimate' change requests, which is an area that could use some work).

21. It seems, and I think most would agree, that the key to an effective BVC system is making sure performance evaluations are fair and accurate and that future solicitations are free of bias and agencies are privy to maximum, current performance information on all bidders.

22. Difficult challenges, but still better than the low-bid alternative. There have been some interesting developments in BVC checks and balances. Systematic formulas, for example, which score contractors on price and non-price and then make awards to the contractor with the maximum total score (as opposed to the agency's vague notion or alleged best business judgments). Other checks exist as well. A best practices survey of such safeguards would be beneficial and worth considering.

23. In closing, it appears that the real question should be not whether to use Best Value, but how to use it?

24. At this point, I'll go back to my opening questions:

(a) Does anyone have any statistics or other information to show how much federal contracting is being done by Best Value vs. Sealed Bidding on a government-wide basis, and in particular for construction; and

(b) Does anyone have any data or know of any studies on performance results in terms of how these two systems ultimately compare.

I also gladly welcome any comments on any of the other points raised above. I obviously went into some extensive detail, but I see this as an important, interesting and timely topic for ARNET discussion.

Gerry Waites