Evaluating Proposals

I was asked recently how to evaluate a proposal that had been sent to over 200 bidders. The Proposal Writer was thinking of developing a checklist and using this to score the bids. While this does make some sense, in reality such as checklist should have been prepared long before the RFP was published. Writing proposals in one thing; knowing how to evaluate them is another. Let’s look at how this process works. 

My first question when I spoke to the writer was if the RFP had included a scoring matrix. It hadn’t. The second was how the bidders could tell which of the requirements was the most important. While the Executive Summary alluded to these, it was couched and vague terms. 

From a neutral’s point of view, it was hard to know exactly what the government agency (i.e. those who issued the proposal) wanted to achieve. Vague requirements create vague proposals. How could it be otherwise? 

How to evaluate proposals ?

What we did in this case was: Reviewed the RFP and made a list of all the requirements. 40 requirements were found. 

Created an Excel file, made three columns, and divided the requirements into three groups: Must Have, Would Like To Have, and Not Necessary. Divided the total points to be awarded into three sections: 70% for must have, 20% for would like to have, and 10% for not necessary. 

Once we had this nailed down, we started to examine the bids. It became clear that some bids had totally missed the mark. Their proposals focussed on technologies and services that had little value for the agency, though in the bidder’s defence, the RFP hadn’t provided much direction. With 200 documents to read, we had to weed out the weaker bids fast. This allowed us to concentrate on the better ones. Several days later, we had boiled down the list to 4 bids. 

Next, we prepared a second checklist. This checklist had five columns: Understanding of Requirements, Proposed Solutions, Pricing, CVs, and Track Record. 

In reality, this second checklist should have been used when evaluating all proposals. However, with over 200 bids to work though this was not going to happen. Instead, we focused on the better bids and read these line by line several times. 

Who Evaluates the Proposals? 

I’ve worked on small projects where it was possible to write, publish, and evaluate each bids. This is not practical on large-scale proposals where you need a broader range of skills and industry knowledge to evaluate the bids. For a recent project, the evaluation team was made of five individuals. Each was an expert in their own field and could be trusted to analyse their portion of the bid accurately. 

The team members were: Project manager – studied the project plan, looked for risks and issues that would impact the deliverables. Necessary for interviewing bidder’s pm at presentations. 

Finance Officer – checked the final bid prince, daily rates, breakdown of costs, company financial background, soundness etc. Necessary for negotiating, especially costs associated with change control. 2 x Technology Experts – examined the solution proposed by the bidder. Having two experts allows you to get contrasting opinions, especially when the solution is very complex. It also protects you from scenarios whereby one reviewer has an inclination towards a particular software technology. 

Bid manager – reviews the bid against the original RFP. Ensures it addresses all requirements and that no (mandatory) requirements have been overlooked or misinterpreted. Drives the project, coordinates reviewers, and schedules the final presentations. As mentioned above, the first set of activities in the evaluation process include: 

Using checklists and scoring matrices to assess bids, i.e. define how close the proposals match the requirements  Compiling the scores from all evaluators. Preparing a final evaluation report. Outline the strength and weakness of each bid. 

After you’ve completed this first phase, you then need to:  Schedule presentations with bidders who have been short-listed. Bring in 3-5 at the most. 

Allocate 5% approx for presentations. Update the proposal’s scores based on the presentations. Award the bid. 

Inform the successful (and unsuccessful) bidders of your decision. Hold debriefing sessions with the unsuccessful bidders. 

The final step is very useful as it helps these bidders understand your reasoning, improve their bids and increase competition by setting the standard for all future proposals. If you don’t provide feedback to the bidders, they’re doomed to repeat the same mistakes in the next round of proposals.