As the experts in complex procurement, we’re often asked by evaluators…
How long will it take to complete the evaluation?
This is usually followed by:
I still have my day job to do.
The usual response to this question is that it is almost impossible to tell until the tenders have been opened.
However, it is possible to provide an estimate so that the evaluators and their managers can plan the evaluation into their workload – in this article we discuss how to calculate this approximation.
There are also a few tips that you, the Authority, can follow to manage the time it takes – we’ll share these along with a few pointers on how to create efficiencies below.
In a well-planned procurement with a structured set of questions, it should be possible to provide guides to bidders on the number of words or pages that are expected in the response.
These guides will be based upon a Subject Matter Expert (SME) having considered what would constitute a model answer and allowing a bit extra.
This means that for a specific evaluator, an estimate can be calculated of the number of pages per bidder they are likely to have to read.
The purchaser will know how many bidders have been invited to respond. This will give a ‘worst case’ estimate of the total number of pages for each evaluator to read.
It may also be possible to analyse previous similar tenders and to see what the average return rate is, thereby refining the estimate.
Research indicates that the time taken for the average adult to read an A4 page, containing approximately 350 words, is about two minutes.
Further, to type a good paragraph of rationale will take between three and four minutes.
The evaluator is likely to need to read the response a couple of times, consider the response against the question that was asked, consult the evaluation guidance before writing their rationale.
You will also need to factor in some breaks, as well as allowing the evaluator some time to review/assess their scores and rationales at the end of their evaluation.
And so, working very much to averages:
- It takes two minutes to read an A4 page of 350 words (allowing for imagery/diagrams and assuming content not too heavy/technical)
- The average response to one question being 10 pages
- The limit of the number of questions ideally being 16
- The number of bidders being between four and six
…for each bidder you could allow up to eight hours for the evaluators to read the questions, formulate their responses and check them.
Obviously, everyone works at a different rate, but this should give a rough estimate that will allow evaluators and managers to plan for the evaluation.
How can you, the buyer, create efficiencies within your evaluation process without compromising on results?
1. Control the number of pages and questions
By specifying page caps for each question, you will be able to understand the number of pages expected in a bid and be able to use this to estimate the time needed from your evaluators. As a guideline:
- Page caps should consider what evidence has been requested
- Page caps should exclude any plans or mandatory requested deliverables e.g., compliance matrix
- Page caps shouldn’t be word counts, it’s easier to specify and control the number of pages rather than the number of words
When setting your page caps, it may be interesting to note that a requirement for more pages by the Authority means increased costs for bidders to write their bid. And so, fewer pages can focus and please your bidders.
As a buyer, managing the number of questions you are asking bidders will help to control timeframes – around 16 questions should be sufficient for most competitions.
2. Ensure your evaluators know what they are doing
Firstly, ensure ALL of your evaluators have attended evaluation training. This will familiarise evaluators with the work required and the software used.
Using an intuitive system like AWARD® can speed up the evaluation process, as it guides evaluators through the process, creating a robust audit trail of scores and rationale as they go. (Find out more about AWARD® here.)
We also recommend you ensure your evaluators are all SMEs – unqualified evaluators shouldn’t be used and will slow down your evaluation process.
3. Effectively manage your evaluators
AWARD® allows evaluators to work anywhere – from home, in the office or on the move – so by using AWARD® you can support them in maximising their available time:
- Submission documents can be downloaded in advance, so an evaluator can read a supplier response even when an internet connection may not be always available (e.g. on a plane or train)
- The evaluation status lets evaluators keep their project teams up to date with the progress of their work, wherever they are
- Support for multiple browser sessions with multiple monitors means that evaluators can focus on the key detail of a bidder’s response
At a minimum, appoint three evaluators for each question evaluation – having less evaluators in the hope it will speed up the process is a false economy.
This helps to combat confirmation bias: the more evaluators you have (within reason) the better chance of a successful evaluation.
Whilst it may not always be possible, to get the most from your evaluators in the time they have, think carefully about when you run the evaluation.
Avoid periods when people are likely to take annual leave such as the school summer holidays, key holiday periods, or other known busy periods.
Whenever your evaluation is due to take place, book it into diaries as soon as possible to ensure availability.
Once the evaluation is underway, monitor rationale quality – catching poor evaluation/evaluator mistakes early will speed up your evaluation process.
This is easily achieved within AWARD® by running answer reports, previewing evaluator work and, as many of our clients do, by utilising the Commerce Decisions team to carry our rationale quality checks.
Evaluators should raise their evaluation clarifications to managers as soon as possible, this ensures industry has a sufficient amount of time to respond as well as accelerating the overall evaluation process.
AWARD®’s ‘Issue’ function provides seamless communications between evaluators and managers enabling an efficient way for evaluation clarifications to be raised, whilst keeping a central record of all project team communications.
4. Be clear and direct with your evaluators
Setting up a clear evaluation timetable is a good way to ensure evaluators and moderators know when work is required.
AWARD®’s closed dependency functionality allows moderators to proceed early if evaluators have completed their sections, creating great time efficiencies.
By establishing a clear line of communication between evaluators and commercial staff in case of any issues, you can avoid the process being unnecessarily slowed down simply because an evaluator has a query and doesn’t know who to ask.
You should also familiarise evaluators with the questions they’ll be evaluating early so that they can come prepared – they shouldn’t be seeing the questions for the first time on their first day of evaluation as this will take up precious time.
If using AWARD®, the easiest way to do this is by asking evaluators to log in early and view their work assignments – they can understand the question and whether they are suitably qualified to evaluate.
Another tip is to ensure you specify the format of the response documents: MS Office and PDFs are the most popular. This avoids receiving unsupported formats and wasting time.
You can also help create time efficiencies during the evaluation process in other areas of your procurement including:
- Conducting a successful early market engagement and publishing full supporting documentation as part of your tendering process. This will reduce the likelihood of clarifications being raised which would otherwise slow down your evaluation process
- If you’re running a two-stage evaluation process (evaluation and consensus) try to get the first evaluation stage right – this will speed up consensus (and later rounds of evaluation e.g., ITN).
Whilst evaluating a tender isn’t a relatively quick process, it’s a fundamental one. It’s crucial to get the best from your evaluators.
Any efficiencies are not designed to rush the process but should be part of a drive for best practice and to optimise the outcome, instilling confidence in your evaluators that this won’t take up time unnecessarily.
Evaluating a tender is a commitment – get their ‘buy-in’ early with the development and weighting of questions.
Ensure they know what they are doing (employ appropriate training at the start); create time efficiencies to maximise their effectiveness, and if asked directly how long it will take, provide a close estimate based on the number of pages, questions, and bidders.
