Evaluation Methods - Multi-Party Evaluation (MPE)

  • Updated

Go-To-Market-Banner.png

Admin-and-Buyers-Banner-shorter.png

The Multi-Party Evaluation (MPE) tool allows multiple people to take part in the same evaluation. The evaluation is managed by the Buyer, who can invite participants via email. These participants do not need a VendorPanel account to conduct their evaluation. 

The invited evaluators will score each Supplier response based on the criteria set by the buyer. A consensus decision is then reached on the scores for each response before a decision is made on the successful Supplier.

If you'd prefer to use another evaluation method (Quick Select or Evaluation Matrix), you can change your evaluation method at any time, see more information here: Changing the Evaluation Method.

General Notes

  • If MPE has been enabled for your enterprise then it can be turned on and off at a Local Group level.
  • Administrators impersonating a Buyer cannot access their MPE process. You need to be directly logged into the buying account to access the MPE process.
  • Price is not suggested as an evaluation criterion as it is displayed on the X-Axis of the results graph. This allows you to examine Score vs. Price.

Reporting:

Buyers can generate a draft PDF report at any stage of the MPE process. The document is identified as a draft at the top of the report and amended to the bottom of all pages. This can be downloaded on the Summary tab of the MPE at any time. A final version will be available to download on the Responses page for the request once the evaluation has been completed.

Please note

If you do not currently have access to use the MPE tool, please contact your Administrator or Customer Success Manager to enquire about purchasing an MPE licence.

Start Multi-Party Evaluation

You can view your responses by clicking the green people icon to the right of your request or tender. You will need to have unlocked the responses before recording your decision. You can find information on viewing and unlocking responses here: Viewing the Responses to a Request.

When viewing your unlocked responses, click the Start Multi-Party Evaluation button.

mceclip0.png

Summary

This is the first step of the Multi-Party Evaluation where you will find the following:

  • Overview of Request and Buyer details
  • Overview of submitted response
  • Exclude responses (optional)
  • Creation of Teams (optional)

MPE teams allow users to organise evaluators into groups that are assigned specific criteria. 

If teams are used, they are allocated the relevant criteria before the Buyer invites the evaluators into their designated teams. Evaluators can only see and evaluate the criteria that have been assigned to their team.

Instructions

  1. Review RFx, Buyer and response information as needed.

  2. If required, exclude any responses to your Request by clicking on the Included link next to the relevant response.

  3. If required, add your Evaluation Teams to your Request. This can be entered as free text.

Click on the Next button when you are ready to move on to the next step.

Criteria

Overview

  • You can either select to use an existing Criteria Template or create your own Criteria.
  • The pricing evaluation method can be chosen from the 3 options:
Independent Factor (recommended)

This is the recommended pathway for price evaluations, treating price independently of your criteria, in order to provide insights into the value for money of responses.

Using this method, pricing is excluded from evaluations until the results step, where the evaluation scores are then compared against the pricing submitted, to create a Value for Money graph.  This holds response Scores on the X axis, and Pricing on the Y axis.
Price Auto-scored

Buyers are also able to choose to have prices evaluated automatically, in line with their other manual evaluations.

Buyers choose the weighting given to the price and are given the opportunity to confirm/amend the given price on the Prices step. The Price score is then factored alongside the other evaluation criteria, to receive a single, whole score for the response.
Price Manual Buyers are able to choose to disable price evaluations in order to allow you to include price as a normal criteria, which is to be manually scored by your evaluators.
  • A minimum of 2 and a maximum of 25 Criteria may be set.
  • Weighting may be customised:
    • Criteria are locked when moving to the consensus step, but the weighting of the criteria can be adjusted up until the results are entered and confirmed. 
  • An audit log is added to the end of the PDF report showing what was changed, including the new and previous values. 
  • Criteria must be evaluated between 2 (Boolean) and 11 (0-10)
  • Criterion titles, Description, and Levels may be named in free-text
  • Allocate the criterion to the relevant Teams

Instructions

If using a criteria template:

  1. Select a template from the dropdown menu and click on the Load template button.

  2. Choose your pricing evaluation method.
    1. If using 'Auto-scored criterion', enter the weighting to be given to the price.

  3. You can make changes to any of the criterion in the template, or you can remove them by clicking on the red X button.

  4. You can also add new criteria to the list using the Add criteria button at the bottom of the page.

  5. If using MPE Teams, allocate your criteria to the appropriate Team(s) that you set in the Summary step.

If creating your own criteria:

  1. Choose your pricing evaluation method.
    • If using 'Auto-scored criterion', enter the weighting to be given to the price.

  2. Click on the Add a criterion button to add your first criteria.

  3. Name the Criteria and give it a description for evaluators to reference.

  4. Decide how many levels the criteria should be scored from (2 - 11), and add/remove levels as required. You may also name the levels as appropriate.

  5. If using MPE Teams, allocate your criteria to the appropriate Team/s.

  6. Add weighting for each criterion (may be easier once you have added all criteria).

    mceclip0.png

  7. Repeat this until all necessary criteria have been added.

Click on the Next button when you are ready to move on to the next step.

Evaluators

Overview

  • Add your Evaluators and customize evaluation settings and document access. The Buyer running the Evaluation is automatically listed as an Evaluator.
  • If an invitation to an evaluator does not reach its recipient and is 'bounced' by their domain name, an onscreen notification will appear notifying you of the email bounce.
  • Evaluators can be excluded at any point.
  • Once Evaluations are received with all criteria being evaluated at least once, you may continue to the next step.
Please note

Continuing to the next step will mean that you will no longer be able to invite or exclude any evaluators, or change any of the evaluation criteria. Evaluators will no longer be able to edit their responses.

Instructions

  1. Add Evaluators by entering their name, email address, assigning them to a team (if using) and clicking on the Add button.
    • Evaluators do not need to be registered on VendorPanel.
    • A minimum of 2 Evaluators are needed (the buyer is counted as one evaluator).
    • Clicking the Add button does not immediately send them the invite.

      mceclip1.png
  1. Select the Choose comment settings button to select either Disabled, Optional or Required for:
    • Overall response comments - one comment for each response:
    • Response criteria comments - comments for each response/criterion combination.
  1. Select the Choose document access button to select what documents are available for the Evaluators to access. This includes a 'no-price' option for each document which excludes the quoted prices from the Supplier's responses.
  1. Select the Customise invitation email text button to provide a custom message for Evaluators with instructions or background. This is optional.

  2. When you are ready to send the invitations, click on the checkbox next to each Evaluator and then click on the Send invite(s) button.

    mceclip3.png
Please note

Once invitations have been sent to the evaluators, adding criteria or removing levels may result in completed evaluators needing to do more work.

If needed, evaluators' statuses will be updated - further communication and re-inviting evaluations may be required

Other changes may mean any completed evaluations should be re-considered:

  • Criteria & levels: when changing the meaning of a criterion or any levels
  • Levels: when re-ordering levels if it changes the way an evaluator would interpret them
  • Levels: when adding new levels that an evaluator may have chosen had they had the chance

You can find steps on how to complete a Multi-Party Evaluation survey as an Evaluator here: Completing a Multi-Party Evaluation (MPE) Survey.

Once all evaluators have submitted their responses, click on the Next button when you are ready to move on to the next step.

Consensus

Overview

  • Review the final scores for all responses for all evaluators.
  • The marked level is the average rating from all participants.
  • Meet with your Evaluators to discuss the results of their evaluations.
  • Adjust the scores/results for each response, if necessary.

Instructions

  1. View consensus results.

  2. View individual Evaluators' results including their comments provided by clicking on the option at the bottom of each supplier's evaluation details. 

  3. Discuss these with your team internally and make any changes to the consensus results where required.

  4. Add in private notes in the private 'Comments/Notes' section to explain the change in result.

mceclip6.png

Click on the Next button when you are ready to move on to the next step. This will lock the consensus categorisation for each supplier and you will no longer be able to edit the categorisation of any vendor responses.

Prices

Overview

  • Review pricing.
  • Update prices based on discussions or options provided in the Supplier's response.

Instructions

  1. Re-examine the pricing provided by Suppliers. 

  2. Add private notes to explain or justify the changes to the original pricing.

    mceclip9.png

Click on the Next button when you are ready to move on to the next step. Prices may be edited until the Results step has been completed and confirmed.

Results

Overview

  • Review the final results of your evaluation.
  • If using Independent factor pricing evaluations, Supplier responses are charted on a graph that compares their evaluation result (Y-Axis) with their final price (X-Axis). 

Instructions

  1. Compare Supplier responses on the graph. 

  2. In the table under Vendor responses, accept (select the tick option) or decline (select the X option) each supplier response. 

  3. Click the Supplier's name to add accept/decline notes, or private notes. At this step, you can select the 'do not send an accept/decline email' option here. 

Click on the Next button when you are ready to move on to the next step.

Confirm

Instructions

To complete your Multi-Party Evaluation, click the Confirm button. This will pull your results and decisions into VendorPanel on the responses page.

If you have not finalised the request and you need to make any changes to your evaluation, you can select the Edit Multi-Party Evaluation button which will restart your evaluation. The evaluations you already completed and the information you have entered will be saved. However, you will need to step through each tab again to complete the evaluation and confirm your decision. 

You can select the Download Multi-Party Evaluation button to download the final Multi-party evaluation report in PDF format.

mceclip10.png

Once your evaluation is complete, you can then finalise the request to lock your decision. You can find steps on how to finalise your request here: Finalising your Request or Tender.

Appendix - Field Character Limits

The following character limits apply to the specified fields:

Step Field Character Limit
Summary Add Teams 250 characters
Criteria Criterion 255 characters
Criterion Description Unlimited
Levels 255 characters
Evaluators Customised Invitation email text Unlimited
Consensus Comments/Notes against a response Unlimited
Prices Comments/notes >9,999,999
Results Private note Unlimited
Note for Supplier Unlimited
Completing Surveys Overall Response Comments Unlimited
Response Criteria Comments Unlimited