Online Assessments

Introduction

Before implementing the Online Assessment (OA) module, it’s crucial to have a clear strategy outlining your needs, the process you’ll follow, and how you’ll measure the results.

For instance, if your goal is to assess vendors, you need to define the process for executing these assessments and establish clear criteria to determine whether a vendor meets your standards or not.

In this episode, we guide you through crafting a strategy, designing the process, and defining how to evaluate the results. Clarifying these elements beforehand is essential to ensure a smooth and effective implementation of the module.

Strategy

The purpose of the strategy is to clarify, within the larger framework, what information you need, who you will evaluate to obtain it, and how you will conduct the evaluation.

The table below is a simple example of what we mean by this stragegy definition. As you can see your organisation might use OAs for multiple purposes at the same time.

What	From Whom	Questionaire	Evaluation	Authentication
SaaS Vendor Assessments (Cyber Security)	All our SaaS suppliers (Third Parties in eramba)	Ask questions about the security of their SaaS platforms	Sufficiently Security/Acceptable/Non-Acceptable	Non-Authenticated
Organisation Risk Assessment	Head of departments in our orgnanisation (Business Units in eramba)	Ask questions related to the risks they see in their day a day	Incomplete/Complete	Non-Authenticated

The strategy will help you determine what you need to implement in Eramba:

• From Whom: Decide the focus of your assessments. Your Online Assessments (OAs) will be linked to either Third Parties (e.g., suppliers, vendors) or Business Units (e.g., internal departments) depending on your objective.
• Questionnaire: Design a well-structured questionnaire that serves two purposes:
  • To efficiently collect the required information from respondents.
  • To allow for quick and effective evaluation of the OA (pass, failed, high risk, low risk, etc).
• Evaluation: Use a custom field to document the outcome of the OA, providing a clear record of the results and the status of the assessment. The result of your OA will automatically reflect on your Third Party or Business Unit.
• Authentication: Decide whether to manage user accounts for respondents (authenticated OAs) or to simplify the process by using a unique URL for access (non-authenticated OAs).

This framework ensures a smooth and tailored implementation of OAs in Eramba. Let me know if you’d like further elaboration!

Process

Once your strategy is clear, the next step is to define an operational process to implement it. This involves answering key questions such as: Who is responsible for each task, and when should it be done? What notifications I need? What custom fields will be required?

The diagram below shows an example of such plan for the strategy "SaaS Vendor Assessments (Cyber Security)" described in the previous chapter:

Each step of the process in eramba:

1. Create Supplier in Eramba: Once the supplier is identified, create them as a Third Party in Eramba. Use custom fields to include relevan t attributes like contact name, address, criticality, and more.
2. Send Non-Authenticated OA: Send the supplier a non-authenticated Online Assessment (OA) using notifications. They will complete the questionnaire and submit their responses, this will also trigger notifications in order to notify you.
3. Evaluate the Score: If the questionnaire’s tailored scoring system results in a score above 50 points, the OA is marked as "Pass" and that greenlit the supplier. We will let them know about this using notifications.
4. Review Feedback for Low Scores: If the score is below 50 points, we will read their responses in detail. Use the Comments & Attachments feature to discuss unclear or problematic answers with the supplier.
5. Address Issues with Findings: If issues are identified and the supplier agrees to address them, create Findings to track these follow-ups. Let them know about the findings using notifications. The OA will also be tagged as Risky, and this status will reflect on the supplier’s profile.
6. Greenlight Without Issues: If no significant issues are identified during the review, the supplier will be greenlit, regardless of the score, as long as their responses are acceptable.

You should define the process that works best with your organisation and try to reflect how that will work in eramba, this is essential as multiple configurations will be required to implement it

In the coming chapters in this episode we will dig deeper into the key components of the OA implementation and operation.

Tracking

The purpose of tracking is to ensure that the results of your Online Assessments (OAs) have a direct impact on the associated items. For example, as shown in the screenshot, if an OA is tagged as “Risky,” the associated Third Party will also be marked as “Risky.”

This linkage provides clear visibility into the status of related items and ensures that the outcomes of assessments are reflected across the relevant modules, allowing for better decision-making and risk management.

When creating an Online Assessment (OA) in Eramba, you can associate it with the following options depending on your assessment’s purpose:

• Organisation / Third Party: Typically used for vendor or supplier assessments to evaluate their compliance or risk levels.
• Organisation / Business Unit: Used for assessing internal departments in your company, often for risk evaluation or internal process reviews.
• Risk Management / Risks (all three): Used when the OA identifies a risk, allowing you to document and link it directly to the relevant risk in your system.
• Asset Management / Data Flows: Useful for assessments aimed at understanding how data moves within your organization, ensuring compliance with data protection or privacy requirements.
• Asset Management / Assets: Applied when the purpose of the OA is to identify or evaluate specific assets within the organization, such as hardware, software, or critical infrastructure.

Questionaires

The more specific and less arbitrary your questions are, the easier they will be to answer and evaluate. Well-designed questionnaires are key to making the process efficient.

In Eramba, there are two types of questions you can use: Open and Drop-Down.

• Open Questions allow respondents to provide free-form answers. While this offers flexibility, the unstructured data must be read, interpreted, and analyzed manually, leaving room for extended discussions.
• Drop-Down Questions provide predefined options, similar to a multiple-choice format. These can be scored, allowing for quick evaluation of pass/fail scenarios. Drop-down questions also enable conditional logic, meaning specific questions can appear or be hidden based on the respondent’s answer.

Whenever possible, we strongly recommend using drop-down questions as they streamline both the response process and the evaluation, making the entire assessment more efficient.

When using scoring in your questionnaires, we recommend designing it so that “good” answers reduce the score and “bad” answers increase it (or vice versa, depending on your preference). This method ensures clarity when interpreting results.

For example: Question: Is your company able to share a SOC2 Report?

• Yes: -10 points
• No: +10 points

You can then define thresholds for interpretation. For instance, if an OA scores above 10 points, it could automatically be tagged as “SCORE: Not Great” to indicate areas of concern. This scoring system simplifies evaluations and allows for automated tagging and decision-making based on defined criteria.

In some cases, this scoring approach is not practical, especially when dealing with generic or open-ended questions. For example:

• What risks do you know about in your organization?
• What assets do you commonly use?

These types of questions require free-form answers that cannot easily be scored or quantified. Instead, they are meant to gather unstructured data for deeper review, interpretation, and discussion. In such cases, the focus shifts to collecting detailed insights rather than assigning scores, and the evaluation must rely on manual analysis of the responses.

Evaluation

The evaluation of an Online Assessment (OA) is optional, but it is essential for defining the “outcome” of your assessment. Typical examples of outcomes might include:

• Complete, Incomplete
• Fail, Pass
• High Maturity, Medium Maturity, Low Maturity
• Acceptable Risk, Crazy Risky
• Etc.

Since OAs can be used for various purposes, you’ll need to define these outcomes yourself. This is achieved by using custom fields, which allow you to create dropdown fields where you can specify the options that best suit your needs. This flexibility ensures that the evaluation aligns perfectly with your organization’s goals and the purpose of the assessment.

When the OA is reviewed you will define its result and that can be (if cofigured) automatically tag the related item (see Tracking above).

Authentication

When you send an Online Assessment (OA) to an entity, the recipient will receive a link, typically via an email sent by Eramba. Clicking the link will take them to the OA portal.

At this stage, there are two options for access:

• Authenticated Access: The person must log in with a username and password. This requires you to create accounts for them in Eramba.
• Non-Authenticated Access: The link sent to them is unique to their OA. No login is required, and only those with the link can access the assessment.

Non-authenticated OAs are now the standard because they simplify the process for both you and the respondent, eliminating the need to manage user accounts while still maintaining access control.