Quotas

Introduction

Quotas allow you to use logic to categorize respondents. Quotas function behind the scenes. When a respondent reaches the quota in the survey, they are either assigned to a quota group or marked as "unassigned" or "over quota."

  • Assigned respondents continue through the survey.
  • Unassigned or over-quota respondents are redirected to another location, such as a survey ending, as determined by the survey author.

For example, a quota could have this logic: 

Group 1: Customers 
Group limit = 100 
If Purchase = Yes 
Group 2: Non-customers 
Group limit = 100 
If Purchase = No 

This logic permits up to 100 respondents from both the Customers and Non-Customers groups to participate in the survey. Respondents who qualify for these groups will proceed to the next question in the survey. Once the quota limits for both groups are reached, no additional respondents can be assigned to these groups, and they will follow the designated over-quota path. 

Adding a quota

A quota element can be added to logic groups. These groups can be placed between any page in the survey and can contain skip logic or quotas.

  1. Click any Add button.
    The survey authoring interface with the add buttons highlighted.
  2. Select Quota from the list.
    Select quota from the add menu.
  3. Configure settings as needed.
    Quota element with settings applied.

Quota groups

Quota groups are the categories respondents can be placed into when they meet specified criteria, with each group having a value and a label:

  • Values: A numerical value based on the group’s order within the quota. When a respondent is assigned to a group, this value is recorded in the data for that quota.
  • Labels: Customizable names given to each group. Labels appear in reports and throughout the software for easier identification and are used in exports where labels replace values.

Quotas default to a sequential fill method, meaning the logic of the first group is always checked first for assignment. For additional options, refer to the Order of execution section in the Skip logic documentation.

Group limit

The group limit defines the maximum number of respondents assigned to a group. Once the limit is reached, no additional respondents can be assigned to that group.

A text box labeled group limit has a value of 100 entered.

Group limits can be defined using numbers or percentages (switch between these options in the quota settings):

  • Numbers: Set the exact number of respondents allowed.
  • Percentages: Define a quota limit, then allocate percentages for each group.

Setting group limits is optional. Turn this off if you want to categorize respondents for reasons other than to manage limits.

Over-sampling

Respondents qualify for a group when they reach the quota in the survey. However, they are only assigned and counted toward the group’s limit once they complete the survey. As a result, over-sampling may occur, meaning more respondents are assigned to a quota group than its limit allows.

Example

If a group limit is set to 50 and only one spot remains, but ten respondents qualify simultaneously, all ten may be assigned to the group. If all ten complete the survey, the final count for that group could exceed the limit, resulting in a total of 59.

Building conditions

Conditions are the rules that determine whether a respondent qualifies for a group.

Quota groups are evaluated in order, following the selected fill method, making both the order and specificity essential for accurate execution. For more details, refer to the Order of execution section in the Skip logic documentation.

If you need to add additional logic, click + Add condition. The extra logic is joined to the above logic using And or Or. If And is used, both rows of logic must be true for a respondent to qualify. If Or is used, either of the two rows of logic can be true to qualify a respondent.

Two logic rows joined with an And will be grouped before two joined with Or. This behaves like multiplication and addition in math order of operations. And behaves like the multiply operator and Or is like addition. Here is an example:

If Q15 = 5
Or Q15 = 7
And Q16 = 1

If the logic above included parentheses, it would evaluate as:

Q15 = 5 Or (Q15 = 7 AND Q16 = 1)

If Q15 is 5, the respondent qualifies for the quota group. However, if Q15 is not 5, then Q15 must be 7, and Q16 must be 1 to qualify.

Skip to destination

Respondents assigned to a quota group automatically progress to the next question in the survey.

Respondents who are unassigned or over quota are redirected based on the destination set at the bottom of the quota logic.

Unassigned and over quota respondents can be sent anywhere but are typically directed to survey endings with disqualified and over quota statuses set for the respondent status.

Skip to destinations are set at the bottom of the card.

Quota fill methods

Fill methods determine the order by which quota groups are considered for assignment. The fill methods available are:

  • Sequential: The default fill method. Quota groups are evaluated sequentially, top to bottom.
  • Least fill: Quota groups are initially sorted based on the remaining capacity (least full) before conditions are evaluated. The two variations of the least fill method are:
    • Counts: The groups are prioritized by the number of respondents currently assigned to them, from low to high. If the groups have the same number of respondents, ties are broken randomly.
    • Percentage: the groups are prioritized by the number of respondents assigned to them, divided by the group limit, from highest remaining capacity to lowest. If the groups have the same number of respondents, ties are broken randomly. This is unavailable if group limits are turned off.

Qualification vs assignment

When a respondent reaches a quota, each group is evaluated. Information about whether the respondent qualified for and was assigned to any groups is collected.

  • Qualification refers to when a respondent meets the criteria of a quota group.
  • Assignment indicates whether the respondent was placed in a group.

Respondents will be assigned if:

  • They qualify for the group.
  • The group limit, if set, has not been reached.

A respondent can be assigned multiple groups if the Assignments setting is set to Multiple. This setting allows respondents to be assigned to as many groups as possible, up to a number defined by the survey author.

Reporting

Every group is evaluated when a quota is encountered, and the Qualifications and Assignments are recorded for each.

Qualifications

Qualifications can be recorded as a value or label:

  • 1 (Qualified): The respondent met the logic conditions.
  • 0 (Unqualified): The respondent did not meet the logic conditions.
  • -1 (Over quota): The respondent met the logic conditions, but the group limit had already been reached.
  • Blank: The respondent did not encounter the quota.

Assignments

Assignments look slightly different depending on whether single or multiple group assignments were used.

For single-group assignments, one column is shown that can include the following values:

  • Group value or label: The quota group the respondent was assigned to.
  • 0 (unassigned): The respondent did not qualify for any quota groups and, as a result, was not assigned to any.
  • -1 (Over quota): The respondent met one or more logic conditions, but those groups reached their limit.
  • Blank: The respondent did not encounter the quota.

For multiple group assignments, a column is shown for each group and can include the following values:

  • 1 (Assigned): the respondent met the logic conditions.
  • 0 (Unassigned): the respondent did not meet the logic conditions.
  • Blank: the respondent did not encounter the quota.

Example

Let us assume we have a survey designed as follows:

  • A question that asks respondents which sodas they drink.
  • A Soda quota with three groups: 1. Pepsi, 2. Coke, and 3. Sprite.
  • The quota is a single assignment and uses the sequential fill method.

Three respondents complete the survey, resulting in the following data records:

Respondent 1 Respondent 2 Respondent 3
Pepsi: -1 Pepsi: 0 Pepsi: -1
Coke: 1 Coke: 0 Coke: 0
Sprite: 1 Sprite: 0 Sprite: -1
Soda quota: 2 Soda quota: 0 Soda quota: -1
  • Respondent 1 drank all three soda brands, qualifying for all three groups, but was ultimately assigned to Coke (marked with its numeric value of 2). Here’s why:
    • The Pepsi group was full, so they weren’t assigned to it despite qualifying.
    • They qualified for Sprite, but it came after Coke. Since the fill method was set to sequential, Coke was prioritized.
  • Respondent 2 did not qualify for any groups because they did not drink any of the three brands. They were given the status of unassigned (numeric value: 0).
  • Respondent 3 said they drank Pepsi and Sprite, but both groups were full when they took the survey, resulting in a status of over quota (numeric value: -1).

N calculation

The method for calculating N (the total number of respondents) varies based on whether single or multiple group assignments are utilized.

  • Single group assignment: N represents the total number of respondents assigned to a group.
  • Multiple group assignment: N represents the total number of respondents who encountered the quota, even if they were not assigned to any groups.

Categorical data that is not exclusive, such as a quota with multiple group assignments, can create challenges when building crosstabs. Learn more in the Crosstabs article.

Quota data exports

  1. Download the survey data file from the Data > Data records tab. This includes the individual quota and quota group values for each respondent.
  2. Download a summary of a quota from the Data > Summary page. This report reflects the data shown in the table.