Demo project: https://app.researchautomators.com/#/23947/Survey

This guide explains how to build a single product evaluation survey that can be reused across multiple products — in this case, energy drink brands and flavors. The key mechanism is a hidden ProductID field that is pre-filled via a URL parameter when distributing the survey. Based on that value, dynamic bookmarks automatically display the correct brand name and flavor throughout the survey, so each respondent sees questions tailored to their assigned product without requiring separate surveys.

Step 1.

Create an Open Comment question. Name it ProductID and give it the same ID. This question acts as the carrier for the product code (T01, T02, T03 etc.) that will drive all the dynamic Bookmarks later in the survey.

Step 2.

Create a Single Select question. In this example, we will name it Brand, same goes for the ID.

Enter all the answer alternatives → Settings → Answer block → Create new, and name the answer block (e.g. “Product”).

Step 3.

For each answer alternative in the Brand question, you need to assign a T-code via the Popup text field. This is what links each answer option to a specific product code.

For the first alternative: click the answer’s settings → Edit texts → Add texts → Popup text → enter T01.

Repeat for each subsequent alternative in ascending order: T02, T03, T04, and so on.

Step 4.

Open the Brand question’s settings → Coded open ends → select the ProductID question created in Step 1 (in this example: Q51 ProductID).

This connection means that when the survey detects a T-code in the ProductID field, it will automatically select the matching Brand answer, the one whose Popup text matches that code.

Step 5.

Create a new Single Select question named Flavor. Repeat the same setup as the Brand question: add all flavor alternatives, create an answer block, and activate Coded open ends pointing to the same ProductID question.

Importantly, the Popup text codes must use the same T-values as the Brand question: T01, T02, T03, T04, so that a single ProductID code selects both the correct brand and its corresponding flavor simultaneously.

Step 6.

Create a Header block and write your introductory text. Use a Bookmark in the header that references the Brand question, so the respondent’s brand name is displayed dynamically (e.g. “About your favorite {Brand}“). Add a page break between the Header block and the Brand question.

Step 7.

Create a question where the respondent rates the flavor. Add a Bookmark in the question text that references the Flavor question’s Coded open ends output, so the question reads dynamically (e.g. “How much do you like {Flavor}?“).

Step 8.

Before distributing, verify the bookmark logic works correctly. Open the survey and manually enter T01 in the ProductID field. On the following page, the Brand and Flavor bookmarks should resolve to the first entries in their respective answer blocks — in this example, Monster and Ultra White.

Step 9.

When distributing, enable panel links for the project. The base panel link in this project will look like this:

https://svar.researchautomators.se/?122200824start&XQID=[ID]&XQLANG=m

To pre-assign a specific product to a respondent, append the ProductID parameter to the end of the URL:

https://svar.researchautomators.se/?122200824start&XQID=[ID]&XQLANG=m&Q51=T01

Each T-code maps to a different brand/flavor combination. Send the appropriate link to each respondent group, there is no need to create separate surveys per product.

A question can be displayed based on a participant’s previous responses in the survey or on background data already available about the participant.

How to Add a Condition

1. Open the question you want to condition.

2. Click the three-dot menu (⋯) in the upper-right corner of the question.

3. Select Condition → Add new.

4. Give your Condition a short name, such as “Has purchased.”

5. Select which previous question or background variable should control the display, and which answer options should trigger it.

6. Click OK.

Conditions on individual response options or rows in a matrix

You can do the same thing at the detail level. Click the three-dot menu next to a specific response option or row in a matrix question instead of clicking the entire question.

Background data

Conditions can also be based on background data, not just on responses in the survey itself.

Advanced: Conditions as expressions

Each Condition is built using an expression. If you click into the expression field (or work with Error message logic), you can view and edit it directly.

The basic form is {Question}:{Answer}, where the question ID and answer ID are enclosed in curly braces. The expression is true if the participant has selected that specific answer.

Example:

• {Fulfilled}:{9} → true if the participant selected answer 9 for the question “Fulfilled”

• {Fulfilled}:{9} OR {Fulfilled}:{10} OR {Fulfilled}:{11} → true if any of the answers 9, 10, or 11 were selected

• {Gender}:{2} AND {Important}:{3} → true only if both conditions are met

For matrix questions, you specify a specific row by adding the row number: {Aspects}.1:A5 means answer 5 in row 1 of the Aspects question.

Comparisons and Calculations

Expressions support mathematical and comparison operators, which are useful for open-ended questions with numerical answers:

• Arithmetic operators: +, , , /

• Comparison operators: >, >=, <, <=, =, <>

Example: {Age} >= 18 is true if the participant answered 18 or more to the “Age” question. This can be combined with AND/OR and parentheses just as shown above.

How to Create Crosstabs

When creating a Dashboard Report, multiple crosstabs are automatically generated through our Rule-Based Report system.

A standard crosstab is created by default, and additional ones are generated based on the number of breakdown variables you have added to the Visual comparison section.

This automation ensures that all relevant comparisons are included without having to manually create each crosstab.

To add a variable to compare with in the Dashboard, click on the Edit-button in the bottom-right corner to open up the list. Then click on the 3-dots next to the variable and choose Crosstab Comparison or Visual Comparison.

Crosstabs in the Report Page

Creating or customizing a crosstab directly in the Report page is simple and flexible.

1. Switch the Presentation Type

To start, open your Report-page and switch the presentation to Table view:

• Locate the Presentation Type icon in the top-left corner of the toolbar.

• Click the icon and select Table from the list of available presentation options.

This changes the visualization from a chart or bars view to a crosstab-style data table presentation.

2. Adjust Datatype and Base

Under the Datatype dropdown, you can select how the data should be displayed:

• Percent positive

• Average

• Frequency – percentage

• And much more.

3. Add or Modify Breakdown Variables

To define what your crosstab compares, you can make sure that the Crosstab Comparison or Visual Comparison options are selected.

These Auto Variables will automatically make comparisons in your crosstab based on what variables you have added to the comparison list. This can for example be: Gender, age groups or time periods

If you only wish to use specific breakdowns to create comparisons with, you can select those instead of the Auto Variables as well.

Example Setup

In this example:

• Question is: Scaled Grid Questions 2+ Answers

• Datatype: Percent Positive

• Crosstab Comparison: Gender

In this example:

• Question is: Calculated Open ends

• Datatype: Open ends sum

• Crosstab Comparison: Age groups

What is a category hierarchy?

A category hierarchy is a structured way of organizing categories into parent–child relationships. Each level represents a layer of specificity, for example, Region → Country → Department → Team.

In Survey Automator, hierarchical categories can be used to export as Multiple Reports. By selecting the parent node as the driver variable, you can automatically generate separate reports for all child nodes, each including comparisons to its parent level.

Why use hierarchical categories?

Hierarchical categories make complex reporting manageable. They allow:

• Automatic aggregation of results from lower to higher levels.

• Drill-down navigation in dashboards and reports (e.g., Total → Region → Unit).

• Consistent data alignment across surveys sharing the same organization tree.

Without hierarchical categories, each level must be handled manually which can increase the risk of mismatched data or inconsistent roll-ups.

How are hierarchical categories different from other categories?

Ordinary categories are flat which means each value exists independently. An example of a flat, background variable would be gender or age groups.

Hierarchical categories, by contrast, define a parent–child dependency, where each node “belongs” to a level above it.

For example:

Global
 ├── Region A
 │    ├── Country X
 │    │    ├── Department 1
 │    │    └── Department 2
 │    └── Country Y
 └── Region B

This structure enables multi-level aggregation and filtering in reports, which cannot be achieved with standard (flat) categories.

Here is another example of how a hierarchical category tree could look after Categories has been exported as an Excel-file from the Distribute-tab:

And this is an example of a flat category-tree without a properly working top-node:

Working with multi-level hierarchical reporting

When a project uses a hierarchy, you enable hierarchical reporting per report/dashboard:

1. Open your Report or Dashboard.

2. Click the cogwheel (⚙) in the top bar and choose Multiple reports.

3. Select a Driver variable (your hierarchical category).

4. Confirm. The system generates a report “instance” for each node in the hierarchy, letting you navigate level by level (e.g., All → Region → Department → Team).

Important details

• Driver variable = your hierarchy. Pick the hierarchical category that mirrors your org tree.

• Top node required. The hierarchy must have a single top (e.g., “All” / “Company” / “Total”). Without it, totals and roll-ups won’t work, and you can’t base multiple reports on it.

Important: Always use a Top Node

Each hierarchy must include a top node — this acts as the root of the tree.

• The top node represents the total population (e.g., “Company” or “All respondents”).

• All child levels (regions, departments, etc.) must branch from this top node.

• You cannot build multiple reports on the same hierarchy unless it has a defined top node.

  Without it, the system cannot aggregate results properly or calculate overall totals.

Tip: Use a neutral label like “All” or “Total” as your top node to make it clear in reports.

Steps after setup

Delete unused descendants

After importing or editing your hierarchy:

• Go to the Distribute-tab

• Click on your Category and select Delete Unused Descendants.

• This helps avoid blank report entries and keeps the structure clean.

Not seeing any responses?

If your charts or report pages show empty or zero data:

1. Check the hierarchy connection

   Confirm that the category variable used in your respondent data matches the hierarchy codes exactly.

2. Check the Integrity feature

   In Report / Dashboard → Cogwheel (⚙) → Integrity, you can define the smallest base required to produce data or reports.

   This means you set a minimum number of responses that must be available for a result to be shown in a report or dashboard.

If you want to copy one specific report page (not the entire report) into another report, you do this from the Page list view using the Duplicate into action.

Prerequisites

• You have access to the project’s Report area.

• The destination report is in the same project (this function works within a project).

Step-by-Step Instructions

1. Open the Report tab
   • Go to your project and open the Report tab.

2. Switch to Page list view
   • Press Ctrl + L to switch to the Page list.
   • Alternatively, click the gear icon in the top-right corner and select Page list.

3. Select the page you want to copy
   • In the Page list, click the row for the page you want to duplicate.
   • You can also select multiple pages if needed.

4. Click “Duplicate into”
   • Click the Duplicate into icon (the duplicate/copy action in the toolbar above).
   • A dropdown/list will appear.

5. Choose the destination report
   • From the list, select the report you want to copy the page into.
   • The system will duplicate the selected page(s) into the chosen report.

Survey Automator can flag statistically significant differences in banner and cross-tab tables, so you can see at a glance which results stand out rather than reflecting random sampling variation. This document explains what is tested, how, and how results are displayed.

What Gets Tested

Significance is calculated for two kinds of values in a table:

• Proportions — the percentage of respondents giving a particular answer.

• Averages — the mean of a numeric or scale value.

For each, Survey Automator compares one figure against a reference figure and asks: is this difference large enough to be unlikely to have arisen by chance?

Comparison Mode

You choose what each column is compared against:

In Total mode, each breakdown group is tested against the full population — useful for spotting which subgroups differ from the average respondent. In Previous mode, columns are tested against each other in sequence — useful for tracking changes across waves or ordered groups.

How the Tests Work

All tests produce a z-score: a measure of how many standard errors a difference is away from zero. A larger absolute z-score means a more surprising difference. The z-score is then compared against a critical threshold tied to your chosen confidence level.

Comparing a subgroup against the total

Proportions. The subgroup’s proportion is compared to the total proportion, with the standard error based on the subgroup’s sample size.

Averages. The subgroup’s mean is compared to the total’s mean (a one-tailed test), using the total’s standard deviation and the subgroup’s sample size to form the standard error.

Comparing two groups against each other

Proportions. The two groups’ proportions are compared using a pooled proportion (combining both groups to estimate the shared variance), with the standard error reflecting both sample sizes.

Averages. The two means are compared using the standard error built from each group’s own standard deviation and sample size. This test only runs when both groups have more than 30 respondents — below that, no significance is calculated, because the test is not reliable on small samples.

Confidence Levels

You select a confidence level, which sets the threshold a z-score must clear to count as significant. Higher confidence demands a larger difference.

These are one-tailed thresholds, so the test distinguishes “significantly higher” from “significantly lower” rather than just “different.”

How Results Are Displayed

Each tested value gets one of three outcomes:

The colors are themeable, so they can be matched to client branding. The same significance marker carries through consistently to every export format — both Excel and PowerPoint reports show the identical highlighting.

Column-to-column comparison (a/b/c letters)

In addition to the higher/lower arrows, columns can be compared pairwise, with each column labelled a letter (a, b, c, …). A letter placed on a value means that value is significantly higher than the lettered column — e.g. a “c” on column A reads “A is significantly higher than C.”

Methodology notes

This section explains the reasoning behind two methodological choices.

Why the tests are one-tailed

The confidence thresholds above (1.282 / 1.645 / 2.326) are one-tailed. This is the correct choice because every significance mark in the product is directional — an up-arrow claims “higher,” a down-arrow claims “lower,” and an a/b/c letter claims “higher than that column.”

What the “vs Total” comparison actually answers

In Total mode, each subgroup is compared against the overall total. The current test answers comparison against the visible total — which has the advantage that the arithmetic matches what is on screen. A vs-Total flag means “differs from the overall benchmark,” and it is a lenient, exploratory comparison — a starting point, not an independent-difference verdict.

Multiple comparisons in a/b/c tables

Pairwise column comparison runs many more tests than the arrows. Each individual letter is a correct one-tailed test at its stated level, but because so many tests run across a table, some letters will appear by chance alone. The letters are not family-wise corrected, so a table full of letters should be read as the sum of many independent tests, not as one collectively-certain picture.

Correlation Tables

Correlation tables use a different, dedicated significance test (a t-test) suited to correlation coefficients. This is separate from the proportion and average testing used in standard banner and cross-tab tables.