Beginner tutorial: gaze cuing

About OpenSesame

OpenSesame is a program for easy of development of behavioral experiments for psychology, neuroscience, and experimental economy. For beginners, OpenSesame has a comprehensive graphical, point-and-click interface. For advanced users, OpenSesame supports Python scripting (not covered in this tutorial).

OpenSesame is freely available under the General Public License v3.

About this tutorial

This tutorial shows how to create a simple but complete psychological experiment using OpenSesame (Mathôt, Schreij, & Theeuwes, 2012). You will use mainly the graphical user interface of OpenSesame (i.e., no Python inline coding), although you will make small modifications to the OpenSesame script. This tutorial takes approximately one hour.

Tutorial screencast

This tutorial is also available as a screencast:

Resources

• Download -- This tutorial assumes that you are running OpenSesame version 3.1.0 or later. To check which version you are running, see the bottom right of the 'Get started' tab (see Figure 2). You can download the most recent version of OpenSesame from:
  • https://osdoc.cogsci.nl/3.2/download
• Documentation -- A dedicated documentation website can be found at:
  • http://osdoc.cogsci.nl/
• Forum -- A support forum can be found at:
  • http://forum.cogsci.nl/

The experiment

In this tutorial, you will create a gaze-cuing experiment, as introduced by Friesen and Kingstone (1998). In this experiment, a face is presented in the center of the screen (Figure 1). This face looks either to the right or to the left. A target letter (an 'F' or an 'H') is presented to the left or right of the face. A distractor stimulus (the letter 'X') is presented on the other side of the face. The task is to indicate as quickly as possible whether the target letter is an 'F' or an 'H'. In the congruent condition, the face looks at the target. In the incongruent condition, the face looks at the distractor. As you may have guessed, the typical finding is that participant respond faster in the congruent condition, compared to to the incongruent condition, even though the direction of gaze is not predictive of the target location. This shows that our attention is automatically guided by other people's gaze, even in situations where this doesn't serve any purpose. (And even when the face is just a smiley!)

Figure 1. The gaze-cuing paradigm (Friesen and Kingstone, 1998) that you will implement in this tutorial. This example depicts a trial in the incongruent condition, because the smiley looks at the distractor ('X') and not at the target ('F').

The experiment consists of a practice and an experimental phase. Visual feedback will be presented after every block of trials, and a sound will be played after every incorrect response.

Experimental design

This design:

• is within-subject, because all participants do all conditions
• is fully-crossed (or full factorial), because all combinations of conditions occur
• has three factors (or factors):
  • gaze side with two levels (left, right)
  • target side with two levels (left, right)
  • target letter with two levels (F, H)
• has N subjects

See also Video 2 for an explanation of the logic and design of the experiment.

Step 1: Create the main sequence

When you start OpenSesame, you see the 'Get started!' tab (Figure 2). (If this is the very first time that you have started OpenSesame, you will see a tab that points you towards this tutorial. If so, you can close this tab.) A list of templates is shown below 'Start a new experiment'. These templates provide convenient starting points for new experiments. After you saved an experiment the first time, recently opened experiments are shown under 'Continue with a recent experiment'.

Figure 2. The 'Get started' dialog on OpenSesame start-up.

Click on 'Default template' to start with a minimal experimental template.

By default there is a main sequence, which is simply called experiment. Click on experiment in the overview area (by default on the left side, see Figure 3) to open its controls in the tab area. The experiment sequence consists of two items: a notepad called getting started and a sketchpad called welcome.

We don't need these two items. Remove getting_started by right-clicking on it in the overview area and selecting 'Delete' (shortcut: Del). Remove welcome in the same way. The experiment sequence is now empty.

Figure 3. The default layout of the OpenSesame interface.

Append a form_text_display item for the instruction display

As the name suggests, a form_text_display is a form that displays text. We are going to use a form_text_display to give instructions to the participant at the beginning of the experiment.

Click on experiment in the overview area to open its controls in the tab area. You will see an empty sequence. Drag a form_text_display from the item toolbar (under 'Form', see Figure 3) onto the experiment sequence in the tab area. When you let go, a new form_text_display item will be inserted into the sequence. (We will get back to this in Step 12.)

Append a loop item, containing a new sequence item, for the practice phase

We need to append a loop item to the experiment sequence. We will use this loop for the practice phase of the experiment. Click on the experiment sequence to open its controls in the tab area.

Drag the loop item from the item toolbar into the sequence just the way you added the form_text_display. New items are inserted below the item that they are dropped on, so if you drop the new loop onto the previously created form_text_display, it will appear where you want it: after the form_text_display. But don't worry if you drop a new item in the wrong place, because you can always re-order things later.

By itself, a loop does not do anything. A loop always needs another item to run. Therefore, you have to fill the new loop item with another item. (If you view the loop item, you will also see a warning: 'No item specified'.) Drag a sequence item from the item toolbar onto the loop item. A pop-up menu will appear, asking you whether you want to insert the sequence after or into the loop item. Select 'Insert into new_loop'. (We will get back to this in Step 2.)

Append a new form_text_display item for the end-of-practice message

After the practice phase, we want to inform the participant that the real experiment will begin. For this we need another form_text_display. Go back to the experiment sequence, and drag a form_text_display from the item toolbar onto the loop item. The same pop-up menu will appear as before. This time, select 'Insert after new_loop'. (We will get back to this in Step 12.)

Append a new loop item, containing the previously created sequence, for the experimental phase

We need a loop item for the experimental phase, just like for the practice phase. Therefore, drag a loop from the item toolbar menu onto _form_text_display.

The newly created loop (called new_loop_1) is empty, and should be filled with a sequence, just like the loop we created before. However, because the trials of the practice and experimental phase are identical, they can use the same sequence. Therefore, instead of dragging a new sequence from the item toolbar, you can re-use the existing one (i.e. create a linked copy).

To do this, right-click on the previously created new_sequence, and select 'Create copy (linked)'. Now, right-click on new_loop_1 and select 'Paste'. In the pop-up menu that appears, select 'Insert into new_loop 1'.

Append a new form_text_display item, for the goodbye message

When the experiment is finished, we should say goodbye to the participant. For this we need another form_text_display item. Go back to the experiment sequence, and drag a form_text_display from the item toolbar onto new_loop_1. In the pop-up menu that appears, select 'Insert after new_loop_1'. (We will get back to this in Step 12.)

Give the new items sensible names

By default, new items have names like new_sequence and new_form_text_display_2. It is good practice to give items sensible names. This makes it much easier to understand the structure of the experiment. If you want, you can also add a description to each item. Item names must consist of alphanumeric characters and/or underscores.

• Select new_form_text_display in the overview area, click on its label in the top of the tab area and rename the item to instructions. (Overview-area shortcut: F2)
• Rename new_loop to practice_loop.
• Rename new_sequence to block_sequence. Because you have re-used this item in new_loop_1, the name automatically changes there as well. (This illustrates why it is efficient to create linked copies whenever this is possible.)
• Rename new_form_text_display_1 to end_of_practice.
• Rename new_loop_1 to experimental_loop.
• Rename new_form_text_display_2 to end_of_experiment.

Give the whole experiment a sensible name

The experiment in its entirety also has a title and a description. Click on 'New experiment' in the overview area. You can rename the experiment in the same way as you renamed its items. The title currently is 'New experiment'. Rename the experiment to 'Tutorial: Gaze cuing'. Unlike item names, the experiment title may contain spaces etc.

The overview area of your experiment now looks like Figure 5. This would be a good time to save your experiment (shortcut: Ctrl+S).

Figure 5. The overview area at the end of the step 1.

Step 2: Create the block sequence

Click on block_sequence in the overview. At the moment this sequence is empty. We want block sequence to consist of a block of trials, followed by a feedback display. For this we need to do the following:

Append a reset_feedback item to reset the feedback variables

We don't want our feedback to be confounded by key presses that participants have made during the instruction phase (or during previous blocks of trials). Therefore, we start each block of trials by resetting the feedback variables. To do this we need a reset_feedback item. Grab reset_feedback from the item toolbar (under 'Response collection') and drag it onto block_sequence.

Append a new loop, containing a new sequence, for a block of trials

For a single trial we need a sequence. For a block of trials, we need to repeat this sequence multiple times. Therefore, for a block of trials we need to wrap a loop around a sequence. Drag a loop from the item toolbar onto reset_feedback. Next, drag a sequence from the item toolbar onto the newly created loop, and select 'Insert into new_loop' in the pop-up menu that appears. (We will get back to this in Step 3.)

Append a feedback item

After every block of trials we want to give feedback to the participant, so that the participant knows how well he/ she is doing. For this we need a feedback item. Drag a feedback from the item toolbar onto new_loop, and select 'Insert after loop' in the pop-up menu that appears. (We will get back to this in Step 10.)

Give the new items sensible names

Rename: (See Step 1 if you don't remember how to do this.)

• new_loop to block_loop
• new_sequence to trial_sequence
• new_reset_feedback to reset_feedback
• new_feedback to feedback

The overview of your experiment now looks like Figure 6. Remember to save your experiment regularly.

Figure 6. The overview area at the end of Step 2.

Step 3: Fill the block loop with independent variables

As the name suggests, block_loop corresponds to a single block of trials. In the previous step we created the block_loop, but we still need define the independent variables that will be varied within the block. Our experiment has three independent variables:

• gaze_cue can be 'left' or 'right'.
• target_pos (the position of the target) can be '-300' or '300'. These values reflect the X-coordinate of the target in pixels (0 = center). Using the coordinates directly, rather than 'left' and 'right', will be convenient when we create the target displays (see Step 5).
• target_letter (the target letter) can be 'F' or 'H'.

Therefore, our experiment has 2 x 2 x 2 = 8 levels. Although 8 levels is not that many (most experiments will have more), we don't need to enter all possible combinations by hand. Click on block_loop in the overview to open its tab. Now click on the 'Full-factorial design' button. In the variable wizard, you simply define all variables by typing the name in the first row and the levels in the rows below the name (see Figure 7). If you select 'Ok', you will see that block_loop has been filled with all 8 possible combinations.

In the resulting loop table, each row corresponds to one run of trial_sequence. Because, in our case, one run of trial_sequence corresponds to one trial, each row in our loop table corresponds to one trial. Each column corresponds to one variable, which can have a different value on each trial.

Figure 7. The loop variable wizard in Step 3.

But we are not done yet. We need to add three more variables: the location of the distractor, the correct response, and the congruency.

• dist_pos -- On the first row of the first empty column, enter 'dist_pos'. This automatically adds a new experimental variable named 'dist_pos'. In the rows below, enter '300' wherever 'target_pos' is -300, and '-300' wherever 'target_pos' is 300. In other words, the target and the distractor should be positioned opposite from each other.
• correct_response -- Create another variable, in another empty column, with the name 'correct_response'. Set 'correct_response' to 'z' where 'target_letter' is 'F', and to 'm' where 'target_letter' is 'H'. This means that the participant should press the 'z' key if she sees an 'F' and the 'm' key if she sees an 'H'. (Feel free to choose different keys if 'z' and 'm' are awkward on your keyboard layout; for example, 'w' and 'n' are better on AZERTY keyboards.)
• congruency -- Create another variable with the name 'congruency'. Set 'congruency' to 'congruent' where 'target_pos' is '-300' and 'gaze_cue' is 'left', and where 'target_pos' is '300' and 'gaze_cue' is 'right'. In other words, a trial is congruent if the face looks at the target. Set 'congruency' to 'incronguent' for the trials on which the face looks at the distractor. The 'congruency' variable is not necessary to run the experiment; however, it is useful for analyzing the data later on.

We need to do one last thing. 'Repeat' is currently set to '1.00'. This means that each cycle will be executed once. So the block now consists of 8 trials, which is a bit short. A reasonable length for a block of trials is 24, so set 'Repeat' to 3.00 (3 repeats x 8 cycles = 24 trials). You don't need to change 'Order', because 'random' is exactly what we want.

The block_loop now looks like Figure 8. Remember to save your experiment regularly.

Figure 8. The block_loop at the end of Step 3.

Step 4: Add images and sound files to the file pool

For our stimuli, we will use images from file. In addition, we will play a sound if the participant makes an error. For this we need a sound file.

You can download the required files here (in most webbrowsers you can right-click the links and choose 'Save Link As' or a similar option):

• gaze_neutral.png
• gaze_left.png
• gaze_right.png
• incorrect.ogg

After you have downloaded these files (to your desktop, for example), you can add them to the file pool. If the file pool is not already visible (by default on the right side of the window), click on the 'Show file pool' button in the main toolbar (shortcut: Ctrl+P). The easiest way to add the four files to the file pool is to drag them from the desktop (or wherever you have downloaded the files to) into the file pool. Alternatively, you can click on the '+' button in the file pool and add files using the file select dialog that appears. The file pool will be automatically saved with your experiment.

Your file pool now looks like Figure 9. Remember to save your experiment regularly.

Figure 9. The file pool at the end of Step 4.

Step 5: Fill the trial sequence with items

A trial in our experiment looks as follows:

1. Fixation dot -- 750 ms, sketchpad item
2. Neutral gaze -- 750 ms, sketchpad item
3. Gaze cue -- 500 ms, sketchpad item
4. Target -- 0 ms, sketchpad item
5. Response collection -- keyboard_response item
6. Play a sound if response was incorrect -- sampler item
7. Log response to file -- logger item

Click on trial_sequence in the overview to open the trial_sequence tab. Pick up a sketchpad from the item toolbar and drag it into the trial_sequence. Repeat this three more time, so that trial_sequence contains four sketchpads. Next, select and append a keyboard_response item, a sampler item, and a logger item.

Again, we will rename the new items, to make sure that the trial_sequence is easy to understand. Rename:

• new_sketchpad to fixation_dot
• new_sketchpad_1 to neutral_gaze
• new_sketchpad_2 to gaze_cue
• new_sketchpad_3 to target
• new_keyboard_response to keyboard_response
• new_sampler to incorrect_sound
• new_logger to logger

The incorrect_sound item should only be executed if an error was made. To do this, we need to change the 'Run if …' statement to [correct] = 0 in the trial_sequence tab. This works, because the keyboard_response item automatically creates a correct variable, which is set to 1 (correct), 0 (incorrect), or undefined (this relies on the correct_response variable that was defined in Step 3). The square brackets indicate that correct should be interpreted as the name of a variable and not as text. To change a run-if statement, double click on it (shortcut: F3).

The trial_sequence now looks like Figure 10.

Figure 10. The trial_sequence at the end of Step 5.

Step 6: Draw the sketchpad items

The sketchpad items that we have created in Step 5 are still blank. It's time to do some drawing!

Set the background color to white

Click on fixation_dot in the overview area to open its tab. The sketchpad is still black, while the images that we have downloaded have a white background. Oops, we forgot to set the background color of the experiment to white (it is black by default)! Click on 'Tutorial: Gaze cuing' in the overview area to open the 'General properties' tab. Change 'Foreground color' to 'black' and 'Background color' to 'white'.

Draw the fixation dot

Go back to the fixation_dot by clicking on fixation_dot in the overview. Now select the fixation-dot element by clicking on the button with the crosshair. If you move your cursor over the sketchpad, you can see the screen coordinates in the top-right. Set the (foreground) color to 'black'. Click on the center of the screen (0, 0) to draw a central fixation dot.

Finally, change the 'Duration' field from 'keypress' to '745', because we want the fixation dot to be presented for 750 ms. Wait ... why didn't we just specify a duration of 750 ms? The reason for this is that the actual display-presentation duration is always rounded up to a value that is compatible with your monitor's refresh rate. This may sound complicated, but for most purposes the following rules of thumb are sufficient:

1. Choose a duration that is possible given your monitor's refresh rate. For example, if your monitor's refresh rate is 60 Hz, it means that every frame lasts 16.7 ms (= 1000 ms/60 Hz). Therefore, on a 60 Hz monitor, you should always select a duration that is a multiple of 16.7 ms, such as 16.7, 33.3, 50, 100, etc.
2. In the duration field of the sketchpad specify a duration that is a few milliseconds less than what you're aiming for. So if you want to present a sketchpad for 50 ms, choose a duration of 45. If you want to present a sketchpad for 1000 ms, choose a duration of 995. Etcetera.

Draw the neutral gaze

Open the neutral_gaze sketchpad. Now select the image tool by clicking on the button with the moon-mountain-landscape-like icon. Click on the center of the screen (0, 0). The 'Select file from pool' dialog will appear. Select the file gaze_neutral.png and click on the 'Select' button. The neutral gaze image will now stare at you from the center of the screen! Finally, like before, change the 'Duration' field from 'keypress' to '745'. (And note again that this means a duration of 750 ms on most monitors!)

Draw the gaze cue

Open the gaze_cue sketchpad, and again select the image tool. Click on the center of the screen (0, 0) and select the file gaze_left.png.

Obviously, we are not done yet, because the gaze cue should not always be 'left', but should depend on the variable gaze_cue, which we have defined in Step 3. However, by drawing the gaze_left.png image to the sketchpad, we have generated a script that needs only a tiny modification to make sure that the proper image is shown. Click on the 'Select view' button at the top-right of the gaze_cue tab and select 'View script'. You will now see the script that corresponds to the sketchpad that we have just created:

set duration keypress
set description "Displays stimuli"
draw image center=1 file="gaze_left.png" scale=1 show_if=always x=0 y=0 z_index=0

The only thing that we need to do is replace gaze_left.png with gaze_[gaze_cue].png. This means that OpenSesame uses the variable gaze_cue (which has the values left and right) to determine which image should be shown.

While we are at it, we might as well change the duration to '495' (rounded up to 500!). The script now looks like this:

set duration 495
set description "Displays stimuli"
draw image center=1 file="gaze_[gaze_cue].png" scale=1 show_if=always x=0 y=0 z_index=0

Click the 'Apply and close' button at the top right to apply your changes to the script and return to the regular item controls. OpenSesame will warn you that the image cannot be shown, because it is defined using variables, and a placeholder image will be shown instead. Don't worry, the correct image will be shown during the experiment!

Draw the target

We want three objects to be part of the target display: the target letter, the distractor letter, and the gaze cue (see Figure 1). As before, we will start by creating a static display using the sketchpad editor. After this, we will only need to make minor changes to the script so that the exact display depends on the variables.

Click on target in the overview to open the target tab and like before, draw the gaze_left.png image at the center of the screen. Now select the draw text tool by clicking on the button with the 'A' icon. Change the foreground color to 'black' (if it isn't already). The default font size is 18 px, which is a bit small for our purpose, so change the font size to 32 px. Now click on (-320, 0) in the sketchpad (the X-coordinate does not need to be exactly 320, since we will change this to a variable anyway). Enter "[target_letter]" in the dialog that appears, to draw the target letter (when drawing text, you can use variables directly). Similarly, click on (320, 0) and draw an 'X' (the distractor is always an 'X').

Now open the script editor by clicking on the 'Select view' button at the top-right of the tab and selecting 'View script'. The script looks like this:

set duration keypress
set description "Displays stimuli"
draw image center=1 file="gaze_left.png" scale=1 show_if=always x=0 y=0 z_index=0
draw textline center=1 color=black font_bold=no font_family=mono font_italic=no font_size=32 html=yes show_if=always text="[target_letter]" x=-320 y=0 z_index=0
draw textline center=1 color=black font_bold=no font_family=mono font_italic=no font_size=32 html=yes show_if=always text=X x=320 y=0 z_index=0

Like before, change gaze_left.png to gaze_[gaze_cue].png. We also need to make the position of the target and the distractor depend on the variables target_pos and dist_pos respectively. To do this, simply change -320 to [target_pos] and 320 to [dist_pos]. Make sure that you leave the 0, which is the Y-coordinate. The script now looks like this:

set duration "keypress"
set description "Displays stimuli"
draw image center=1 file="gaze_[gaze_cue].png" scale=1 show_if=always x=0 y=0 z_index=0
draw textline center=1 color=black font_bold=no font_family=mono font_italic=no font_size=32 html=yes show_if=always text="[target_letter]" x=[target_pos] y=0 z_index=0
draw textline center=1 color=black font_bold=no font_family=mono font_italic=no font_size=32 html=yes show_if=always text=X x=[dist_pos] y=0 z_index=0

Click on the 'Apply and close' button to apply the script and go back to the regular item controls.

Finally, set the 'Duration' field to '0'. This does not mean that the target is presented for only 0 ms, but that the experiment will advance to the next item (the keyboard_response) right away. Since the keyboard_response waits for a response, but doesn't change what's on the screen, the target will remain visible until a response has been given.

Remember to save your experiment regularly.

Step 7: Configure the keyboard response item

Click on keyboard_response in the overview to open its tab. You see three options: Correct response, Allowed responses, and Timeout.

We have already set the correct_response variable in Step 3. Unless we explicitly specify a correct response, OpenSesame automatically uses the correct_response variable if it is available. Therefore, we don't need to change the 'Correct response' field here.

We do need to set the allowed responses. Enter 'z;m' in the allowed-responses field (or other keys if you have chosen different response keys). The semicolon is used to separate responses. The keyboard_response now only accepts 'z' and 'm' keys. All other key presses are ignored, with the exception of 'escape', which pauses the experiment.

We also want to set a timeout, which is the maximum interval that the keyboard_response waits before deciding that the response is incorrect and setting the 'response' variable to 'None'. '2000' (ms) is a good value.

The keyboard_response now looks like Figure 12.

Figure 12. The keyboard_response at the end of Step 7.

Step 8: Configure the incorrect (sampler) item

The incorrect_sound item doesn't need much work: We only need to select the sound that should be played. Click on incorrect_sound in the overview to open its tab. Click on the 'Browse' button and select incorrect.ogg from the file pool.

The sampler now looks like Figure 13.

Figure 13. The incorrect_sound item at the end of Step 8.

Step 9: Configure the variable logger

Actually, we don't need to configure the variable logger, but let's take a look at it anyway. Click on logger in the overview to open its tab. You see that the option 'Log all variables (recommended)' is selected. This means that OpenSesame logs everything, which is fine.

Step 10: Draw the feedback item

After every block of trials, we want to present feedback to the participant to let him/ her know how well he/ she is doing. Therefore, in Step 2, we added a feedback item, simply named feedback to the end of block_sequence.

Click on feedback in the overview to open its tab, select the draw text tool, change the foreground color to 'black' (if it isn't already), and click at (0, 0). Now enter the following text:

End of block

Your average response time was [avg_rt] ms
Your accuracy was [acc] %

Press any key to continue

Because we want the feedback item to remain visible as long as the participant wants (i.e. until he/ she presses a key), we leave 'Duration' field set to 'keypress'.

The feedback item now looks like Figure 14.

Figure 14. The feedback item at the end of Step 10.

Step 11: Set the length of the practice phase and experimental phase

We have previously created the practice_loop and experiment_loop items, which both call block_sequence (i.e., a block of trials). However, right now they call block_sequence only once, which means that both the practice and the experimental phase consist of only a single block of trials.

Click on practice_loop to open its tab and set 'Repeat' to '2.00'. This means that the practice phase consists of two blocks.

Click on experimental_loop to open its tab and set 'Repeat' to '8.00'. This means that the experimental phase consists of eight blocks.

Step 12: Write the instruction, end_of_practice and end_of_experiment forms

I think you can handle this step your own! Simply open the appropriate items and add some text to present instructions, an end-of-practice message, and an end-of-experiment message.

Step 13: Run the experiment!

You're done! Click on the 'Run in window' (shortcut: Ctrl+W) or 'Run fullscreen' (shortcut: Ctrl+R) buttons in the toolbar to run your experiment.

Finally: Some general considerations regarding timing and backend selection

In the 'General properties' tab of the experiment (the tab that you open by clicking on the experiment name), you can select a backend. The backend is the layer of software that controls the display, input devices, sound, etc. Most experiments work with all backends, but there are reasons to prefer one backend over the other, mostly related to timing. Currently there are four backends (depending on your system, not all three may be available):

• legacy -- a 'safe' backend, based on PyGame. It provides reliable performance on most platforms, but, due to a lack of hardware acceleration, its timing properties are not as good as those of the other backends.
• psycho -- a hardware accelerated backend, based on PsychoPy (Peirce, 2007).
• xpyriment -- a hardware-accelerated backend, based on Expyriment (Krause & Lindeman, 2013)
• droid -- a backend that allows you to run your experiment on an Android device with the OpenSesame runtime for Android.

See also:

• https://osdoc.cogsci.nl/3.2/manual/timing
• https://osdoc.cogsci.nl/3.2/manual/backends

See also Damian (2010), Brand and Bradley (2011), and Ulrich and Giray (1989) for some general considerations about when to worry, and when not to worry about timing.

References