Logging and reading data files

Always triple check whether your data has been logged correctly before running your experiment!

Using the logger item

OpenSesame will not log your data automatically. Instead, you need to insert a logger item, typically at the end of your trial sequence.

Figure 1. The logger item.

The simplest way to use the logger is by leaving the option 'Automatically log all variables' enabled. That way, all variables that OpenSesame knows about are written the log file, except for those that are explicitly excluded (see below).

You can explicitly include which variables you want to log. The main reason for doing so is when you find that some variables are missing (because OpenSesame did not automatically detect them), or if you have disabled the option 'Automatically log all variables',

You can also explicitly exclude certain variables from the log file. The main reason for doing so is to keep the log files clean by excluding variables that are generally not useful.

In general, you should create only one logger item, and reuse that logger at different locations in your experiment if necessary (i.e. use linked copies of the same logger item). If you create multiple loggers (rather than using a single logger multiple times), they will all write to the same log file, and the result will be a mess!

Using Python inline script

You can write to the log file using the log object:

log.write('This will be written to the log file!')

For more information, see:

• https://osdoc.cogsci.nl/4.0/manual/python/log

You should generally not write to the log file directly and use a logger item at the same time; doing so will result in messy log files.

Format of the data files

If you have used the standard logger item, data files are in the following format format (simply standard csv):

• plain-text
• comma-separated
• double-quoted (literal double-quotes are escaped with backward slashes)
• unix-style line endings
• UTF-8 encoded
• column names on the first row

Which variables are logged?

By default, variables that are defined in the user interface, such as columns in a loop table or response variables are always logged.

By default, variables that are defined in an inline_script or inline_javascript are logged if they are numbers (int and float), strings (str and bytes), and None values. This is to avoid log files from becoming unreasonably large due to logging of long lists and other large values. (As of OpenSesame 4.0, there is no longer a need to use the var (Python) or vars (JavaScript) object.)

If you want to explicitly log a variable that is not logged by default, you can use the 'Include' field in the logger item.

Reading and processing data files

In Python with pandas or DataMatrix

In Python, you can use pandas to read csv files.

import pandas
df = pandas.read_csv('subject-1.csv')
print(df)

Or DataMatrix:

from datamatrix import io
dm = io.readtxt('subject-1.csv')
print(dm)

In R

In R, you can simply use the read.csv() function to read a single data file.

df = read.csv('subject-1.csv', encoding = 'UTF-8')
head(df)

In addition, you can use the read_opensesame() function from the readbulk package to easily read and merge multiple data files into one large data frame. The package is available on CRAN and can be installed via install.packages('readbulk').

# Read and merge all data files stored in the folder 'raw_data'
library(readbulk)
df = read_opensesame('raw_data')

In JASP

JASP, an open-source statistics package, opens csv files straight away.

In LibreOffice Calc

If you open a csv file in LibreOffice Calc, you have to indicate the exact data format, as indicated in Figure 2. (The default settings are often correct.)

Figure 2.

In Microsoft Excel

In Microsoft Excel, you need to use the Text Import Wizard.

Merging multiple data files into one large file

For some purposes, such as using pivot tables, it may be convenient to merge all data files into one large file. With Python DataMatrix, you can do this with the following script:

import os
from datamatrix import DataMatrix, io, operations as ops

# Change this to the folder that contains the .csv files
SRC_FOLDER = 'student_data'
# Change this to a list of column names that you want to keep
COLUMNS_TO_KEEP = [
    'RT_search',
    'load',
    'memory_resp'
]


dm = DataMatrix()
for basename in os.listdir(SRC_FOLDER):
    path = os.path.join(SRC_FOLDER, basename)
    print('Reading {}'.format(path))
    dm <<= ops.keep_only(io.readtxt(path), *COLUMNS_TO_KEEP)
io.writetxt(dm, 'merged-data.csv')

Logging in OSWeb

When you run an experiment in a browser with OSWeb, logging works differently from when you run an experiment on the desktop.

Specifically, when you launch an OSWeb experiment directly from within OpenSesame, the log file is downloaded at the end of the experiment. This log file is in .json format. When you launch an OSWeb experiment from JATOS, there is no log file as such, but rather all data is sent to JATOS from where it can be downloaded.

See also:

• https://osdoc.cogsci.nl/4.0/manual/osweb/workflow