PSYC 241. Introduction to Programming Web-Based Experiments for Psychology Research

Prof. Timothy Brady
Wednesdays 9am-12pm
Crick conference room, 3rd floor Mandler Hall
Bring a laptop every day!

About this class: The core mission of this class is to help everybody be able to write web-based psychology studies that go beyond simple surveys; collect data on these experiments using crowdsourcing platorms like MTurk/Prolific/Online SONA; and download/access the data from the experiments for analysis offline in R/MATLAB/Python. It is broadly aimed at people who have some basic programming skills but have not previously written experiments for the web, although if you have no programming experience at all you should be able to keep up and hopefully get something out of the class as well. If you have ever written your own web-based experiment using HTML/Javascript, or have existing Javascript experience, this class is likely to be too introductory for you. Because of the nature of skill-based courses like this, we won't be able to learn everything in a group setting and will do a fair amount of independent work in the classroom starting in approximately week 3, including spending classroom time working on final projects -- this will allow everybody to get help from me and from each other.

January 8th

Slides from today: PDF PPT

Who am I
Who everybody else is and what everybody hopes to learn
Broad structure of the class
Basics of the web (browsers, servers, DNS, HTTP, etc.)
Client-side technologies: HTML/CSS/Javascript/JQuery/jsPsych
Server-side technologies: PHP, Python, psiTurk
Glitch.com Test
Introduction to Mechanical Turk

Terminology
The website
The sandbox
Demographics

Introduction to Prolific
Introduction to Online SONA
IRB issues and how payments work on MTurk/Prolific
Using MTurk's interface to post our first experiment on the sandbox
Resources for getting a head start on basic HTML/CSS at home:
https://www.khanacademy.org/computing/computer-programming/html-css
https://internetingishard.com/html-and-css/introduction/
https://scrimba.com/g/ghtmlcss

January 15

For today, read Stewart et al. 2017, TiCS PDF

Overview of some non-code based ways to run web experiments
- MTurk interface
- Testable.org
- TELLab
- PsychoPy/ Pavlovia
- Survey Monkey
- Qualtrics
Design issues when running web experiments
Coding our first simple experiment "by hand" (just jQuery):
- DIV vs SPAN, block vs. inline
- Basic CSS boxes: margin, padding, border
- SCRIPT - functions, events, hiding and showing elements
Some MTurk resources that might be handy:
Code we wrote:

<div id="testDiv">
   This is a brief workbook test.
</div>

<div>
  These are your instructions.  These are your instructions.  These are your instructions. These are your instructions. These are your instructions. These are your instructions. These are your instructions. These are your instructions. These are your instructions. These are your instructions. These are your instructions. These are your instructions. These are your instructions. 
  <img class="instructionImg" src="http://bradylab.ucsd.edu/images/brain.png">
</div>

<div>
  These are your instructions too. 
  <img class="instructionImg" src="http://bradylab.ucsd.edu/images/brain.png">
</div>

<p>
  Hello <strong>Test</strong> this is Tim
</p>
<p id="paragraph2" class="stimulus">
  Hello 
</p>
<p class="stimulus, tim">
  Hello this is Tim too - stim2
</p>
<strong class="tim">test</strong> 
hello

January 22

Homework for today: Make a completely blank document (but copy and paste the jQuery line into it!). Make a single DIV with some text in it, and give it an ID. Then make it 500px wide, centered and have a gray background. Finally, make it so that when you click it, it disappears.

Overview of HTML, CSS, Javascript syntax and theory
Coding a more complex experiment using just jQuery
Making a full "template" for many basic studies

Build basic "frame" of study in HTML and CSS
Use Javascript to construct a trial structure
Use timers and keypress events to move through the trial structure
Gather data for saving

What we wrote on the whiteboard:
HTML:

CSS:

Javascript:

jQuery:
Code we wrote:

<html>
  <head>
    <title>Our test experiment</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <style>
      body {
        font-size: 16pt;
        font-family: Helvetica, Arial;
      }
      #instructions {
        width: 400px;
        margin-left: auto;
        margin-right: auto;
      }
      button {
        font-size: 16pt;
        font-weight: bold;
      }
      .trial {
        display: none;
        font-size: 40pt;
        text-align: center;
      }
      #done {
        display: none;
      }
    </style>
  </head>
  <body>
    
    <div id="instructions">
      <p>
        You will see a sequence of words and non-words. For each one, decide as
        quickly as you can whether it is a word or non-word. If it is a word,
        press 'w'. If it is not a word, press 'p'.
      </p>
      <button id="startExperimentButton">
        Click here when you are ready to proceed.
      </button>
    </div>

<div class="trial" id="trial_1">
      cactus
    </div>
    <div class="trial" id="trial_2">
      apple
    </div>
    <div class="trial" id="trial_3">
      hyup
    </div>
    <div class="trial" id="trial_4">
      fern
    </div>
    <div class="trial" id="trial_5">
      foil
    </div>

<div id="done">
      Thanks for doing a great job!
    </div>

/* This gets called when they click the button 
        on the instructions page */
      function StartExperiment() {
        $("#instructions").hide();
        ShowTrial();
      }

function ShowTrial() {
        setTimeout(Runs500msAfterTrial, 500);
      }

function Runs500msAfterTrial() {
        /* This runs 500ms after ShowTrial is called */
        $("#trial_" + curTrial).show();
        curTrialStarted = new Date();

/* Set up key listener */
        $(document).on("keydown.wordOrNot", PressedKey);
        console.log("after key listener");
      }

function PressedKey(event) {
        /*  console.log(event.which); */
        console.log("stimulus", $("#trial_" + curTrial).text());
        if (event.which == 87) {
          /* w for word */
          console.log("said word!");
        }
        if (event.which == 80) {
          /* p for non-word */
          console.log("said non-word");
        }
        if (event.which == 80 || event.which == 87) {
          /* Calculate reaction time */
          var curTime = new Date();
          console.log(curTime - curTrialStarted);

/* Hide current word and stop listening for keys */
          $("#trial_" + curTrial).hide();
          $(document).off("keydown.wordOrNot");

/* Are there more trials left? */
          curTrial = curTrial + 1;
          if (curTrial <= nTrials) {
            setTimeout(ShowTrial, 500);
          } else {
            $("#done").show();
          }
        }
      }

$("#startExperimentButton").on("click", StartExperiment);
    </script>
  </body>
</html>

January 29

Homework for today: Take our experiment template and remix it -- don't edit directly! And make it so that if they are correct about it being a word or non-word, it just goes on to the next trial as usual. But if they are incorrect, it waits an extra 2 seconds to go on. And add some words -- make it 10 trials total.

Expanding our experiment template:

Reminder about functions (calling with () vs. passing as variables):
Objects -- like structs (but you can put functions in them):
Arrays/lists:
Loops:
Making a trial structure and re-using a DIV

Saving data to a server
Code we wrote:

<html>
  <head>
    <title>Our test experiment</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <style>
      body {
        font-size: 16pt;
        font-family: Helvetica, Arial;
      }
      #instructions {
        width: 400px;
        margin-left: auto;
        margin-right: auto;
      }
      button {
        font-size: 16pt;
        font-weight: bold;
      }
      #trialDiv {
        display: none;
        font-size: 40pt;
        text-align: center;
      }
      #done {
        display: none;
      }
    </style>
  </head>
  <body>
    
    <div id="instructions">
      <p>
        You will see a sequence of words and non-words. For each one, decide as
        quickly as you can whether it is a word or non-word. If it is a word,
        press 'w'. If it is not a word, press 'p'.
      </p>
      <button id="startExperimentButton">
        Click here when you are ready to proceed.
      </button>
    </div>

<div id="trialDiv"></div>

<div id="done">
      Thanks for doing a great job!
    </div>

function ShowTrial() {
        setTimeout(Runs500msAfterTrial, 500);
      }

function Runs500msAfterTrial() {
        /* This runs 500ms after ShowTrial is called */
        
        /* set to the right word for this trial */
        $("#trialDiv").text(trialList[curTrial].textToShow); 
        $("#trialDiv").show();
        curTrialTimeStarted = new Date();

/* Set up key listener */
        $(document).on("keydown.wordOrNot", PressedKey);
      }

function PressedKey(event) {
        if (event.which == 87) {
          /* w for word */
          trialList[curTrial].saidWord = true; /* add a new property to the object */
        }
        if (event.which == 80) {
          /* p for non-word */
          trialList[curTrial].saidWord = false;
        }
        if (event.which == 80 || event.which == 87) {
          /* Calculate reaction time */
          var curTime = new Date();
          trialList[curTrial].RT = curTime - curTrialTimeStarted;
          
          /* Hide current word and stop listening for keys */
          $("#trialDiv").hide();
          $(document).off("keydown.wordOrNot");

/* Are there more trials left? */
          curTrial = curTrial + 1;
          if (curTrial < trialList.length) {
            setTimeout(ShowTrial, 500);
          } else {
            setTimeout(DoneWithExperiment, 500);
          }
        }
      }

function DoneWithExperiment() {
        $("#done").show();        
        var dataToServer = {
          id: subjectId,
          experimenter: "TimClassTest",
          experimentName: "WordOrNonWord",
          curData: JSON.stringify(trialList)
        };
        $.post("https://psyc241.ucsd.edu/Turk/save.php",
          dataToServer);
        
      }
      
      $("#startExperimentButton").on("click", StartExperiment);
    </script>
  </body>
</html>

February 5

Homework for today: Take our experiment template and remix it -- don't edit directly! Add a few more trials, and for each one, add another part to the trial list: which color each word is in (red or blue, half and half). Then when the word is shown, make it shown in that color instead of black.

Server-side scripting, working with data, and the format I usually save data in (JSON)
Server-side scripting
- Basics of PHP
- Basics of Python
- Explaining our generic template "saving data" server-side script
Very brief introduction to psiTurk as a way to host a server/store a database of data etc.
Loading JSON in R, MATLAB or Python for analysis

R code:

Loading multiple subjects at once - R code:

Matlab code:

Python 3:

Reviewing our existing code and modifying to use images as a refresher (from https://psyc241.ucsd.edu/Tim/Objects/)
Code we wrote:

<html>
  <head>
    <title>Our test experiment</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <style>
      body {
        font-size: 16pt;
        font-family: Helvetica, Arial;
      }
      #instructions {
        width: 400px;
        margin-left: auto;
        margin-right: auto;
      }
      button {
        font-size: 16pt;
        font-weight: bold;
      }
      #trialDiv {
        display: none;
        font-size: 40pt;
        text-align: center;
      }
      #done {
        display: none;
      }
      
      #preload {
        display: none;
      }
    </style>
  </head>
  <body>
    
    <div id="instructions">
      <p>
        You will see two images of objects, one on the left and one on the right. Press 'w' if the one on the left is larger in the real world, and 'p' if the one on the right is larger in the real world. So if left is an elephant, and right is a paperclip, press 'w'. If right is an elephant and left is a paperclip, press 'p'.
      </p>
      <button id="startExperimentButton">
        Click here when you are ready to proceed.
      </button>
    </div>

<div id="trialDiv">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj1.jpg" id="leftObject">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj2.jpg" id="rightObject">
    </div>
  
    <div id="done">
      Thanks for doing a great job!
    </div>
    
    <div id="preload">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj1.jpg">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj2.jpg">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj3.jpg">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj4.jpg">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj5.jpg">
      <img src="https://psyc241.ucsd.edu/Tim/Objects/obj6.jpg">
    </div>

function ShowTrial() {
        setTimeout(Runs500msAfterTrial, 500);
      }

function Runs500msAfterTrial() {
        /* This runs 500ms after ShowTrial is called */
        
        /* set to the right word for this trial */
        $("#leftObject").attr("src", "https://psyc241.ucsd.edu/Tim/Objects/obj" +
                              trialList[curTrial].leftObject + ".jpg");
        $("#rightObject").attr("src", "https://psyc241.ucsd.edu/Tim/Objects/obj" +
                              trialList[curTrial].rightObject + ".jpg");
        $("#leftObject").css("width", trialList[curTrial].leftObjSize);
        $("#rightObject").css("width", trialList[curTrial].rightObjSize);
        $("#trialDiv").show();
        curTrialTimeStarted = new Date();

/* Set up key listener */
        $(document).on("keydown.wordOrNot", PressedKey);
      }

function PressedKey(event) {
        if (event.which == 87) {
          /* w for left obj */
          trialList[curTrial].saidLeft = true; /* add a new property to the object */
        }
        if (event.which == 80) {
          /* p for right obj */
          trialList[curTrial].saidLeft = false;
        }
        if (event.which == 80 || event.which == 87) {
          /* Calculate reaction time */
          var curTime = new Date();
          trialList[curTrial].RT = curTime - curTrialTimeStarted;
          
          /* Hide current word and stop listening for keys */
          $("#trialDiv").hide();
          $(document).off("keydown.wordOrNot");

function DoneWithExperiment() {
        $("#done").show();        
        var dataToServer = {
          id: subjectId,
          experimenter: "TimClassTest",
          experimentName: "WhichObjectBigger",
          curData: JSON.stringify(trialList)
        };
        /* AJAX */
        $.post("https://psyc241.ucsd.edu/Turk/save.php",
          dataToServer);
      }
      
      $("#startExperimentButton").on("click", StartExperiment);
    </script>
  </body>
</html>

Discussing/planning your own experiments

February 12

jsPsych as a way to simplify the creation of basic experiments
Introduction to jsPsych
Coding a simple experiment in jsPsych
Code we wrote:

<html>
  <head>
    <title>Our test experiment</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <script src="https://psyc241.ucsd.edu/jspsych-6.1.0/jspsych.js"></script>
    <script src="https://psyc241.ucsd.edu/jspsych-6.1.0/plugins/jspsych-html-keyboard-response.js"></script>
    <script src="https://psyc241.ucsd.edu/jspsych-6.1.0/plugins/jspsych-image-slider-response.js"></script>
    <script src="https://psyc241.ucsd.edu/jspsych-6.1.0/plugins/jspsych-categorize-html.js"></script>
    <script src="https://psyc241.ucsd.edu/jspsych-6.1.0/plugins/jspsych-survey-likert.js"></script>
    <link
      href="https://psyc241.ucsd.edu/jspsych-6.1.0/css/jspsych.css"
      rel="stylesheet"
      type="text/css"
    />
    <style></style>
  </head>
  <body>
    <script>
      var trialList = [];
      var welcomeTrial = {
        type: "html-keyboard-response",
        stimulus: "Press a key to start the experiment."
      };
      trialList.push(welcomeTrial);

var instructionsTrials = {
        type: "html-keyboard-response",
        stimulus:
          "Press w for word, and n for non-word. Press a key to continue.",
        on_finish: saveData
      };
      trialList.push(instructionsTrials);

var categorizeTrial = {
        type: "categorize-html",
        prompt: "Press w for word, n for non-word",
        feedback_duration: 500,
        show_stim_with_feedback: false,
        choices: ["w", "n"],
        key_answer: 87,
        timeline: [
          { stimulus: "<b>dog</b>" },
          { stimulus: "<b>cat</b>" },
          { stimulus: "<b>phw</b>", key_answer: 78 }
        ]
      };
      trialList.push(categorizeTrial);

var scale_1 = [
        "Strongly Disagree",
        "Disagree",
        "Neutral",
        "Agree",
        "Strongly Agree"
      ];

var likertTrial = {
        type: "survey-likert",
        questions: [
          { prompt: "I loved this experiment", labels: scale_1 },
          { prompt: "I love this class", labels: scale_1 }
        ]
      };
      trialList.push(likertTrial);

Continuing work on your own experiments

February 19

INPUT, radio buttons, TEXTAREA
Posting a HIT on Turk, SONA and Prolific using our (final) template.
Turk:

<html>
  <head>
    <title>Judge objects</title>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    <script src="https://psyc241.ucsd.edu/jspsych-6.1.0/jspsych.js"></script>
    <style>
      body {
        font-size: 16pt;
        font-family: Helvetica, Arial;
      }
      #instructions {
        width: 400px;
        margin-left: auto;
        margin-right: auto;
      }
      input, button, textarea {
        font-size: 16pt;
        font-weight: bold;
      }
      #trialDiv {
        display: none;
        font-size: 40pt;
        text-align: center;
      }
      #done {
        display: none;
      }
      #debriefing {
        display: none;
      }
      #preload {
        display: none;
      }
    </style>
  </head>
  <body>
    
    <div id="instructions">
      <p>
        You will see two images of objects, one on the left and one on the
        right. Press 'w' if the one on the left is larger in the real world, and
        'p' if the one on the right is larger in the real world. So if left is
        an elephant, and right is a paperclip, press 'w'. If right is an
        elephant and left is a paperclip, press 'p'.
      </p>
      <button id="startExperimentButton">
        Click here when you are ready to proceed.
      </button>
    </div>

<div id="debriefing">
      <p>
        Enter your demographics information here:    
      </p>
      <p>
        Age: <input type="number" id="age">
      </p>
      <p>
        Gender:<br>
          <input type="radio" name="gender" value="male"> Male<br>
          <input type="radio" name="gender" value="female"> Female<br>
      </p>
      <p>
        Was the experiment fun? <br>
        <textarea id="wasItFun" rows="4" cols="50">
          
        </textarea>
      </p>
      
      <button id="doneWithDebriefing">
        Submit
      </button>
    </div>
    
    <div id="done">
      Thanks for doing a great job!
    </div>

/*
      var trialList = [
        { leftObject: 1, rightObject: 2, leftObjSize: 300, rightObjSize: 500 },
        { leftObject: 3, rightObject: 4, leftObjSize: 500, rightObjSize: 300 },
        { leftObject: 5, rightObject: 6, leftObjSize: 300, rightObjSize: 500 },
        { leftObject: 7, rightObject: 8, leftObjSize: 500, rightObjSize: 300 },
        { leftObject: 9, rightObject: 10, leftObjSize: 300, rightObjSize: 500 }
      ];*/
      
      var factors = {
          leftObject: [1,3,5,7,9],
          rightObject: [2,4,6,8,10],
          leftObjSize: [300,500],
          rightObjSize: [500,300]
      }
      //trialList = jsPsych.randomization.repeat(trialList, 2);
      trialList = jsPsych.randomization.factorial(factors, 1);
      
      var htmlStr = "";
      for (var i = 1; i <= 10; i++) {
        htmlStr =
          htmlStr +
          "<img src='https://psyc241.ucsd.edu/Tim/Objects/obj" +
          i +
          ".jpg'>";
      }
      $("#preload").html(htmlStr);

$("#startExperimentButton").on("click", StartExperiment);
      
      /* Could shuffle here? */

/* This gets called when they click the button 
        on the instructions page */
      function StartExperiment() {
        $("#instructions").hide();
        ShowTrial();
      }

function ShowTrial() {
        setTimeout(Runs500msAfterTrial, 500);
      }

function Runs500msAfterTrial() {
        /* This runs 500ms after ShowTrial is called */

/* set to the right word for this trial */
        $("#leftObject").attr(
          "src",
          "https://psyc241.ucsd.edu/Tim/Objects/obj" +
            trialList[curTrial].leftObject +
            ".jpg"
        );
        $("#rightObject").attr(
          "src",
          "https://psyc241.ucsd.edu/Tim/Objects/obj" +
            trialList[curTrial].rightObject +
            ".jpg"
        );
        $("#leftObject").css("width", trialList[curTrial].leftObjSize);
        $("#rightObject").css("width", trialList[curTrial].rightObjSize);
        $("#trialDiv").show();
        curTrialTimeStarted = new Date();

/* Set up key listener */
        $(document).on("keydown.wordOrNot", PressedKey);
      }

function PressedKey(event) {
        if (event.which == 87) {
          /* w for left obj */
          trialList[
            curTrial
          ].saidLeft = true; /* add a new property to the object */
        }
        if (event.which == 80) {
          /* p for right obj */
          trialList[curTrial].saidLeft = false;
        }
        if (event.which == 80 || event.which == 87) {
          /* Calculate reaction time */
          var curTime = new Date();
          trialList[curTrial].RT = curTime - curTrialTimeStarted;

/* Hide current word and stop listening for keys */
          $("#trialDiv").hide();
          $(document).off("keydown.wordOrNot");

Prolific:

<div id="debriefing">
      <p>
        Enter your demographics information here:    
      </p>
      <p>
        Age: <input type="number" id="age">
      </p>
      <p>
        Gender:<br>
          <input type="radio" name="gender" value="male"> Male<br>
          <input type="radio" name="gender" value="female"> Female<br>
      </p>
      <p>
        Was the experiment fun? <br>
        <textarea id="wasItFun" rows="4" cols="50"></textarea>
      </p>
      
      <button id="doneWithDebriefing">
        Submit
      </button>
    </div>
    
    <div id="done">
      Thanks for doing a great job!
    </div>

/*
      var trialList = [
        { leftObject: 1, rightObject: 2, leftObjSize: 300, rightObjSize: 500 },
        { leftObject: 3, rightObject: 4, leftObjSize: 500, rightObjSize: 300 },
        { leftObject: 5, rightObject: 6, leftObjSize: 300, rightObjSize: 500 },
        { leftObject: 7, rightObject: 8, leftObjSize: 500, rightObjSize: 300 },
        { leftObject: 9, rightObject: 10, leftObjSize: 300, rightObjSize: 500 }
      ];*/
      
      var factors = {
          leftObject: [1],
          rightObject: [2],
          leftObjSize: [300,500],
          rightObjSize: [500,300]
      }
      //trialList = jsPsych.randomization.repeat(trialList, 2);
      trialList = jsPsych.randomization.factorial(factors, 1);
      
      var htmlStr = "";
      for (var i = 1; i <= 10; i++) {
        htmlStr =
          htmlStr +
          "<img src='https://psyc241.ucsd.edu/Tim/Objects/obj" +
          i +
          ".jpg'>";
      }
      $("#preload").html(htmlStr);

$("#startExperimentButton").on("click", StartExperiment);
      
      /* Could shuffle here? */

/* This gets called when they click the button 
        on the instructions page */
      function StartExperiment() {
        $("#instructions").hide();
        ShowTrial();
      }

function ShowTrial() {
        setTimeout(Runs500msAfterTrial, 500);
      }

function Runs500msAfterTrial() {
        /* This runs 500ms after ShowTrial is called */

/* Set up key listener */
        $(document).on("keydown.wordOrNot", PressedKey);
      }

function PressedKey(event) {
        if (event.which == 87) {
          /* w for left obj */
          trialList[
            curTrial
          ].saidLeft = true; /* add a new property to the object */
        }
        if (event.which == 80) {
          /* p for right obj */
          trialList[curTrial].saidLeft = false;
        }
        if (event.which == 80 || event.which == 87) {
          /* Calculate reaction time */
          var curTime = new Date();
          trialList[curTrial].RT = curTime - curTrialTimeStarted;

/* Hide current word and stop listening for keys */
          $("#trialDiv").hide();
          $(document).off("keydown.wordOrNot");

SONA: