Methods and Guidelines
From EuroVR Knowledge Base
There are a variety of methods and guidelines for designing and evaluating virtual environments as listed below.
Contents |
Design Methods
Evaluation Methods
There a large number of evaluation methods which have been used to assess various aspects of VR/VEs. These methods include those assessing usability, user interaction and behaviour, user performance and outcomes, user experience, sickness and physical ergonomics, physical and social presence, and psychophysical and psychophysiological measures.
General Usability
- Cognitive walkthrough
- Formative Evaluation (both formal and informal)
- Sequential Evaluation
- Summative Evaluation
- User-based statistical evaluation
- Testbed evaluation
- Heuristic or guidelines-based expert evaluation
- Inspection Tool (VIEW-IT)
- MAUVE (Multi-Attribute Usability evaluation tool for Virtual Environments)
- Usability Questionnaires
- VRUSE questionnaire (Kalawsky, 1999)
- Verbal protocol
- Cooperative Evaluation
- Transparency measure
- Quality Evaluation of Projection-Based VR Displays
User Interactions and Behaviour
- Behaviour during VE viewing
- Ethnographic Approach
- Experience Sampling Method (ESM)
- Video analysis
- Interaction Analysis
- Facial Expression Study
- Social Responses
- Multiple Activity Analysis
- Observation
User Performance and Outcomes
- Completion Time and Error Rate
- Traditional experimental design
- Number of Actions
- Secondary Task Performance
- National Aeronautics and Space Administration Task Load Index (NASA-TLX)
- Subjective Workload Assessment Technique (SWAT)
- Cost Benefit Analysis
- Cost effectiveness
- Socio economic consolidation
- Transfer of skills
User Experience
- Background information questionnaire
- Enjoyment Questionnaire
- Focus Groups
- Free-Format Self Reports
- Interview techniques
- Repertory Grid Analysis
- Post-immersion assessment of experience
Sickness and Physical Ergonomics
- Individual characteristics questonnaire
- Short Symptom Checklist (SSC)
- Simulator Sickness Questionnaire (SSQ)
- Stress Arousal Checklist (SACL)
- Physical discomfort questionnaire
- Ergonomics Questionnaire
- Rapid Upper Limb Assessment (RULA)
Physical and Social Presence
- Presence questionnaire (PQ)
- Igroup Presence Questionnaire (IPQ)
- Presence – Cho et al. (2003)
- Presence – Barfield and Weghorst (1993)
- Presence – Dinh et al. (1999)
- Presence – Gerhard et al. (2001)
- ITC Sense of Presence Inventory (ITC-SOPI)
- Presence – Kim and Biocca (1997)
- Presence – Krauss et al. (2001)
- Presence – Murray et al. (2000)
- Presence – Nichols et al. (2000)
- Object Presence Questionnaire (OPQ)
- Presence & Realism – Parent, A. (1998)
- Reality Judgment and Presence Questionnaire
- Swedish User-Viewer Presence Questionnaire
- Continuous Assessment of Presence
- Presence Probe
- Breaks in Presence (BIPs)
- Duration Estimation
- Memory Characteristic Questionnaire (MCQ)
- Attention/Awareness study
- Spatial Memory study
- Spatial Memory Awareness States
- Gravity-Referenced Eye Level (GREL)
- Presence – Lombard & Ditton (2000)
- Presence – Nowak & Biocca (2003)
- Presence – Schroeder et al. (2001)
- Presence – Thie & Van Wijk (1998)
- Autoconfrontation Method for Presence
- Slater-Usoh-Steed Questionnaire (SUS)
- Presence Questionnaire
- Free-Modulus Magnitude Estimation
- Presence - Bailenson et al. (2001)
- Presence - Basdogan et al. (2000)
- CMC Questionnaire/Social Presence and Privacy Questionnaire (SPPQ)
- GlobalEdQuestionnaire
- IPO Social Presence Questionnaire (IPO-SPQ)
- Networked Minds Questionnaire
- Para-Social Presence Questionnaire
- Semantic Differential Technique
- Content Analysis for Presence
- Postural Responses
- Pointing (Conflicting Cues)
- Reflex Responses
Psychophysical and Psychophysiological Measures
- Paired Comparison
- Virtual Reality Turing Test
- Subjective Tilt Angle
- Psychophysical technique to measure the Just Noticeable Difference (JND) or the Point of Subjective Equality (PSE)
- Cross-Modality Matching (CMM)
- Nulling
- Facial Electromyography (EMG)
- Electromyography (EMG)
- Functional Magnetic Resonance Imaging (fMRI)
- Electroencephalography (EEG)
- Ocular response (including eye blink rate, blink duration and lid closure amplitude)
- Eye tracking
- Event related potentials (ERPs)
- Cardiovascular respiratory system
- Dermal response system
- Respiratory response system (including respiration rate, tidal volume)
- Endocrine response
Restricted Area
- INTUITION review on evaluation methods - report reviewing the methods listed above and discussing issues to consider when choosing a method.
- Evaluation Methods Usage in Projects - report proving feedback from INTUITION partners regarding their use of different evaluation methods.
- Design Methods Usage in Projects - report proving feedback from INTUITION partners regarding their use of different design methods.
- Lessons Learned - This section presents a collection of practical advice, suggestions and guidelines obtained from the working experience of INTUITION partners in a wide variety of different applications.
