Appendix A: Printed Materials for VisAGE Usability Testing

Usability Testing of NASA/Goddard’s VisAGE Prototype

Date: _________________ Participant Number: __________

Participant Information:

Age: _______ M or F ______

Vision: ______ I do not wear glasses or contacts. ______ I wear glasses or contacts.

Color vision: _______ I have normal color vision. ______ My color vision is deficient in some way.

(please explain on back of this sheet)

Please mark a point on each of the following rating scales:

General computer experience:

|------------------------------------------|------------------------------------------|

Totally inexperienced Moderate experience Expert level

Word processing experience:

|------------------------------------------|------------------------------------------|

Totally inexperienced Moderate experience Expert level

 

Programming experience:

|------------------------------------------|------------------------------------------|

Totally inexperienced Moderate experience Expert level

Electronic spreadsheet experience:

|------------------------------------------|------------------------------------------|

Totally inexperienced Moderate experience Expert level

 

Computer graphics experience:

|------------------------------------------|------------------------------------------|

Totally inexperienced Moderate experience Expert level

 

Experience with 3D games and other 3D applications:

|------------------------------------------|------------------------------------------|

Totally inexperienced Moderate experience Expert level

Instructions: During the testing session, you will be asked to respond to questions about various visual representations of data. It is important that you answer based on your perception of the displays you will see. You should be able to answer every question. The research assistant will go over the procedure with you but is not permitted to help you answer the questions. The research assistant will set up the visualizations and record your answers. Please proceed when you are ready. Thank you for participating!

Participant Number: __________

Heater Source #1= HTR1 and HTR2 displayed together

Questions for 2D Text with Data Feed = Heater Source #1: Please click "Start Test"

    1. What is the value displayed for Heater 2 ? ____________________

 

b. What is the value displayed for Heater 1? ____________________

 
    1. Over the last three time periods, the temperature for Heater 1 has been going:

____ up ____ down _____ staying about the same ____ can’t tell

d. I would expect the temperature for Heater 1 to _____ go up, _____ go down, or ____ stay about the same in the next time period. _____ Cannot predict
    1. Over the last three time periods, the temperature for Heater 2 has been going:

____ up _____ down _____ staying about the same ____ can’t tell

f. I would expect the temperature for Heater 2 to ____ go up, ____ go down, or _____ stay about the same in the next time period. _____ Cannot predict

Please click "Stop Test" when you have completed these questions.

g. Other comments?

 

Participant Number: __________

Heater Source #1= HTR1 and HTR2 displayed together

 

Questions for 3D Text with Data Feed = Heater Source #1 Click "Start Test"

a. What is the value displayed for Heater 2 ? ____________________

b. What is the value displayed for Heater 1? ____________________
    1. Over the last three time periods, the temperature for Heater 1 has been going:

____ up ____ down _____staying about the same ____ can’t tell

d. I would expect the temperature for Heater 1 to _____ go up, _____ go down, or _____ stay about the same in the next time period. _____ Cannot predict
    1. Over the last three time periods, the temperature for Heater 2 has been going: ____ up _____ down _____ staying about the same ____ can’t tell
    1. I would expect the temperature for Heater 2 to _____ go up, _____ go down, or _____ stay about the same in the next time period. _____ Cannot predict.

Please click "Stop Test," when you have completed these questions.

    1. Does the third dimension aid your understanding of the data? ___Yes ___ No ; Explain:

 

 

 

h. Other comments?

Participant Number: __________

 

Questions on other features of the 3D text display:

a. Does the color coding imply anything to you? _____ Yes ____ No _____________Explain

 
    1. Please select the zoom control and zoom the display in and out several times. Please rate the ease of using the zoom control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

c. Please select the rotate control and rotate the display as much as seems possible. Please rate the ease of using the zoom control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

d. Please select the pan control and pan the display around the screen. Please rate the ease of using the pan control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

e. Please select the undo-move control and apply it to your previous move. Please rate the ease of using the undo-move control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

f. Please select the redo-move control and apply it to your previous undo move. Please rate the ease of using the redo-move control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

g. Other comments?

Participant Number: __________

 

Questions on other features of the 3D bar chart:

a. Does the color coding imply anything to you? _____ Yes ____ No _____________Explain

 

b. Please select the zoom control and zoom the bar chart in and out several times. Please rate the ease of using the zoom control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

c. Please select the rotate control and rotate the bar chart as much as seems possible. Please rate the ease of using the zoom control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

d. Please select the pan control and pan the bar chart around the screen. Please rate the ease of using the pan control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

e. Please select the undo-move control and apply it to your previous move. Please rate the ease of using the undo-move control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

f. Please select the redo-move control and apply it to your previous undo move. Please rate the ease of using the redo-move control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

    1. Does the third dimension aid your understanding of the data? ___ Yes ___ No; Explain:

 

 

h. Other comments?

Participant Number: __________

 

Questions on other features of the 3D strip chart:

a. Does the color coding imply anything to you? _____ Yes ____ No _____________Explain

 

b. Please select the zoom control and zoom the strip chart in and out several times. Please rate the ease of using the zoom control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

c. Please select the rotate control and rotate the strip chart as much as seems possible. Please rate the ease of using the zoom control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

d. Please select the pan control and pan the strip chart around the screen. Please rate the ease of using the pan control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

e. Please select the undo-move control and apply it to your previous move. Please rate the ease of using the undo-move control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

f. Please select the redo-move control and apply it to your previous undo move. Please rate the ease of using the redo-move control:

|-------------------------------------------------|-----------------------------------------------------|

Very easy Somewhat hard Very hard

 

g. Other comments?

 

 

 

Participant Number: __________

Heater Source #1= HTR1 and HTR2 displayed together

 

Questions for 2D Bar Chart with Data Feed = Heater Source #1: Click "Start Test"

    1. Does the display need to be repositioned ? ______ No _____ Yes

If No, why not? ______________________________________________________________

If Yes, what control(s) would you recommend using? _________________________________

b. What is the temperature of Heater 1 ? ____________________

c. What is the temperature of Heater 2? ____________________

d. The temperature for Heater 2 is going: ____ up ____ down _____ staying about the same ____ can’t tell

    1. The temperature for Heater 1 is going: ____ up ____ down _____ staying about the same ____ can’t tell

Please click "Stop Test" when you have completed these questions.

f. Other comments?

 

 

Participant Number: __________

Heater Source #1= HTR1 and HTR2 displayed together

 

Questions for 3D Bar Chart with Data Feed = Heater Source #1 Click "Start Test"

    1. Does the display need to be repositioned ? ______ No _____ Yes

If No, why not? ______________________________________________________________

If Yes, what control(s) would you recommend using? _________________________________

b. What is the temperature of Heater 1 ? ____________________

c. What is the temperature of Heater 2? ____________________

  1. The temperature for Heater 2 is going: ____ up ____ down _____ staying about the same

_____ can’t tell

    1. The temperature for Heater 1is going: ____ up ____ down _____ staying about the same ____ can’t tell

Please click "Stop Test" when you have completed these questions.

Participant Number: __________

Heater Source #1= HTR1 and HTR2 displayed together

 

Questions for 2D Strip Chart with Data Feed = Heater Source #1: Click "Start Test"

    1. What is the current temperature of Heater 1 ? ____________________
    2. What was the temperature of Heater 1 at the oldest time displayed? _________
    3. What is the current temperature of Heater 2? ____________________

d. What was the temperature of Heater 2 at the most recent time displayed before the current time? _______________

e. Which heater is currently warmer? _____ Heater 1 ______ Heater 2

    1. As displayed, the temperature for Heater 2 is going: ____ up ____ down _____ staying about the same ____ can’t tell

g. I would expect the temperature for Heater 2 to ____ go up, ____ go down, _____ stay about the same in the immediate future. ______ Cannot predict

    1. As displayed, the temperature for Heater 1 is going: ____ up ____ down ____ staying about the same ____ can’t tell

i. I would expect the temperature for Heater 1 to ____ go down, _____ go up, _____ stay about the same in the immediate future. _____ Cannot predict.

Please click "Stop Test"

    1. Other comments?

 

Participant Number: __________

Heater Source #1= HTR1 and HTR2 displayed together

 

Questions for 3D Strip Chart with Data Feed = Heater Source #1 Please Click "Start Test"

    1. Without manipulating the display, can you tell which strip represents Heater 1 and which strip represents Heater 2? _____ Yes ____ No _____ Not sure
    1. Please use the controls, if necessary, so that you can distinguish Heater 1 from Heater 2.

Controls used: ______________________________________________________

    1. Which heater is warmer? ________ Heater 1 ________ Heater 2
    1. Does the color coding suggest anything? _____ No _____ Yes ____________ Explain
    1. Can you make any predictions about Heater 1 or Heater 2 based on this display?

The temperature of Heater 1 will ___________________________________

The temperature of Heater 2 will ___________________________________

Please click "Stop Test"

    1. Does the third dimension aid your understanding of the data? ___ Yes ___ No; Explain:

 

 

g. Other comments?

Participant Number: __________

Heater Source #1= HTR1 and HTR2 displayed together

 

Questions for a 2D Pie Chart with Data Feed = Heater Source #1 Please click "Start Test"

    1. Which heater is warmer? ________ Heater 1 _________ Heater 2

b. The temperature for Heater 2 is going: ____ up ____ down _____ staying about the same ____ can’t tell

c. The temperature for Heater 1 is going: ____ up ____ down ____ staying about the same ____ can’t tell

Please click "Stop Test."

    1. Other comments?

 

Participant Number: __________

Heater Source #2 = HTR1, HTR2, HTR3, HTR4, and HTR5 displayed together.

 

Questions for 2D Text with Data Feed = Heater Source #2: Please click "Start Test"

    1. What is the value displayed for Heater 2 ? ____________________

 

b. What is the value displayed for Heater 1? ____________________

 
    1. Over the last three time periods, the temperature for Heater 1 has been going:

____ up ____ down _____ staying about the same ____ can’t tell

d. I would expect the temperature for Heater 1 to _____ go up, _____ go down, or ____ stay about the same in the next time period. _____ Cannot predict
    1. Over the last three time periods, the temperature for Heater 2 has been going:

____ up _____ down _____ staying about the same ____ can’t tell

f. I would expect the temperature for Heater 2 to ____ go up, ____ go down, or _____ stay about the same in the next time period. _____ Cannot predict

Please click "Stop Test" when you have completed these questions.

g. Other comments?

 

 

Participant Number: __________

Heater Source #2 = HTR1, HTR2, HTR3, HTR4, and HTR5 displayed together.

 

Questions for 3D Text with Data Feed = Heater Source #2 Click "Start Test"

a. What is the value displayed for Heater 2 ? ____________________

b. What is the value displayed for Heater 1? ____________________
    1. Over the last three time periods, the temperature for Heater 1 has been going:

____ up ____ down _____staying about the same ____ can’t tell

d. I would expect the temperature for Heater 1 to _____ go up, _____ go down, or _____ stay about the same in the next time period. _____ Cannot predict
    1. Over the last three time periods, the temperature for Heater 2 has been going: ____ up _____ down _____ staying about the same ____ can’t tell
    1. I would expect the temperature for Heater 2 to _____ go up, _____ go down, or _____ stay about the same in the next time period. _____ Cannot predict.

Please click "Stop Test," when you have completed these questions.

    1. Does the third dimension aid your understanding of the data? ___ Yes ___ No; Explain:

 

 

h. Other comments?

 

Participant Number: __________

Heater Source #2 = HTR1, HTR2, HTR3, HTR4, and HTR5 displayed together.

 

Questions for 2D Bar Chart with Data Feed = Heater Source #2: Click "Start Test"

    1. Does the display need to be repositioned ? ______ No _____ Yes

If No, why not? ______________________________________________________________

If Yes, what control(s) would you recommend using? _________________________________

b. What is the temperature of Heater 1 ? ____________________

c. What is the temperature of Heater 2? ____________________

d. The temperature for Heater 2 is going: ____ up ____ down _____ staying about the same ____ can’t tell

    1. The temperature for Heater 1 is going: ____ up ____ down _____ staying about the same ____ can’t tell

Please click "Stop Test" when you have completed these questions.

f. Other comments?

 

Participant Number: __________

Heater Source #2 = HTR1, HTR2, HTR3, HTR4, and HTR5 displayed together.

 

Questions for 3D Bar Chart with Data Feed = Heater Source #2 Click "Start Test"

    1. Does the display need to be repositioned ? ______ No _____ Yes

If No, why not? ______________________________________________________________

If Yes, what control(s) would you recommend using? _________________________________

b. What is the temperature of Heater 1 ? ____________________

c. What is the temperature of Heater 2? ____________________

  1. The temperature for Heater 2 is going: ____ up ____ down _____ staying about the same

_____ can’t tell

    1. The temperature for Heater 1is going: ____ up ____ down _____ staying about the same ____ can’t tell

Please click "Stop Test" when you have completed these questions.

    1. Does the third dimension aid your understanding of the data? ___ Yes ___ No; Explain:

 

 

g. Other comments?


Participant Number: __________

Heater Source #2 = HTR1, HTR2, HTR3, HTR4, and HTR5 displayed together.

Questions for 2D Strip Chart with Data Feed = Heater Source #2: Click "Start Test"

    1. What is the current temperature of Heater 1 ? ____________________
    2. What was the temperature of Heater 1 at the oldest time displayed? _________
    3. What is the current temperature of Heater 2? ____________________

d. What was the temperature of Heater 2 at the most recent time displayed before the current time? _______________

e. Which heater is currently warmer? _____ Heater 1 ______ Heater 2

    1. As displayed, the temperature for Heater 2 is going: ____ up ____ down _____ staying about the same ____ can’t tell

g. I would expect the temperature for Heater 2 to ____ go up, ____ go down, _____ stay about the same in the immediate future. ______ Cannot predict

    1. As displayed, the temperature for Heater 1 is going: ____ up ____ down ____ staying about the same ____ can’t tell

i. I would expect the temperature for Heater 1 to ____ go down, _____ go up, _____ stay about the same in the immediate future. _____ Cannot predict.

Please click "Stop Test"

    1. Other comments?

 

Participant Number: __________

Heater Source #2 = HTR1, HTR2, HTR3, HTR4, and HTR5 displayed together.

 

Questions for 3D Strip Chart with Data Feed = Heater Source #2 Please Click "Start Test"

    1. Without manipulating the display, can you tell which strip represents Heater 1 and which strip represents Heater 2? _____ Yes ____ No _____ Not sure
    1. Please use the controls, if necessary, so that you can distinguish Heater 1 from Heater 2.

Controls used: ______________________________________________________

    1. Which heater is warmer? ________ Heater 1 ________ Heater 2
    1. Does the color coding suggest anything? _____ No _____ Yes ____________ Explain
    1. Can you make any predictions about Heater 1 or Heater 2 based on this display?

The temperature of Heater 1 will ___________________________________

The temperature of Heater 2 will ___________________________________

Please click "Stop Test"

    1. Does the third dimension aid your understanding of the data? ___ Yes ___ No; Explain:

 

g. Other comments?

Participant Number: __________

Heater Source #2 = HTR1, HTR2, HTR3, HTR4, and HTR5 displayed together.

 

Questions for a 2D Pie Chart with Data Feed = Heater Source #2 Please click "Start Test"

    1. Which heater is warmer? ________ Heater 1 _________ Heater 2

b. The temperature for Heater 2 is going: ____ up ____ down _____ staying about the same ____ can’t tell

c. The temperature for Heater 1 is going: ____ up ____ down ____ staying about the same ____ can’t tell

Please click "Stop Test."

    1. Other comments?

 

 

 

 

Participant Number: __________

 

Please rate the usefulness of each of the following visualization types for your operational setting:

If you do not remember a particular type, please ask to see it again.

  1. 2D text

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

 

  1. 3D text

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

 

  1. 2D bar chart

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. 3D bar chart

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. 2D strip chart

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. 3D strip chart

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. 2D pie chart

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

 

Participant Number: __________

Usefulness of display controls. For each type of 3D data, please rate the usefulness of the display controls, that is, the zoom, pan, and rotate capabilities:

If you do not remember a data-control combination, please ask to see it again.

  1. Zoom capability for 3D text:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

b. Panning capability for 3D text:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. Rotating capability for 3D text:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

  1. Panning capability for 3D bar chart:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. Rotating capability for 3D bar chart:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. Zooming capability for 3D bar chart:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. Rotating capability for 3D strip chart:

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

Participant Number: ________

 

 

h. Zooming capability for 3D strip chart:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

  1. Panning capability for 3D strip chart:

 

|-------------------------------------------------|-----------------------------------------------------|

Totally useless Moderately useful Very useful

 

__________

Do you have suggestions for other types of visualizations that would be useful in your work?

 

 

 

 

 

 

Comments: