Protobi blog

Wait! Before you send the survey for programming, here are a few quick ways to simplify the survey for the respondent, the client, and the analyst.

A few of these pick on an actual customer satisfaction survey Amtrak sent me. This is unfair. The trip was great, and it’s clear that Amtrak listens to its riders to keep the experience nice. But a lot of surveys are similar so it’s a good example.

Ask short questions

Instead of the long preamble above, how about simply asking:

“How was your recent Amtrak trip from New York to Boston?”

Similarly, the items are all under “At the boarding station…” so we might shorten those to:

“Accurate information”

“Friendly/helpful personnel”

One of my mentors often emphasized that we should assume respondents, particularly physicians, are functionally illiterate. Not that they can’t read, it’s just that so many words are put in front of them, they can’t read them all and still do their jobs. So use fewer words.

Don’t write the scale in the question

As in the example above, you don’t need to preface your questions with “On a scale of …”. On many survey platforms the respondent doesn’t even see the scale numbers, they just see a row of radio buttons nicely labeled. You can just ask the question.

In this survey, they do show the scale numbers but to no benefit. Why is this a 100 point scale if there are only eleven buttons? It’s unnecessary to enumerate the scale numbers in the scale itself.

Don’t ask answers you already know

This was the first question on the Amtrak survey. The survey platform already knew my operating system, browser, version, screen size, IP address, etc. This was unrelated to customer satisfaction, so could have been dropped.

Use best practices for demographics

An occupational hazard of working in Cambridge MA is that entire kickoff meetings can go sideways on the semantics of race vs ethnicity, sex vs gender, and Red Sox vs Yankees.

For simplicity follow established patterns, e.g. Wikimedia’s Survey Best Practices, so you don’t make mistakes like assuming respondents have one race, or that they even want to answer it. Actually, is it so relevant to your research goals that you need to ask it?

Use intermediate variables

You can ask the survey progammer to create intermediate variables. These can make the rest of the survey a lot simpler.

For instance, a recent survey had a fair bit skip logic like the above. Assuming that this complexity is actually required, a good way to make it clearer it is to instruct the survey programmer to create new variables:

These intermediate variables make the instructions easier to read, and go into the survey data, so you can branch on them for analysis too.

Use smart question numbers

A lot of information about your survey is lost when exporting the data. For instance, the entire structure of your survey, that neat organization into sections and subsections.

Protobi can reconstruct your survey structure using its Autogroup feature. Number your questions with the most rapidly changing element on the right. E.g. qd_1, qd_2, …

Protobi will parse the question number, splitting at boundaries between letters, numbers and underscores and sort things that look like numbers numerically rather than alphabetically.

Conversely, if questions “D2” and “D2a” don’t have anything to do with each other really, then give them different numbers, like “D2” and “D3”. It’ll make your life easier at analysis time.

Ask the question you want to answer

The following appeared in a recent questionnaire, eliciting patient numbers as absolute counts. However, these were all converted to percentages for analysis and reporting. The physician respondents probably answered them by first thinking about percentages. In this case, it would have been simpler to ask the questions as percentages.

Ask us for a quick review

Writing a survey you’re thinking of analyzing with Protobi? Send us a draft at support@protobi.com or set up a time. Our experts can review it and suggest smart + lazy ways to make it easier to program, answer and analyze.

We won’t take much of your time, and can likely save you quite a lot. Pretty much every survey presents a number of easy ways to keep it short and simple.