Over the past ten years, The Urban Transport Institute has developed a travel survey methodology based on the personal delivery and collection of self-completion travel diaries to randomly sampled households, supplemented by pre-contact letters, telephone motivational calls, phone and postal reminders and telephone clarification calls.
The method has been used and refined in a variety of surveys in Victoria, Queensland, Western Australia and New Zealand. The main steps in the process, as implemented in VISTA09, are:
Prepare Pre-Contact Letters (PCL)
At the beginning of the survey period, a sample of households is drawn from the GIS cadastre file sample frame. In each of the 52 weeks of the survey, those households designated for contact are identified, and a list of addresses is made for each of the fieldwork regions, accompanied by a GIS map showing the location of each of the addresses within the cluster. For VISTA09, 16 fieldwork regions were used with 56 addresses sampled in each region (42 Primary Addresses and 14 Replacement Addresses).
Address Checking and Pre-Contact Letter Delivery
On the Tuesday preceding each week of Travel Days in the survey, field staff in each of the 16 regions check for the existence and suitability of each of the Primary Addresses. At this stage, some of these addresses will be classified as Sample Loss (i.e. no such address, address not findable, vacant block, business address, empty premises etc), and will need to be replaced by the nearest of the 14 pre-sampled Replacement Addresses. Other addresses may need to be corrected (e.g. for apartments, there may need to be correction or clarification of apartment numbers, where the apartment number is either missing or mis-recorded in the sample frame). PCL field staff are given strict instructions about what corrections they are allowed to make to the addresses.
Once the In-Scope addresses have been established, field staff deliver a Pre-Contact Envelope (addressed to The Householder at that address) to each In-Scope address (the letter is placed in the mailbox, with no contact being made with the householder at this time). The Pre-Contact Envelope is an official Government envelope containing a Pre-Contact Letter from the Department of Transport and a Survey Brochure describing the survey.
Correction of Sample Addresses
On the Wednesday of each week, the field corrections made on the Tuesday are entered into the Sample Address database, and a final list of 42 In-Scope addresses are produced for each region for the delivery of the Questionnaires.
Survey Pack Delivery (SPD) Preparations
Each Friday, the Survey Packs are assembled for delivery, with the updated list of households accompanied by an updated map, with any comments from PCL field staff to enable the easy and correct location and identification of households for the delivery of questionnaires (e.g. big white house with poplar tree and red letterbox).
Survey Pack Delivery
On the following Saturday and Sunday, SPD field staff (different from those who delivered the Pre-Contact Letters on the previous Tuesday) deliver the questionnaires in each of the 16 regions. The questionnaires cover Travel Days from the next Monday through to the following Sunday. Every attempt is made to personally contact a household member to deliver the questionnaire in person (since this has been found in previous surveys to significantly increase the response rate). A target of 70% personal delivery is set (and generally achieved) for the VISTA survey. Interviewers pay up to four visits to a household during the weekend to try to contact them (two attempts on Saturday and two attempts, if needed, on Sunday). If contact is made, the interviewer introduces themselves, briefly explains the nature of the survey and answers any questions about the survey. If the household agrees to accept the questionnaires, then the interviewer ascertains whether the four travel diaries in the Survey Pack are sufficient for that household and, if not, then provides extra Travel Diaries for the household. They also inform the household when they will return to pick up the completed questionnaires, and ask where they might leave the questionnaire out for collection if they are not at home. They also ask for a contact phone number in case we need to contact them during the survey.
If contact cannot be made after the 4 visits, a “We-Missed-You” postcard is left at the household with an envelope containing all the survey materials (including 6 Travel Diaries to cover most household sizes). Information is also left about the pickup of the questionnaires the following week, and instructions for where to leave the completed questionnaires if the household does not expect to be home when they are collected.
If a personal refusal is received at this stage, then the field staff are instructed to first enquire whether the reason for the refusal is “transport-related” (e.g. I don’t travel much, so why am I in the Travel Survey?). They then attempt to convert such refusals to an acceptance by stressing the importance of obtaining responses for all types of households. If they cannot be converted, or if the reason for the refusal is not transport-related, then the field staff immediately ask two “refusal questions”: how many people are in the household and how many vehicles are in the household. This information gives some information about refusals. These questions are asked anytime in the survey process that a personal refusal is encountered. Based on Cialdini’s principle of Reciprocity (Cialdini, 2007), we have found that the majority of refusals are willing to answer these “refusal questions” (65% at the survey delivery stage and 75% at the survey pickup stage), even though they have just refused to do the full survey.
On the evening of each household’s Travel Day, they are phoned by the Survey Office to check that they have received the survey materials, to answer any questions they might now have, and to remind them about their Travel Day. Phone numbers are obtained from households when survey packs are delivered to them, or from phone number matching services for those households not personally contacted.
Survey Pack Pickup (SPP) for Weekday Travel Days
On the following Sunday, a different set of SPP field staff goes out to the 16 regions to collect the questionnaires for those households that had a weekday Travel Day (the weekend Travel Days are treated slightly differently as described below). All collections are performed during daylight hours. Two attempts are made to personally contact the household (since the effect of personal contact at this stage is not as strong as in the delivery stage). As soon as possible after collecting questionnaires from the household, the SPP field staff perform a quick edit check to identify whether the full number of Travel Diaries have been completed and handed over. If any Travel Diaries are missing, the interviewer returns to the household to enquire whether the Diaries were inadvertently excluded from the package of completed questionnaires.
If the household is not at home, but they have left their questionnaires out for collection, then the questionnaires are collected and a “Thank-you” postcard is left to indicate that the questionnaires have been collected. If the household is not at home (on the second attempt at making contact), and they have not left their questionnaires out for collection, the interviewer leaves a “We-Missed-You” postcard together with a reply-paid envelope for the return of the questionnaires by mail to the survey office.
Survey Pack Pickup (SPP) for Weekend Travel Days
On the following day (Monday), SPP field staff go back to the 16 regions to collect the questionnaires for those households that had a weekend Travel Day. This separate procedure is required to allow households with a weekend Travel Day sufficient time to complete their questionnaires. Again, two attempts are made to personally contact the household, with the same procedures employed as described above for the Sunday collections. Given that this collection is being performed on a weekday, when fewer people are at home during the day, it is expected that fewer householders will be personally contacted for questionnaire collection.
Visual Checks of Questionnaires
When questionnaires are returned to the Melbourne field office, the contents of the Survey Pack plastic bag are checked, to see whether the questionnaires have actually been completed or are blank, to see whether any Travel Diaries are missing and to remove surplus materials from the bags before they are sent on for data coding. This process is repeated for questionnaires returned through the mail, with the ID numbers of these respondents recorded (using the HotSpot Administration Program) to signify that they have been received, and hence removed from any Reminder activities.
Once the questionnaires have been received from respondents, the next task is to enter the data into computers in digital format. The coding method chosen for this survey uses a custom-written program, designed and produced by TUTI, for data entry and editing. The program (Speedit – Survey Program for Entry and Editing of Data Involving Travel) is described more fully in a later section. In the VISTA09 project, data was generally coded, edited and clarified within two weeks of its return from the field.
The Speedit program performs both data entry and data editing at the same time. One of the advantages of using Speedit for data entry and editing is that, because all geocoding is performed as the data is entered, it is possible to run detailed editing checks involving trip distances and speeds, since these can be calculated as soon as the geocodes for successive locations are found. Thus, in addition to the usual range checks and logic checks, these more complex trip editing routines can be run immediately, thus enabling the data coder to immediately correct problems with the data or flag the data for follow-up clarification calls (which typically commence within a week of the respondent filling out the survey forms).
Identification of Data Problems
Following the coding and editing of data from the questionnaires, the data are merged into an integrated file for each household. This file is then analysed looking for missing data and for data entries that fail the range checks or logic checks. These households, and the problem data, are then earmarked for further investigation.
Those households in which problem data have been identified and not rectified, and for which a phone number is available (approximately 99% of total surveys), are then phoned to seek clarification about the problem data. The clarified information is then input into the data file. About 40% of responding households were phoned during this process for VISTA09.
To ensure the quality of the data entry and geocoding processes, two validation process are employed. In the Pilot survey and in the early weeks of the Main survey, data entered for 10% of the responding households was subject to a complete data validation process. To do this, a senior data enterer independently completed the entry of the chosen survey, and an automated check for differences between the two entries was made. This validation process is primarily used to check the early entries by a new data enterer, and to identify and correct systematic entry errors they might be making. A significant goal of the independent entry is to test the quality of the geocoding for each stop. The level of this validation falls over the course of the Main survey, such that about 2% of all responding households in the Main survey are validated in this manner.
The second method of validation (a “check off” by a senior data entry supervisor before final acceptance of the data) is applied to all households over the course of the survey. The "check off" procedure involves making sure that the correct number and type of people and vehicles is entered; that the correct number and type of stops have been made; and that those stops occur at the correct times, with the correct mode and type of activity, and are linked to the other members of the household (if relevant). Any errors detected by Speedit during entry are also available to the person performing this validation process, and must be resolved to their satisfaction. If any errors are detected, the survey data is corrected, usually by the original data-enterer, so that they realise the error they made and improve their entry of data for future households.
Postal Return of Completed Questionnaires
In the weeks after the collection of questionnaires from the households, about 25-30% of those households that were left a reply-paid envelope will return their completed questionnaires through the mail. These returns are edited and clarified progressively as they arrive in the survey office.
Reminder Calls and Letters
About four days after the collection of completed questionnaires from households, those who have not yet responded are contacted via phone to remind them to return the questionnaire if they have completed it. Those who are not contactable by phone are sent a reminder letter.
Data Weighting and Expansion
The sample data is weighted and expanded using the 2006 Census data as the reference secondary data set. In addition to this demographic weighting to adjust for sample design and sampling bias, two other sources of potential bias are also specifically addressed. The first is a non-reporting bias due to proxy reporting and time lag in completion of the travel diary (Richardson, 2006). The second potential bias is non-response bias. The extent of this bias is investigated by the use of “non-response” questions that are asked of all refusals, to gain an idea of the demographic and mobility characteristics of refusals at different stages of the survey process.
TUTI have found that the above method gives good quality travel data with a high value for money. Other methods (such as face to face interviews) may give slightly better quality data, but at a much higher price. At the either end of the spectrum, other methods (such as CATI or mailout/mailback self-completion surveys) may be somewhat cheaper but give household travel data of far inferior quality.