Author Archives: Christy

The difference between random representative and open access surveys

When conducting community research there are two key ways that we can source respondents. Below explains the differences between these two methods to help you select the most appropriate one for your project (some projects will use both).

Representative Surveys

Purpose: to understand the whole community.

In a representative survey, we aim to get a snapshot of the entire community. This is done through a process called “sampling.” At ASDF Research, we work with Local Governments by using their rates database, which lists all households in the area. We randomly select a certain number of households from different geographic areas (like suburbs, small areas called SA1s, wards, or precincts).

These selected households are then contacted to participate in the survey. Since every household has an equal chance of being chosen, the survey results reflect a diverse mix of household types and cover the entire area evenly.

Use: Help understand the needs across the whole community, with the ability to compare findings across areas or demographics. This method is particularly useful at the early stages of planning, to help guide focus and scope.

Open Access Surveys

Purpose: to gather opinions from people interested in the topic.

An open access survey allows anyone who is interested in the topic to share their views. These surveys can be distributed in various ways, such as on Council engagement websites (like “Have Your Say”), social media, posters/signs in public places, or pop-up stalls at events.

Use: Help understand the needs, behaviours and opinions of those who saw the promotion and are interested in the topic. This method is useful to receive feedback on draft strategies and plans, to fine-tune the content.

Summary

Both methods have value, but perform different functions:

  • Representative surveys are used to help understand the needs of the whole community, to guide the development of plans and strategies.
  • Open access surveys are used to check in with those with an interest in the topic to fine-tune draft plans and strategies.

ASDF Research can help with all stages of this process, including survey design, sampling, distribution and analysis.

Victorian public health and wellbeing outcomes dashboard

The Victorian Government has launched their Victorian public health and wellbeing outcomes dashboard.

https://www.health.vic.gov.au/victorian-public-health-and-wellbeing-outcomes-dashboard

This is an interactive dashboard providing access to a range of tracking data about the health and wellbeing of the Victorian community, with an option to filter the data down to LGA level information. Its a very useful tool for updating health and wellbeing profiles, as well as assisting with planning and evaluation of programs and services.
Incidentally, ASDF Research maintains a database of links to other publicly available data about the community, searchable by topic area: https://asdfresearch.com.au/researchkb/

2021 Census

It was 2021 Census release day yesterday! To access a community profile for your LGA, go to this URL and type in your Council name in the search box, then scroll down to click on the ‘Community Profile’ button.
ASDF Research can help your Council update your Health and Wellbeing Profile document with the new data if needed, or put together an infographic snapshot of findings, drop us a line!

Understanding sample size

When undertaking research one usually aims to reach a particular sample size (n=#). This post will explain why, and a few key things to consider.
Note that this is only really relevant for quantitative research (e.g. surveys) where you aim to report the percentage of people who have an opinion or engage in a behaviour. This is not relevant for qualitative research, which focusses on stories (not percentages).

Why do we think about sample sizes?

The ‘sample size’ is the number of people who complete the survey / participate in the research. When doing a survey with the community you will rarely be able to speak to every single person (unless it is the Census). Because of this, there will naturally be opinions and behaviours missed in the results. The more people you speak to the more opinions and behaviours your findings will cover, resulting in increased accuracy.
The stats gurus worked out a formula to help identify how many people are needed in a sample size to enable you to be relatively confident that your findings cover most opinions and behaviours. This formula works by considering the population (total number of people who could complete the survey) and error margin/confidence interval (what sort of variance you are happy to have).
The best practice ‘error margin’ is +/-5% at 95% confidence. This means that if your survey shows 50% of people undertake a behaviour, you can be 95% confident that if you had surveyed everyone in your population the result would be between 45% (-5%) and 55% (+5%) showing that behaviour.

So what sample size should you aim for?

For any population over 200,000, you need a sample size of n=384 to have a +/-5% error margin.
You can figure out the number for a population by using the calculator on this site: https://www.surveysystem.com/sscalc.htm
Use the ‘Determine sample size’ box and put in the population of your area then set the confidence interval to 5 (this means the error margin is +/- 5% which is industry standard).

However…

The number in itself is relatively meaningless if the method used to reach those 384 people isn’t representative. For instance, if your sample is made up of 350 females and 34 males, your %s are not going to be reflective of the whole population, even after considering error margins. One of the biggest errors Councils make is to just distribute their survey through comms channels (in which case you only get those engaged with Council / interested in the topic, who aren’t representative of the broader community).

At ASDF Research we have developed a really good method for getting proper representative samples for Councils. Get in touch to find out more!

What is going on with VPHS?

The Victorian Population Health Survey (VPHS) is an annual survey run by VicHealth to track the health and wellbeing of Victorians. Every 3 years since 2008, the survey has been conducted at an LGA level (n=400 sample per Local Government area) and this local data is often used by Councils to inform their health and wellbeing planning and track outcomes.

The 2008, 2011, 2014 and 2017 data by LGA can be found here: VHISS

As you can see by the sequence of years, 2020 should have been the next round of LGA level surveying, however this didn’t occur due to COVID-19. Instead, a special COVID-19 version of the survey was run in 2020 at LGA level (The survey questions can be found at the end of this report).

If you want to see the results of this COVID-19 survey by LGA you can source the data through their dashboard here: https://vahi.vic.gov.au/report (Dashboard download at the bottom of the page), or request specific data through the VAHI Data Request Hub.

The next standard LGA level VPHS survey is scheduled to be conducted in 2023.

This information is correct as at 29 June 2022 based on information provided on the VPHS website and the VicHealth Health Intelligence Unit.

COVID-19 Coronavirus disease response planning

In light of the WHO classification of COVID-19 as a pandemic, ASDF Research has implemented the following risk management processes. These risk management processes do not impact on the scope of any current ASDF Research project commitments.

  • In the interest of minimising public risk, until further notice, ASDF Research will not be undertaking any public-facing research activities. This includes face-to-face interviewing and on-location interviewing. Online surveys, mail-out surveys, and home-based office activities will continue uninterrupted.
  • Face-to-face client meetings will be replaced with videoconferencing meetings.
  • ASDF Research has the resources necessary to continue operations for 3 months without the need for utilising external supply chains.
  • Networking with other Qualified Professional Researchers (QPR) will be maintained should the health of primary consultants be compromised, necessitating external assistance to meet project milestones. see COVID-19 IRG 16032020 (pdf, 61kb) for further information.
  • Given the business is already home-based with high levels of data security measures in place, no additional data security protocols will be needed.

 For further information please contact us.

2016 Homelessness data

Last week Census released the 2016 homeless estimates (see knowledge base entry here). We have put together an interactive dashboard of the findings comparing 2011 and 2016 rate per 10,000 population by Victorian LGAs:

We have also set up this dashboard in excel format where you can select 2 Victorian LGAs for comparison. This Excel version presents the information in a way that is easy to print a snapshot for your region.

Click here to download: Homelessness by LGA 2016 (Excel file, 43kb)

The table below shows the homeless operational groups included in this count, and the proportion each contributes towards the overall homelessness figure Australia-wide.

 

2016 %

Persons living in improvised dwellings, tents or sleeping out

7%

Persons in supported accommodation for the homeless

18%

Persons staying temporarily with other households

15%

Persons living in boarding houses

15%

Persons in other temporary lodging

1%

Persons living in ‘severely’ crowded dwellings

44%

Prizes and Incentives for completing surveys

There are a number of things to think about when deciding whether to offer a prize for participating in a social research survey.

The three main things to consider are: is it really necessary; is it an appropriate prize; and does it comply with lottery laws? You must also comply with privacy law when collecting, storing and using personal information (name, phone, email) supplied to enter the draw.

Definition: An incentive is an item (product or voucher) offered to a research participant either at the time of participation (offered to all participants) or as a prize draw (occurring at the end of the fieldwork).

  1. Is it really necessary?
    Our community should want to give us their feedback without us having to pay for it. We don’t want to set a precedent that it is only worth providing feedback if there is a reward.  Offering an incentive also increases the likelihood that people will just complete the survey to enter the prize draw (resulting in answers which are not as honest), or complete it multiple times. When offering an incentive we need to factor in methods to identify people who are ‘skimming’ (just completing the survey for the prize), as well as those who complete it multiple times for multiple chances at winning.
  2. Is the prize appropriate?
    When offering an incentive, it is important to ensure that it isn’t going to skew who participates in the survey. For instance, if you offer a public transport voucher, you are going to get a disproportionate number of respondents who use public transport, and those who live in areas without access to public transport who might otherwise have participated, may not bother. Try to pick something that would be of value to all demographics and locations.
  3. Does the prize comply with lottery laws?
    The following information is specific to Victoria, Australia. If people are likely to be responding from other states or Countries, you will need to review the laws in each location.
    If offering an incentive as a prize draw (that is, people provide their details and then a random entrant is drawn to win) it is recognised under Victorian lottery law as a ‘trade promotion lottery’. When running a trade promotion lottery, entering must be free and you must include the following information when the respondent enters the prize draw:

    • Closing date
    • Where and when the prize will be drawn
    • Where the names will be published (If the prize is over $1,000 you must publish the name of the winner.)
    • Any other entry requirements (such as they must have completed at least 80% of the survey and only one entry per person will be accepted).

Other items for consideration are:

    • You must notify the winner in writing
    • Records must be kept for 3 years to prove random selection.
    • Winner must be selected using a randomisation algorithm (so each person has equal chance of being selected).
    • The prize must be delivered to the winner within 28 days of being drawn.
    • The winner may be substituted for another draw only if reasonable efforts have been made to contact them and were unsuccessful.
    • If you need to change the prize after commencement of the survey, the new prize must be of equal or greater value and the winner needs to agree in writing, or you need to make reasonable attempts to provide the alternative.

Privacy law

In order to comply with privacy law, you must follow the following steps regarding the personal information collected to enter the draw (name, email, phone number, address etc):

  1. In the introduction to the section asking for their personal information for the prize draw, include a link to your privacy policy:
  2. When collecting this information, it must not be physically stored outside of Australia (that is, you can’t use Surveymonkey or Google forms. ASDF research has a locally installed online surveying tool, hosted in Australia. Please see our Online Surveying information sheet for further details).
  3. Ensure that you do not store the contact information in the same data file as the survey responses.
  4. The contact information provided must not be used for any other purpose, unless written permission is provided by the individual. That is, if you collect their name and email address for a prize draw, you are not allowed to add them to your enewsletter list. You can include a checkbox asking if they would like to be added to the list, but this must be default unchecked.

Tips for writing survey invite emails

Avoiding spam filters:

A lot of email providers have spam filters that look for certain traits within the subject text and email. In order to reduce the risk that your email will automatically be sent to the spam folder, avoid the following words and symbols:

  • Urgent
  • Money back guarantee
  • Money
  • Breakthough
  • Click here
  • !!!!
  • [Using CAPS]
  • [Coloured fonts]
  • [1 large image inserted into the body of the email]
  • Cheap
  • Free
  • Offer
  • Opportunity
  • Save
  • Shop
  • Subscribe
  • Win / winner / winning / won
  • Guarantee
  • Investment
  • Discount

Email Subject:

  • Keep it short (35 characters or less)
  • No CAPS
  • No explanation points
  • No dollar signs
  • Avoid spam trigger words

Content:

  • Keep it short
  • Make sure the whole email is the same font style and size
  • People don’t generally read past the first line, so make sure there is a link to the survey either in the first sentence, or on the second line of the email
  • If you have prize/incentive, put this in bold. Avoid using the word ‘win’ or ‘prize’. (As a thank you for your time, you will be given the chance to enter a draw for [prize])
  • Don’t use fancy formatting (background colours, images etc) – not only will this be caught by spam filters, but those who have set their email to receive text only won’t see it.
  • Minimise the number of links in the text
  • Keep email size to under 30mb
  • Include the following content:
    • Why they are being invited to participate
    • Survey URL
    • How long the survey will take (make sure this is accurate, overestimating is good)
    • How you will use the findings / how the survey will benefit them
    • If anonymous, say so and link to your privacy policy
    • How the findings will be analysed (group form)
    • End date for completion
    • Incentive / prize
    • Your sign-off / footer

Other considerations:

  • Make sure the list you are using it is OK to use these contacts (not contravening privacy law)
  • If you are sending to lots of people at the same time, not using mail merge, make sure you use bcc (so recipients can’t see other recipient email addresses).
  • Did you know that you can mail merge to email using Word?

Example email:

Subject

[Your organisation] is seeking your feedback on xx

Body

Hi [Name],

[Your organisation] is currently running a survey to find out your views on xx. To go to the survey, please click on the link below.

[link]

If this link isn’t working, please copy the URL and paste it into your browser address bar (at the top of the window, NOT Google search).

As a thank you for your time, you will be given the chance to enter a draw for [prize].

The survey should take around X minutes to complete.

You have been selected to participate in this research because you xx. Findings will be used to help us xx.

Your responses to this survey are completely anonymous and will be grouped with other participants for analysis. For further information please see our [Link]privacy policy[/link].

Please complete this survey by [date].

Regards,

[Your name]

[Footer]