GETTING YOUR DATA: MYSTERY SHOPPING

Download
Activity 3.10

Activity: 3.10 Mystery Shopping Training

Purpose of Activity:

To make participants familiar with the instrument being used for mystery shopping and to train shoppers and observers on how to conduct the mystery shopping visits.

By the end of the activity Participants will:

  • Understand what mystery shopping is and why it is being used
  • Understand how mystery shopping fits in to the larger campaign
  • Be familiar with some research terms
  • Be familiar with the mystery shopping instrument
  • Understand all the key terms in the instrument
  • Understand who/where should be targeted by the mystery shopping visits

Before this Activity Participants Will Need to:

Determine where you want to go for the mystery shopping
Have finalized your mystery shopping instrument
Develop scenarios for your mystery shoppers

Materials Needed:

Paper
Writing Utensils
Research questions
Copies of the mystery shopping scenarios
Copies of the mystery shopping instrument
List of “Key Terms” in your mystery shopping instrument

Key Terms

Mystery Shopping
Sample
Respondent

Intended Audience:

Organization members

Time Needed:

2 hours

Part I: “What is mystery shopping and why are we are we doing it?” (10 minutes)

Facilitator Instructions:

1. Ask a volunteer who’s been consistently involved in the research project to describe the research project and how it fits into your campaign to the rest of the group.
2. Explain that mystery shopping is a type of observation research used to evaluate businesses or programs. People pose as consumers and track their observations about how a business or program operates and whether or not they are in compliance with the law.
3. Introduce how the mystery shopping fits into your research project.
4. Popcorn questions about how the mystery shopping is connected to organizing.

Part II: “Who are we targeting?” (10 minutes)

1. What are our research questions? What information do we need to gather to answer these questions? Who do we need to observe in order to gather the information we need?
2. Explain that the type of person or business that we observe will be our sample.
3. Ask the group to tell you why we targeting this particular group of people? Clarify that we are trying to find out if these businesses or agencies are committing fraud against our members so we can hold them accountable and improve services.
4. Discuss who you are not targeting and how to make sure you are surveying the right people.

Part III: How To Approach People for Mystery Shopping (30 minutes)

1. Handout copies of “the Types of Shoppers and Scenarios” you have created for your mystery shoppers.
2. Explain the different Types of shoppers and explain that each person will assume the identity of one of these. Read it through once as a group and answer any questions.
3. Have participants role play the scenarios in partners
4. Come back to the big group to debrief
5. Discuss any issues that need special attention in doing outreach for the survey (confidentiality, discussing sensitive issues, etc.)

Part IV: Getting Familiar with the Mystery Shopping Instrument (1 hour)

1. Before the meeting, prepare a list of key terms that show up frequently in the survey that are specific to your campaign.
2. Give each participant a handout that includes:

  • The list of key terms
  • A copy of the instrument
  • The “tips for mystery shopping” handout (see p. x in toolkit)

3. Go over the list of key terms with members, clarify any questions that arise.
4. Walk through the survey with the group and discuss how to use each part of it.
5. Discuss the questions that are tricky.

  • Are there any questions that seemed hard to understand or explain?
  • Are there any words that are confusing or hard to read?
  • Are there directions in the survey that are confusing?

6. Troubleshoot – brainstorm how to address some the issues that came up
7. Next go over the general tips for mystery shopping.

Part V: Wrap up check out (10 minutes)

1. Discuss questions, and concerns that came up throughout the day’s activity.
2. Discuss next steps in conducting mystery shopping.

Download
Tool 3.15

Tool: 3.15 Tips for Mystery Shopping

Tips for Mystery Shoppers

1. Focus on your role
2. Be consistent with your story/scenario
3. Do not include the observer in the conversation (let the observer observe)
4. Start right away
5. Find out title, name and contact information of the person speaking to you. (Ask for fliers, business card, etc.)
6. Ask what types of services they provide
7. Ask about the costs of the services (upfront costs, payment plans, associated fees and costs, refunds if not satisfied, etc.)
8. Get specific information about the how they provide services (requirements and qualifications for consumer, what personal information they need, timeline, etc.)
9. Find out if they will give you a contract

Tips for Observers

1. Focus on your role (be observant)
2. Try not to be part of the conversation unless you are asked a question
3. Do not use your cell phone during the conversation
4. Look around for posters, pictures, fliers on the walls
5. Look for relevant documentation (certificates of service, license numbers, relevant laws and statutes, etc.)
6. Collect fliers, pamphlets, brochures, business cards, and other information
7. Once you are done observing, pay attention to the conversation

Download
Tool 3.16

Tool: 3.16 Sample Mystery Shopping Scenarios

Sample Mystery Shopping Scenarios

  • Your name is _____________________ and you are seeking a job in construction. You know how to do most things, but you are best at welding. You need employment as soon as possible, so ask the agency if they can guarantee that they will be able to find you a job. If they ask, you don’t have a work permit.
  • Your name is _________________. You have experience working in restaurants as a cook and washing dishes in your country of origin (feel free to elaborate more on this). If they ask, you arrived very recently to the U.S. and you don’t have a work permit.
  • Your name is ______________________, you are seeking work as a nanny. You have experience, letters of recommendation and you would like to find something similar to what you have been doing. You are only interested in being a nanny. If they ask, you don’t have a work permit.
  • Your name is ______________. You arrived to the United States 5 years ago. You crossed the Mexican border in May of 2005. You currently work at a coffee shop. You would like to get papers so that you can be in the U.S. legally and fortunately you have a friend who offered to marry you. Your friend became a U.S. citizen one year ago. If they ask: No one has petitioned for you before.
  • Your name is ________. You arrived to the United States two years ago and got here by crossing the Mexican border in September of 2008. You currently work in construction. You want to know if you can get your papers. A friend of yours told you that they got their papers through a work permit and you want to know if you can do the same. If they ask: No one has ever petitioned for you before.

Download
Tool 3.17

Tool: 3.17 Template for Mystery Shopping Form – Employment Agencies

EMPLOYMENT AGENCY SERVICES

INSTRUCTIONS: Fill out this form immediately after you have finished your mystery shopping visit. When filling out this form, please write clearly and provide as much detail as possible. If checkboxes are provided please clearly mark the boxes. Please remember to get business cards and brochures from each of the businesses when possible.

Name of Mystery Shopper: _______________________________________________
Primary Language: __________________
Country of Origin: ____________________
Name of the Observer: ___________________________________________________
Primary Language: __________________
Country of Origin: ____________________
Scenario Used (circle one): A B C
Date: ___________________

INSTRUCTIONS: For questions 1 to 6 please look inside and outside of the agency and record your observations. (observer: fill out these questions)

1. Name of Agency ________________________________________
2. Address of Agency __________________________Zip__________
3. Was there a sign posted outside stating that the business is an employment agency?

  • Yes
  • No

4. Does the agency have the New York City’s Employment Agency Laws posted visibly?

  • Yes
  • No

5. Does the agency have a License number posted on the wall?

  • Yes (license number?)_____________________________
  • No

Is the owner’s name on it?

  • Yes (license name?)_____________________________
  • No

6. Is the license issued by Department of Consumer Affairs? Does it have this logo on it? (attach it to the back)

  • Yes
  • No


INSTRUCTIONS: Questions 7 to 17 will ask about the person that you spoke with at the agency, the services, payment options and agreements you were offered. If you spoke with more than one person, please answer all the questions for the person you spent the most time speaking with.

7. Name and Job Title of person (s) with whom you spoke at the Agency: Name____________________Title___________________________
Name____________________Title___________________________
8. Did this person guarantee that they would find you a job?

  • Yes
  • No
  • I don’t remember

9. Did the agency require a deposit or charge you a fee before finding you a job?

  • Yes (how much?)$______________
  • No
  • I don’t remember

10. What kind of job did they say they could find you and how much did they say it would pay? (check all that apply)

  • Cleaning/Domestic Work $_____________ hour/day/week/month
  • Child Care/Elderly Care $_____________hour/day/week/month
  • Busboy/Washing Dishes/Restaurant $_____________hour/day/week/month
  • Construction $_____________hour/day/week/month
  • Other: ______________________ $_____________hour/day/week/month

11. If they get you a job, will they charge you an additional fee (called a placement charge)?

  • Yes (how much?)$______________
  • No
  • They didn’t tell me

12. Will they deduct your initial fee from the placement charge?

  • Yes
  • No
  • They didn’t tell me

13. Did they say that you pay the placement charge in installments?

  • Yes (number of installments?)_______
  • No
  • They didn’t tell me

14. Will the agency provide a refund if they are unable to find you a job?

  • Yes (In how long?)_______days ______months ______years
  • No
  • They didn’t tell me

15. Will the agency provide a refund if the worker doesn’t like the job?

  • Yes (In how long?)_______days ______months ______years
  • No
  • They didn’t tell me

16. Did the agency offer you a written contract?

  • Yes
  • No

Additional Comments:
________________________________________________
________________________________________________
________________________________________________
________________________________________________
________________________________________________
************************************************************************

[For office use only:]
Is the pay they are offering just and legal under New York State Law?

  • Yes
  • No

Is the placement fee just and legal under New York State Law?

  • Yes
  • No

Does the agency have a valid license?

  • Yes
  • No

License number: _____________________________________
Name on license: _____________________________________
Date license was issued: ______day ______month ______year

Download
Tool 3.18

Tool: 3.18 Template for Mystery Shopping Form – Immigrant Services

IMMIGRATION SERVICE PROVIDERS/IMMIGRATION ATTORNEYS

INSTRUCTIONS: Fill out this form immediately after you have finished your mystery shopping visit. When filling out this form, please write clearly and provide as much detail as possible. If checkboxes are provided please clearly mark the boxes. Please remember to get business cards and brochures from each of the businesses when possible.

Name of Mystery Shopper: _______________________________________________
Primary Language: _____________________________________________________
Country of Origin: ______________________________________________________
Name of the Observer: ___________________________________________________
Primary Language: _____________________________________________________
Country of Origin: ______________________________________________________
Scenario Used (circle one): A B
Date: __________________

INSTRUCTIONS: For questions 1 to 6 please look inside and outside of the agency for posters, advertisements and literature and record your observations.(Observer: fill out this section)

1. Name of Agency ________________________________________
2. Address of Agency __________________________Zip__________
3. Does the Agency advertise that they provide immigration services? (in signs, flyers or advertisements)

  • Yes
  • No
  • Unclear

4. What types of immigration services does the Agency claim to provide (in signs, flyers or advertisements)? (check all that apply):

  • Type forms
  • Gather documents
  • Provide Legal Advice
  • Take photographs
  • Arrange medical exam
  • None of these
  • Immigration Attorney Office
  • Other:_____________________________________________

_____________________________________________________
5. What types of non-immigration services does the Agency claim to provide? (in signs, flyers or advertisements) (check all that apply):

  • Notary
  • Taxes
  • Car insurance
  • Legal translations
  • Real Estate
  • Employment Agency/Services
  • Mortgage Law
  • Divorce Law
  • Criminal Law
  • Accident Law
  • Other:____________________________________________

6. Did the Agency have any of the following signs on the wall? (Check all that apply):
If an Attorney,

  • Certificate from the NY State Bar

If not an Attorney,

  • Not an Attorney
  • Name of Bond Agency
  • Call 311 (for complaints)
  • Notary Public Certification
  • List of Prices and Services
  • None
  • Other: _________________________________________


INSTRUCTIONS: Questions 7 to 15 will ask about the person that you spoke with at the agency and the services you were offered. If you spoke with more than one person, please answer all the questions for the person you spent the most time speaking with.

7. Name and Job Title of person (s) with whom you spoke at the Agency: Name________________________Title________________________
Name________________________Title________________________
8. Did the person you speak with identify themselves as: (check all that apply)

  • Attorney/Lawyer
  • “Immigration Specialist”
  • Notary
  • Paralegal
  • I don’t remember
  • Other: __________________________________________

9. If they said they were an attorney, do they claim to specialize in immigration law?

  • Yes
  • No
  • I don’t remember

10. Did they tell you if you qualify to change your status?

  • Yes, I CAN adjust my status
  • No, I CAN NOT adjust my status
  • They said they were not authorized to determine that.

11. What kind of immigration legal services were you offered and how long did they say the services would take? (Check all that apply)

  • Green Card ___days _____months ____years  __ No time given
  • Employment help/work permit ___days _____months ____years __ No time given
  • Visa ___days_____months____years __ No time given
  • “Papers” ___days_____months____years __ No time given
  • Other: ______________________________________________

___days_____months____years __ No time given

  • None

12. Did they offer a referral for any services?

  • Yes
  • No
  • I don’t remember

If yes…
Where? ________________________________________
Did they charge a fee for the referral?

  • Yes (how much?) $_________
  • No
  • I don’t remember

13. Did this person guarantee that their services would be successful?

  • Yes
  • No
  • I don’t remember

14. Did this person mention influences they may have with the U.S. Government?

  • Yes
  • No
  • I don’t remember

15. Did this person mention influences with governments of countries other than the U.S.?

  • Yes
  • No
  • I don’t remember

16. Did this person give you a total cost for all of their services?

  • Yes
  • No
  • I did not ask
  • I don’t remember

17. If yes, how much would the services cost? $ _____________________
18. Did they offer you a written contract?

  • Yes
  • No
  • I did not ask

If yes…Would the contract be in Spanish?

  • Yes, it was in Spanish
  • No, it was not in Spanish

Comments: (please write any other observations)
__________________________________________________
__________________________________________________
__________________________________________________
__________________________________________________
__________________________________________________
__________________________________________________
************************************************************************

[FOR OFFICE USE ONLY:]
Is this immigration service provider an Agency or an individual?

  • Agency
  • Individual
  • Other: __________________

Is the agency a registered business or registered non-profit (501©3) organization?

  • Registered business
  • Non-profit
  • Other: __________________

Was a Lawyer present?

  • Yes
  • No

If lawyer:
Name of Lawyer: _______________________________________
Is the attorney accredited by the New York State Bar?

  • Yes
  • No

Is this lawyer an immigration specialist?

  • Yes
  • No

If not a lawyer:
Name of person: ____________________________________________________
Does the person have any type of licensing?

  • Yes___________________________________

Type of license: _____________________________
License #: _________________________________

  • No

For both lawyers and non-lawyers:
Are there complaints filed against this person?

  • Yes
  • No

Where are the complaints filed? Details ________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
What information turns up about this person (either lawyer or non-lawyer) through an internet search?
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________

Download
the
Case Study
Here

Case Study: 3.5 New Immigrant Community Empowerment’s Report: Dreams and Schemes in Queens, New York

Download
the
Report
Here

Background on Organization and Issue

New Immigrant Community Empowerment (NICE) is a community-based, non-profit organization that works to ensure that new immigrants can build social, political and economic power in their communities and beyond.
Every day, immigrants in New York City struggle to find work, support their families, and understand their immigration options. In this process, many seek assistance from a variety of services and businesses targeted at immigrant consumers. Aware of the vulnerability of new, primarily undocumented immigrants, many businesses, and individuals target and prey upon this community. Of note are the practices of Immigration Service Providers/Immigration Attorneys and Employment Agencies, services and institutions that defraud the community..
In recent years, members of NICE have consistently reported negative experiences with these providers, prompting NICE to investigate, analyze, and develop solutions. In order to gather evidence for NICE’s campaign for better regulation and oversight of these predatory services, NICE and UJC developed a participatory action research project to document this problem.

Below is a description of the NICE Immigrant Consumer Fraud Research Project, based on the Participatory Action Research guiding framework (see Tools 2.1 and 2.2).

WHAT…

Were the Organizing Goals connected to this research?

  • To document predatory, substandard and fraudulent practices of services targeted at new immigrants in Jackson Heights, Elmhurst and Corona, Queens.
  • To highlight the gaps in government enforcement and oversight of services targeted at new immigrants.
  • To improve government enforcement and increase regulatory standards for immigrant service providers/immigration attorneys and employment agencies targeted at new immigrants.
  • To empower community members to access better services and understand their rights as consumers of these services.
  • To improve the lives of low-wage, newly arrived immigrants by curbing predatory, substandard and fraudulent practices and services in their communities.

Overall questions did NICE want to answer through their research?

  • How are the lives and economic well being of new immigrants impacted by predatory, substandard, and fraudulent services?
  • Why do new immigrants use these services?
  • What are the current practices of services targeted at new immigrants, such as Immigration Service Providers and employment agencies?
  • What are enforcement and regulatory shortfalls for services targeted at this community?

Information did NICE need to collect to answer these research questions?

  • Relevant laws and regulations governing immigration service providers/immigration attorneys and employment agencies.
  • Primary data through mystery shopping visits about how businesses present their credentials and services to consumers.
  • Secondary data about businesses and individual providers to support the data from mystery shopping.
  • Stories from new immigrants about being the victims of consumer fraud.
  • Data on the immigrant population in the Jackson Heights, Corona, and Elmhurst neighborhoods of Queens, New York.
  • Stories about the effect of consumer fraud on new immigrants.

WHY…

Is this research useful or important to NICE?

  • Internally: to develop the leadership of its members through their direct participation in the research; to strengthen their Immigrant Consumer Justice campaign by gathering data and creating a report that can support the campaign goals; and allow NICE members to share their stories and experiences.
  • Externally: to educate New York City and State government officials about the effect consumer fraud has on the community as a whole, and immigrants in particular; to get media attention towards the need for more regulation and laws governing immigration service providers/immigration attorneys and employment agencies; to pass legislation clearly defining, and more effectively prohibiting, the unauthorized practice of law; to advocate for amending existing law to prohibit the charging of advance fees to only the most vulnerable, low-wage workers.

WHO…

Are the stakeholders in this issue?

  • Immigrants seeking to adjust their immigration status and find work.

Is NICE trying to influence?

  • Members of the NYS Assembly and Senate, NYS Attorney General, NYC Department of Consumer Affairs, County District Attorneys, NYS Bar, The Mayor’s Office of Immigrant Affairs, and the Executive Office for Immigration Review.

HOW…

Did NICE gather information (what methods did they use)?

  • NICE used mystery shopping to document how immigration service providers/immigration attorneys and employment agencies routinely violate the law and how this leads to new immigrants being the victims of consumer fraud. The mystery shoppers, predominantly immigrants from Mexico and Ecuador, used assigned scenarios to pose as customers seeking services. Each was paired with an observer who looked for signage and credentials and collected literature and business cards. Upon completion of the visit, each shopper-observer pair documented their interactions on a standardized form designed to evaluate providers’ compliance with relevant laws and regulations. NICE also used secondary research, surveys, focus groups, census analysis, and legal research in their data collection process.

Did research support NICE’s organizing efforts?

In 2012, NICE released the final report summarizing the research findings and policy recommendations. The report detailed the extent to which consumer fraud affects immigrants. The release was attended by over 80 people community members, academics, the press, New York City Council member Daniel Dromm, and representatives from various city, state, and national government offices, including: US Senator Kirsten Gillibrand, NYS Senators José Peralta and Jose Serrano, NYS Assembly Member Michael DenDekker, NYC Council Members Leroy Comrie, Julissa Ferreras and Jumaane Williams, the New York Attorney General, the Mayor’s Office of Immigrant Affairs, and the Manhattan District Attorney.

Did research impact policy change?

In December of 2016, Governor Cuomo signed the Justice for Jobseekers Bill into law, after years of advocacy work by the Justice 4 Jobseekers campaign, led by New Immigrant Community Empowerment. The bill seeks to prevent fraud in employment agencies by providing stricter licensing and practice procedures, and avenues for enforcement.

Click here to read the report.