Methodology for Products & Services

In today’s cluttered marketplace being able to identify the best product or service quickly and with confidence is highly valued and appreciated. After all, it’s a fact that over 90% of female consumers value the opinion of others when making purchasing decisions. The Women’s Choice Award has identified America’s most recommended brands and services to help women make informed consumer choices.

‘9 Out of 10 Recommended’ Award Methodology:

Brands may qualify for the Women’s Choice Award, ‘9 Out of 10 Recommended’ designation via 2 qualification processes:

National, Unaided Survey: National, unaided surveys ask women across America if they would recommend a specific brand. So, instead of comparing brands, women are asked if they would recommend a specific brand to their friends and family. Within the survey, each question includes product images to provide additional guidance. And, answer choices allow participants to select ‘N/A’ if they have not had experience with the brand in question.

On average, each brand-specific question will receive approximately 350 responses, of which at least 100 participants must have had experience with the brand/product and provide their recommendation rating. Any brands that earn over an 85% recommendation rating qualify for the Women’s Choice Award, ‘9 Out of 10 Recommended’ award.

Customer Database Survey (Phase I) and Testing Panel (Phase II): Brands may qualify through a 2-part qualification process. Phase I involves conducting a survey of the brand’s database of customers, requiring an 85% recommendation rating in order to move on to phase II, the product or taste testing panel.

Phase I: Customer Database Survey

A customized survey of 4 or 5 mandatory questions is developed. These questions include questions on gender, general timing of the customer’s most recent purchase of that product or service, zip code (for a chain or multiple location retailer), customer satisfaction rating, and whether the customer would recommend the company to his/her friends and family. The company is required to send the survey to their entire database of customers until the required number of responses are obtained. The number required varies based on size of the database, but the general requirement is to receive at least an 8% responses rate from the customer database. In cases in which emails are non-existent or not plentiful, a company may share their database with us to obtain responses through a phone survey.

Companies must earn over an 85% recommendation rating from their female customers in order to qualify or move forward with the phase II testing panel.

Requirements, limitations and processes are outlined as follows:

Surveys are developed and monitored by Women’s Choice Award and distributed to both male and female customers of the company’s database.
Surveys may be conducted electronically, via phone, or through a combination of the two in an effort to obtain a statistically relevant sample size.
A statistically relevant sample size will be dependent upon objective factors such as the size of the company’s customer database, number of female responses, and responses from customers who have made a transaction in the past 3 years.
Survey qualification is valid for a period of 4 years.
Companies that have not been in business for at least one full calendar year, are not eligible for qualification. Additionally, products or services that have not been in the marketplace for at least twelve (12) consecutive months prior to qualification are not eligible for qualification.
Companies that do not qualify are provided with the results of their survey and a brief overview of those results in an effort to support the company’s desire to provide their female customers with a superior customer experience. Companies that do not qualify are permitted to attempt the qualification process again after 6 months.
Companies do not pay to participate in the electronic survey qualification process but do enter into a marketing program and licensing agreement upon qualification. If a telephone survey is required to determine qualification, a $250 non-refundable fee is required. If an alternate means of qualification is determined and requires administrative time or customer incentive, the brand is required to pay those fees at-cost and/or provide the product for consumer testing.
Companies that have conducted an outside survey may utilize those results for qualification if the following survey criteria are met:
- Conducted by a credible, third-party survey company
- Conducted within the last 4 years
- Results demonstrate a statistically valid sampling of the company’s database
- Results can be segmented to display female responses
- Inclusion of one survey question regarding the propensity of customers to recommend that company

Phase II: Testing Panel

Service companies that can only be evaluated based upon customer service experience are required to complete Phase I, but are exempt from Phase II. Examples of service companies include but are not limited to: moving companies, storage facilities, phone services, rental car companies. In addition, products that can only be evaluated based on preexisting conditions may also be exempt from Phase II testing. Examples of products may include: products that treat a medical condition such as migraines, cold sores, specific illnesses, etc.

Phase II qualification may take place in a focus group setting or through a product testing or taste testing panel. Focus group participants or panelists are required to analyze the product(s) using our 9-point criteria, as well as the 5-point Likert scale and the 9-point Hedonic scale to gauge overall likeability and additional features, where applicable.

Products that require an extended review (i.e. products that are being tested for effectiveness over a prolonged period of time), may require a longer testing/review period. The extended period of time will be determined by the nature of the product and the length of time that is required to determine effectiveness.

When all data is gathered from panelists, the results are analyzed using a weighing system to determine qualification. Criteria factors may include:

Safety
Ease of use
Durability
Design
Composition
Warranty
Competitor comparisons
Overall Likability
Likelihood to recommend

Criteria for Taste Testing Panels:

Sensory Evaluation Using The 9-Point Hedonic Scale:

Appearance/Presentation
Smell/Aroma
Taste/Flavor
Texture/Consistency
Overall Likability

Additional Taste Testing Panel Criteria:

Ingredients
Product packaging
Blind taste testing (i.e. using comparable products on the market to determine preferences)
Likelihood to recommend

‘America’s Most Recommended’ Award Methodology:

Products and services earn the Women’s Choice Award ‘America’s Most Recommended’ designation based on national, aided surveys. These surveys ascertain which brands are most recommended by women across America. Categories in which products are surveyed include: Automotive; Beauty & Wellness; Best for Mom & Baby; Home; Pet Care; and Travel.

The survey process is simple and objective. Using our research team, survey committee, and/or advisory board, we create surveys specific to an industry and its categories, listing all of the leading products and services that are recognized on a national level. Utilizing our database of women nationwide, we conduct surveys resulting in approximately 500 – 1,200 unique responses. The number of survey responses for each category may vary depending upon statistical validity needed when several brands are taking the lead versus when there is a clear front-runner. By obtaining a minimum of 500 responses, there is only a 5% error of margin. When obtaining 1,000 or more responses, there is only a 3% or less margin of error.

Surveys are distributed to anywhere from 10,000 to 200,000 women to obtain the targeted number of responses. Responses are provided solely through the participation of female consumers across America taking the survey, and is therefore, entirely objective.

To obtain statistically relevant data, survey demographics are evenly spread throughout the nation’s population and may include key demographic segments such as gender, region, age range and household income.

A product or service that earns the award, maintains its standing for 4 years, at which point surveys must be redistributed to determine updated award recipients.

Formulating the Questions

All questions ultimately have the same contextual format; women are asked ‘When selecting the best in [category], which of the following brands/services would you most recommend? Select all that apply’. The questions are formulated so that women are not forced to choose any one particular product or service. We recognize that there are numerous products and/or services within any given category that may have earned the trust of our female consumers, and therefore we provide our participants with the option to ‘select all that apply’ in an effort to achieve the most accurate information. We also provide an option for women to elect ‘Not Applicable’ or ‘Other’ so that our results are not skewed and women are not forced to provide a response if they have not had enough experience with any of the products or services listed.

Recommendation Rating vs. Satisfaction Rating

Our gender study research, conducted with the Wharton School of Business demonstrates that a recommendation carries a much stronger value than simply asking about satisfaction ratings, so we have created our surveys based on this important premise. When a woman is satisfied, it means very little to gauging her level of loyalty to the brand. However, when a woman is willing to recommend a brand or service to others, it means that that business has earned her loyalty and those are the businesses we wish to recognize for their efforts and commitment to the number one consumer- women.

Statistical Ties and Validity

A statistical tie occurs when two or more brands taking the lead in a survey have earned responses within 3 percentage points of each other. Additional factors related to percentage points accrued and validity are as follows:

(a) Within any national survey, if the brand that receives the most votes in one category only accounts for 5% or less of the total votes, the survey responses are not considered to be statistically valid and will either be disqualified or the survey will be redistributed until the requisite percentage of votes is reached. (b) If the brand that receives the most votes in one category accounts for more than 5% but less than 10% of the total votes in that particular category, the responses from the ‘N/A’ section are disqualified and the percentages are recalculated accordingly. By revising the percentages to account for only those responses that selected a specific brand, the percentage differences between the respective brands widen, thereby allowing the most recommended brand to emerge more definitively. In these instances, the 3% range policy for statistical ties is applied accordingly to the revised percentages. (c) If the brand that receives the most votes in one category accounts for 10% or more of the total votes in that particular category, the responses are considered to be statistically representative as long as a requisite number of survey responses are obtained.