Quality Control Policy
Macromill will continue to provide the highest quality data in accordance with the following quality control policy.
- Introduction
- Monitor Quality Control
- Data Quality Control
- Information Security Management
Macromill's Quality Control
We aim to provide "reliable" data of the highest quality in the industry.
Macromill is committed to the construction, operation and management of high-quality monitors based on the following monitor policy.
Internet research is now the most representative research method in Japan and is used by a wide variety of companies. This is because the major characteristics of Internet research - high quality, low cost, and speed of delivery - have enabled companies to make quick marketing decisions.
When Internet research first emerged, issues such as the representativeness of survey participants, fraudulent registration, and data comparisons with conventional survey methods were pointed out. The survey industry as a whole has been active in promoting quality control efforts and understanding of appropriate usage, and today it is trusted as the most representative survey method.
Macromill places particular emphasis on providing reliable data. In addition to organizing a dedicated quality control team to constantly pay attention to quality, we are also implementing various initiatives to ensure high quality, such as eliminating fraudulent responses by utilizing technology and developing new quality control methods that utilize the results of specialized research.
In recent years, information leaks have become a major problem. Macromill believes that part of quality is to protect the information entrusted to us by our customers, and therefore we are actively working to strengthen information security.
We will continue to pursue the best quality control methods "in response to the times.
The rapid evolution of ICT and the widespread use of smartphones have brought about major changes in the lifestyles, work styles, and values of many people, and these changes will continue to accelerate in the future. These changes also affect the environment and attitudes of survey respondents, and consequently have a significant impact on the accuracy of their responses. All of this means that continuing quality control in the same manner as in the past will lead to a decline in quality.
Macromill will continue to pursue optimal quality control methods in response to the rapidly changing times to ensure stable data provision.
Macromill's Quality Control System
Macromill has established a strict quality control system in terms of "monitor quality control," "data quality control," and "information security control" in order to provide data that not only satisfies our customers but also earns their "trust".
Macromill Monitor Quality Standards
Pure Research-Only Panels
Macromill manages its monitors as pure research-only monitors. We do not distribute direct mail or present advertisements for sales promotion purposes.
*From the standpoint of monitor management, we may not be able to accept surveys on sensitive topics such as politics and religion.
- Adherence to marketing research industry charter
- Macromill Monitor's quality control complies with the "ICC/ESOMAR International Code of Conduct" of ESOMAR (European Society for Opinion and Market Research), which is an international code of conduct for the marketing research industry, and the "Marketing Research Code of Conduct" of the Japan Marketing Research Association. The Company is a member of the following organizations
The monitor must be able to answer the survey correctly.
The "number of active participants" announced by Macromill is the number of participants who can be correctly requested to participate in a survey. Macromill does not include the following types of monitors in the "number of valid survey participants.
- Monitors not included in the valid monitors
- Excludes monitors with unreachable email addresses or incorrect registration.
Monitors who found extremely inconsistent or fraudulent responses.
Monitors who have not responded to questionnaires for a long period of time
Monitors that determine that the registration is fraudulent (duplicate or false)
Those who are deemed by Macromill to be inappropriate as survey participants for other reasons.
monitor quality control cycle
Efforts to maintain high quality
Quality control during monitor acquisition and monitor registration
When acquiring monitors, we take into consideration the balance of the panel to ensure that it is representative of all Internet users in Japan, and conduct systematic acquisition of monitors. In addition, we have implemented a function in our system to exclude fraudulent registrations.
- Attribute-conscious monitor acquisition
- We strive to acquire monitors who are not biased toward any particular demographic, such as gender, age, or occupation.
- Monitor acquisition tailored to the multi-device era
- In order to acquire not only PC users but also smartphone and tablet users, we ensure the representativeness of the latest Internet users by placing advertisements in various media such as SNS and news applications.
- Prohibition of duplicate registration
- To prevent duplication when registering monitors, the system implements a function to prohibit the following types of registrations.
(1) Registration with the same e-mail address
(2) Registrations with duplicated information other than e-mail address.
If either (1) or (2) above applies to you, we will consider you to be the same person and will not accept your registration.
- Confirmation of email arrival at registration
- To prevent registration by fictitious e-mail addresses, the system implements a function to confirm the arrival of e-mail addresses before registration is completed.
Quality control after monitor registration
Once registered, monitors are subject to various checks by a dedicated monitor quality control staff member. Monitors whose registration information or response attitude is determined to be problematic are suspended from membership and excluded from the distribution of survey request e-mails.
- Primary check for fraudulent registration information [Weekly]
- We check the registration information of newly registered monitors. If the registration information is found to be fraudulent, we consider it to be a fraudulent registration and suspend the membership.
- Secondary check for fraudulent registration information [Monthly]
- We check the registration information of all monitors. If the registration information meets some duplicate conditions, we consider the registration to be an impersonation and suspend the membership.
- Inactivity check [Monthly]
- If a participant has not responded to a survey for more than a certain period of time, we will assume that he/she has no intention of responding to the survey and suspend his/her membership.
- Detecting and dealing with fraudulent respondents through trap surveys [Semi-annually].
- We conduct "trap surveys" to identify fraudulent respondents. Trap surveys are conducted using multiple surveys with traps placed throughout to detect inconsistent responses within the same survey instrument and inconsistent responses across multiple surveys. Error points are calculated based on the degree of incorrect responses, and monitors who exceed a certain standard are excluded from the valid monitors.
- Mandatory attribute information updates [Annually]
- Attribute information is updated once a year by performing a "mandatory update of registration information" for all monitors. This means that monitors must confirm and update their registration information to participate in the survey, regardless of whether or not their registration information has changed.
- Detection of fraudulent survey responses [Always]
- If inconsistent or inauthentic responses are found more than once within a certain period of time, we take measures to automatically exclude the monitor from the list of valid monitors.
Efforts to maintain a stable supply
User Measures in the Multi-device Era
The number of survey participants who take surveys not only from PCs but also from smartphones and tablets is increasing every year, and this trend is especially noticeable among the younger generation. We are building a multi-device environment, for example, by constructing a survey screen that allows participants to answer surveys from both PCs and smartphones using the same URL, and by developing a response application so that participants can answer surveys via the application.
Strengthening Engagement with Monitors
In order to encourage continuous survey responses, we are actively promoting communication measures that strengthen engagement with our monitors, such as sending thank-you emails and birthday messages, survey campaigns, and producing goods featuring Macromill's original character "Anke-To".
Use of affiliated monitors
By using monitors from partner companies, we are able to secure subjects with a low frequency of occurrence and collect large samples. When using partner monitors, the system controls the distribution of survey requests to the same monitors in order to improve the quality of response data.
- Duplicate checks by the system at the time of collaboration
- When Macromill monitors and monitors of partner companies are used together, the system automatically checks for duplicate registrants and delivers the survey excluding those who have registered in duplicate.
- Automatic checks by the system when responding
- In order to prevent double responses from monitors who are registered as both Macromill monitors and affiliated monitors and who could not be detected by the system's duplicate check, the system identifies the cookie information of monitors when they answer a survey and controls the system so that they cannot answer the same survey twice or more on the same device.
Data quality standards
The data should be highly market-reflective.
Macromill remains committed to providing data that is unbiased and market-reflective, i.e., highly representative of the survey results. To this end, we are continually working on the following initiatives.
- Research for representativeness and data quality
- Understanding of monitor characteristics and attribute trends
Identification of data trends with other survey methods
Understanding Data Trends by Respondent Device
Develop appropriate survey volume, survey design, and educational activities
- Macromill Monitor Lifestyle Survey
- In order to understand the characteristics of our monitors, we conducted the "Macromill Monitor Lifestyle Survey" and compiled the overall picture in this report. We believe it is important to understand the characteristics of online research monitors in advance in order to obtain better survey results. This research report compares the results of the survey with those of other research methods, such as in-home visits, to fully grasp the characteristics of the online research monitors.
The data must be correctly answered.
In order to deliver high-quality data that supports our clients' marketing decision-making, Macromill has established a checking system for each operational process and delivers only high-quality data that has passed the checks in each process.
data quality control cycle
Thorough process control
Pre-inspection verification process
- 1. check the survey form
- In the screen creation process prior to the start of the actual survey, the RD (Research Director) checks the survey forms for all surveys requested by the customer from the perspective of "response load" and "response accuracy" and makes suggestions before starting the actual survey.
[Check Point of View]
1. is the answer format appropriate?
2. whether there are individual differences in interpretation
3. whether there are too many or too few options
4. whether one question contains two or more issues
5. are the response instructions sufficient?
6. does it contain any industry terminology or jargon that is difficult to understand?
7. whether the pre-survey makes it easy to guess the conditions to be covered by the main survey
- 2. check the created survey screen and logic settings
- The questionnaire screen created by RD will be checked by a dedicated third-party checker based on the requested survey form to ensure that it faithfully reflects the survey and that there are no inconsistent responses in the process of answering the questionnaire.
Multiple checks are performed prior to the actual inspection to reduce human error and ensure data quality.
- 3. check for appropriate number of deliveries and collections
- Appropriate distribution settings are made to reach the target number of questionnaires collected in 24 hours. 24 hours or more is set as the actual survey period to improve data quality by reducing the ratio of early respondents and bias due to occupation and respondents' lifestyles.
Pre-delivery verification process
- 4. data recovery
- *See System Functions that Enhance Quality
- 5. checking of aggregate data
- In order to confirm that the collected survey data is properly answered data, data checks are performed by our expert staff in tabulation, utilizing our proprietary check sheets developed through our extensive survey experience. This is how we maintain and improve the quality of delivered data.
[Check Point of View]
1. whether the data conforms to the survey conditions (allocation, number of samples collected, and subjects of tabulation)
Are there any inconsistencies in the questionnaire?
3. is the weight back appropriate?
4. are there any inconsistencies when integrating data from different surveys?
5. are there any inconsistencies in the data of the tabulation results?
- 6. check the report prepared
- The report is checked by the preparer and a dedicated third party to ensure that it has reached a level of completeness worthy of delivery, and is finally delivered to the customer.
[Check Point of View]
1. are the scores on the graphs and tables of numbers correct?
2. are the supplements and notes correct?
3. is the content consistent with the data?
4. is there consistency in fonts, colors, and layout?
5. are the comments consistent with the purpose of the survey?
System features that enhance quality
Functions to improve response accuracy
Macromill's online research system has functions to control various survey screens. In addition, these functions are enhanced through periodic version upgrades to improve the accuracy of responses.
- Survey screen control function
- Required Response Functions
This function displays an error if the respondent has not answered the question, and ensures that the question is answered.
Pop-up viewing required function
This function displays an error if a user tries to proceed with a response without clicking a button when another screen, such as a product image, is popped up in the survey form.
Select condition (respondent only) function
When set for a question for which you want to limit respondents, this function prevents the question itself from being displayed to respondents who do not meet the specified conditions.
Piping Function
This function displays only the choices selected in the previous question in the next question.
Exclusion Contradiction Checking Function
This function allows an error message to be displayed when a multiple-answer question has a choice that, if selected at the same time as another choice, would result in a contradictory answer (e.g., "None of the above apply").
Choice randomization / Question randomization
This function randomly switches the order of choices and questions for each respondent. This allows for highly accurate response results.
Functions to improve accuracy of delivered data
- Ability to exclude incorrect responses by registering NG words
- The system has a function that does not use as aggregate data any NG words (e.g., aaa, aaa, etc.) that have been pre-registered in the free response.
- Ability to exclude responses from being counted
- The system functions not to use as aggregate data any responses that are clearly inconsistent between the pre-survey (screening survey) and the main survey, or any an inattentive response attitude is observed.
- Response Time-Based Exclusion (*patented)
- Macromill has developed a proprietary system function for deleting short-time respondents. This function reduces the problem of the general method of deleting short-time respondents, and provides more accurate and higher quality survey data by extracting and deleting the most appropriate short-time respondents (3%). (Patent registration date: August 5, 2011 Patent number: No. 4795496)
Research activities aimed at improving quality
Appropriate survey design in the multi-device era
Internet Research Design Policies for the Multi-Device Era
-
The spread of smartphones has changed the devices used by respondents to answer surveys, and it is important to design from the viewpoint of respondents, taking into account the survey volume, the number of matrix questions, the number of options, and the visibility of the screen itself.
There are concerns that the quality of the survey may deteriorate due to the effects of designing a survey with too large a volume or poor visibility, such as an increase in the number of respondents leaving the survey in the middle of the survey or refusing to answer, or a decrease in the response rate to the survey itself. In addition, this could lead to the withdrawal of monitors, which in the long run could affect the continuous provision of Internet research services.
Macromill has created the following "Internet Research Design Policy for the Multi-device Era" to ensure that Internet research will continue to be utilized as an ongoing and beneficial tool in the future.
Internet Research Design Policy
- I. The time required to complete this survey is "10 minutes" or less.
- II. Response time for the pre-survey is "5 minutes" or less.
- III. Matrix question items x choices are "10 x 10" or less.
- IV. SA（Single Answer）/ MA（Multiple Answer） options are "30" or less.
- V. Question text should be no more than "50 characters(standard for Japanese surveys)".
Surveys and research by professional organizations
- Permanent online research quality control study
- Internet access devices are shifting from PCs to smartphones. In line with this shift, the ratio of smartphones as Internet research response devices is also increasing. In order for online research to continue to exist as a necessary infrastructure for marketing research in the future, we are constantly conducting in-house research from an objective standpoint on how to design survey forms that are easy to answer and how to present survey screens.
- Creation of research methods using the latest technology
- With the rapid evolution of IoT, not only information and communication devices such as PCs, but also various other devices are becoming equipped with communication functions.
Until now, we have relied on questionnaires that rely on the memory of monitors to understand consumers' daily behavior, but we are researching the possibility of using information automatically recorded by communication devices for marketing research. And we are conducting research to create new marketing research methods by combining consumer awareness and detailed behavioral records.
Information Security Standards
To be able to entrust us with your information with peace of mind.
As a marketing research company, Macromill considers the strict management of confidential information entrusted to us by our customers to be an extremely important responsibility. We are striving to strengthen our information security by creating a framework for promoting information security activities and providing education and training for our employees.
- Basic Policy on Information Security
- Basic Policy on Information Security
In accordance with Macromill's Information Security Policy, we have established Information Security Regulations and an Information Security Manual for employees to follow. We strive to continuously enhance information security, adapting to changes in the environment and technological innovations.
Measures must be implemented in all processes.
Macromill complies with laws, national guidelines, and other norms related to information security, and has established a management system that implements measures at all stages of the research process. Macromill strives to continuously improve its information security measures and management system.
System Information Leakage Countermeasures
Organizational and Personnel Measures
- Information Security Policy
- Macromill fully recognizes the importance of information security and considers it an important management issue in its business strategy to provide quality services that can be trusted by customers and business partners by taking high-level measures against the risk of information leakage.
In order to further strengthen information security, Macromill has established a basic policy on information security, which all directors, employees, and other relevant personnel must comply with, and will strive to maintain and build an appropriate information management system by maintaining a high level of awareness of information security. -
Macromill has obtained certification for ISO/IEC 27001:2022, the International Organization for Standardization's standard for information security management systems.
- Formulation of privacy policy
- Macromill monitors' personal information protection is in compliance with the "ICC/ESOMAR International Code of Conduct" of ESOMAR (European Society for Market and Public Opinion Research), the international code of conduct for the marketing research industry, and the "Marketing Research Code of Conduct" of the Japan Marketing Research Association. We comply with the "ICC/ESOMAR International Code of Conduct" of ESOMAR (European Society for Market Research and Opinion) and the "Marketing Research Code of Conduct" of the Japan Marketing Research Association.
In addition, Macromill has taken appropriate measures to ensure the anonymity of monitors and the storage and management of personal information, and has established a "Personal Information Protection Policy" and formulated and operates a personal information protection management system to ensure that it is appropriately and effectively implemented. For details, please refer to the "Privacy Policy" on the Macromill website.
Macromill, Inc. Privacy Policy
https://monitor.macromill.com/policy/privacy.html
Physical and technical measures
- Enhanced research system processing capacity and robustness
- Macromill's proprietary research system, "AIRs," achieves high processing efficiency and reliability by establishing dedicated servers for web services such as creating and accepting survey forms, distributing survey request e-mails, and tabulating data, and by multiplexing the servers. In addition, the performance of these servers is regularly evaluated, and appropriate reinforcements are made to ensure a high level of stability. The research system is monitored 24 hours a day to ensure prompt response in the event of a failure.
- Ensure security with firewalls and IDS
- Firewalls are installed to block unauthorized access from external networks to the research system network environment, preventing data and programs from being stolen, tampered with, or destroyed. Furthermore, we have established an even more robust security environment by adding multiple layers of firewalls. We have also introduced an Intrusion Detection System (IDS) to detect and analyze attacks on our systems, enabling us to respond quickly in the unlikely event of an accident.
- TLS encryption of communications
- To protect the privacy and confidentiality of our customers' information, as well as the personal information and responses of our survey participants, Macromill uses a strong special encryption technology called TLS (Transport Layer Security), which has been approved by a third-party professional organization for information transmission over the Internet. TLS (Transport Layer Security) is a strong special encryption technology.
On the Macromill website, information is securely transmitted and received in an encrypted security environment, such as when monitors register and respond to surveys, and when RD creates survey forms.
- Introduction of security inspections by a specialized security company
- We have introduced consulting services from an external security specialist company to diagnose the security level of our research systems and research operations. We are working to maintain and improve the level of security by conducting checks from the perspective of a third-party specialist organization and taking necessary measures to address security vulnerabilities.
Countermeasures against information leakage from monitors
Organizational and Personnel Measures
- Confidentiality to monitors
- In order to protect the information contained in surveys, Macromill prohibits its survey participants from divulging the contents of surveys (including videos and images) to third parties. We remind our survey participants of this prohibition at the beginning of every survey request e-mail and survey screen.
- Identification and confidentiality agreement at the time of interview
- In order to protect the information contained in the survey, we ask the participants to present their ID cards and verify their identities when conducting interview surveys or on-site surveys. In addition, only participants who have agreed to a confidentiality agreement are asked to participate in each survey to ensure that the content of the interview is not divulged.
- Shipping management when goods are shipped
- When conducting surveys involving the shipment of products such as trial products, we work to prevent the loss of products by shipping products with a slip for collection, confirming the arrival of the products, and managing the slips upon customer request.
Physical and technical measures
- Shipping management when goods are shipped
- The following technical security measures are implemented in the survey form to protect the information contained in the survey.
(1) Restrict access to the survey instrument to only those who are eligible for the survey through identity authentication
(2) Generation of hard-to-predict survey URLs
(3) Limit on the number of times the survey form can be displayed
(4) Content copy control of survey sheets
Prohibit the copying of questions, options, and other text in the survey form.
Prohibit the copying and storage of images or videos presented in the survey form.
(5) A numbering function that embeds a specific ID in the image in the survey form to enable tracking of the source of the leakage.
- Security Measures for Video Surveys
- The CMMill survey service, which uses video content such as commercial materials, utilizes streaming technology to play back video within the survey instrument.
By employing streaming technology that does not leave video data on the viewer's computer, we strive to protect copyrights, including unauthorized copying of video materials.
(1) Restrictions on video display by checking the domain of the display source
Videos cannot be played back unless indicated by the Macromill survey form. Even in the unlikely event that you paste a playback URL on an Internet bulletin board, etc., you will not be able to view the video because it is not an indication of playback from the survey form.
(2) Limit the playback period of videos
Security measures are in place to limit the playback period of such survey video content.
Measures to prevent information leakage from practitioners
Organizational and Personnel Measures
- Information management training for all employees
- Macromill fully recognizes the importance of information security and has established an Information Security Policy and Information Security Regulations.
In addition, to ensure that information is handled appropriately in the course of daily business operations and that information security is practiced, we provide training and periodic e-learning education to directors, employees, and other relevant personnel, focusing on "personal information protection" and including items related to the importance of intellectual property and infringement risk and its prevention, in an effort to continuously improve information security knowledge and morale. We are continuously working to improve information security knowledge and morale. In addition, information security-related materials are posted on the company intranet, where employees can access them at any time.
- Method of delivery of survey results including personal information
- Macromill uses one of the following delivery methods to protect information when survey results include personal information, such as in open-type surveys.
(1) Online delivery by "one-time URL
A URL (one-time URL) with a limited number of uses will be issued to the customer's e-mail address, which can be used only once for downloading.
(2) Delivered via a secure file delivery service specified by Macromill.
(3) We deliver media such as CD-Rs by high-security transportation service (e.g., security delivery).
- Email Misdelivery Prevention Plug-in
- Macromill has developed its own plug-ins to prevent e-mail misdirection, and has incorporated a system that prompts the user to confirm the recipient and text when sending e-mail, and to check the attached files.
Physical and technical measures
- Personal Information Management in Security Rooms
- Macromill's Personnel who handle personal information will do so in accordance with the "Personal Information Handling Manual" that Macromill has established independently.
In addition, Personnel who handle personal information practice in a security room and take the following physical measures
Secure Terminals
Personal information can only be handled by designated Secure Terminals, and applications that enable e-mail and messaging cannot be installed on Secure Terminals. In addition, memory cards and other storage media cannot be used without the appropriate consent process established within the company.
Dedicated Secure Terminal Networks
Secure Terminals can only access dedicated networks, not general servers.
Dedicated Servers for Personal Information
Personal information is stored and managed exclusively on dedicated servers.
- Multi-Factor Authentication (MFA) via One-Time Passwords
- Macromill has introduced one-time passwords for access to research system operation and management screens. By authorizing access through personal authentication using one-time passwords, we have achieved a stronger internal security environment. This system restricts access to personal information of survey subjects and confidential survey data, thereby reinforcing measures against information leaks.
