Importance of qualitative data mining in a research

By Ashni walia & Priya Chetty on October 4, 2019

Data mining is an essential part of any research where testing of a hypothesis or a framework or a model is required. Qualitative data mining plays a crucial role in most fields of study such as business, social sciences, and humanities. The methods of mining data may vary according to the discipline, but the principle procedure is the same for all. It starts with finding out what data is required for the study, followed by the selection of the sample population.

When first-hand data is mined through primary sources of information, such as the public or an entity, it is referred to as ‘primary data’. There are many methods of primary data mining. These include surveys, interviews, focus group discussions and experiments. They are broadly classified into two types:

  1. quantitative and,
  2. qualitative.

What is qualitative data?

Qualitative data is non-numeric in nature and is mined in the form of words, and phrases. It often focuses on capturing the feelings, emotions or perceptions of an individual. It mainly addresses the ‘how’ and ‘why’ of a phenomenon. Unstructured methods of data mining are used for qualitative data. Another important function of qualitative data is to support the statistical findings of a research problem.

The questions are open-ended i.e. the questions that simply cannot be answered in the “yes” or “no” or pre-decided list of options. Such questions require statements to be framed by the interviewee. Hence the qualitative approach to data collection is good for exploring the effect or the consequences of the certain program. Popular qualitative data collection methods are focus groups, observations and interviews.

The main strength of qualitative data is its ability to provide complex descriptions of how people feel or experience a phenomenon. Thus, it presents the human side of an issue like behaviour, beliefs, opinions and emotions. This approach to data collection is also helpful in exploring intangible factors such as social norms, socioeconomic issues, gender roles, ethnicity, etc. Such factors are otherwise hard to explore.

Face-to-face personal interview method of qualitative data mining

This method of qualitative data mining is primarily based on a personal approach. The interviewer collects data directly from the subject on a one-on-one basis. This approach is most useful when the data collected must be highly personalized. The questions involved in the interview are generally unplanned. They take the shape of the flow of the interview. However, if the interviewer wants to standardize the data to some extent for easier analysis, a semi-structured interview can be conducted.

However, this method does have certain limitations as well, such as language barriers, cultural differences and geographical distances. Moreover, the person who is conducting the interview must possess excellent interviewing skills in order to elicit accurate responses.

Focus group discussion method of qualitative data mining

This method is similar to face to face interview method. The primary difference is that focus groups involve a group discussion setting. This method is useful when the objective of the data is to observe attitudes and behaviours in any social situation. Depending on the data being sought, the members involved in the discussion will be selected.


In a study on the impact of the marital status of women on recovery from alcoholism, the focus group will involve women who are married, alcoholics, and have been recovering from alcoholism.

The topic on which the data must be collected will be presented to the group and the moderator will open the floor for the discussion. The success of this generally lies in the hands of the moderator. The interviewer should be capable and efficient in controlling such discussions.

Observational method of qualitative data mining

Under this method, the researcher immerses in the settings where the potential respondents are. It involves the participatory stance on the part of the researcher. Therefore, they observe everything in that setting and take notes. Apart from taking notes, the researcher can use other documentation methods such as video recording, photography or audio recording. The data collected through observation is reliable and representative since it was observed by the researcher. However, one of the major drawbacks of this is the risk of validity. The researcher’s participation may have an impact on the natural setting. Their presence may cause the subject to behave or react differently.

Field surveys or questionnaires

This method comprises short questions and is mostly open-ended. The respondents are asked to provide the answer, in their own words. Since this method involves structured questionnaires, it is ideal for use in large populations. However, the detailed answers provided by a large number of respondents make the data analysis process quite difficult and time-consuming.

Case study method of mining qualitative data

Another method of mining qualitative data is taking a close look at the case studies that are present in the research problem. The versatility of this method lies in the fact that it can be used in the case of both simple and complex subjects. It can help strengthen the case by capturing more variables. Though this is a versatile and flexible method, the reliability of data is always under question.

Common challenges

Qualitative data mining and analysis is not as easy as it seems. Rather it involves complex theoretical and philosophical frameworks. It also involves rigorous analysis without the aid of mathematical tools. For example, experiences related to health, illness, and medical intervention cannot simply be counted and measured, here the researcher needs to understand the underlying meaning of the respondents.

Priya is the co-founder and Managing Partner of Project Guru, a research and analytics firm based in Gurgaon. She is responsible for the human resource planning and operations functions. Her expertise in analytics has been used in a number of service-based industries like education and financial services.

Her foundational educational is from St. Xaviers High School (Mumbai). She also holds MBA degree in Marketing and Finance from the Indian Institute of Planning and Management, Delhi (2008).

Some of the notable projects she has worked on include:

  • Using systems thinking to improve sustainability in operations: A study carried out in Malaysia in partnership with Universiti Kuala Lumpur.
  • Assessing customer satisfaction with in-house doctors of Jiva Ayurveda (a project executed for the company)
  • Predicting the potential impact of green hydrogen microgirds (A project executed for the Government of South Africa)

She is a key contributor to the in-house research platform Knowledge Tank.

She currently holds over 300 citations from her contributions to the platform.

She has also been a guest speaker at various institutes such as JIMS (Delhi), BPIT (Delhi), and SVU (Tirupati).


I am a master's in Economics from Amity university. Besides my keen interest in Economics i have been an active member of the team Enactus. Apart from the academics i love reading fictions. 


1 thought on “Importance of qualitative data mining in a research”