Data Collection

This is first step of all Data Science projects. And just like the name states, it is simply the step where we obtain all available data needed from various data sources.

The way to go about Data Collection is strongly based on the problem which is to be solved. There are various ways of gathering data which includes:

• Web scraping.
• Querying databases.
• Questionnaires and surveys.
• Reading from excel sheets and other documents.
• Other crowd-sourcing methods.

Data Cleaning

Data cleaning is the process of identifying and removing (or correcting) inaccurate records from a dataset, table, or database and refers to recognizing unfinished, unreliable, inaccurate, or non-relevant parts of the data and then restoring, remodeling, or removing the dirty or crude data.

Data cleaning techniques may be performed as batch processing through scripting or interactively with data cleansing tools.

After cleaning, a dataset should be uniform with other related datasets in the operation. The discrepancies identified or eliminated may have been basically caused by user entry mistakes, by corruption in storage or transmission, or by various data dictionary descriptions of similar items in various stores.

Data Processing

This involves the massaging and manipulation of data to get the necessary insights, trends, and patterns. This process covers Data Exploration and Model Development.

Data Exploration is used to understand, summarize and analyze the contents of a dataset, usually to find answers to the existing problem or to prepare for model development. This is where Exploratory Data Analysis (EDA) comes in. The data at this step is critically studied, insights deduced, outliers taken care of and new features engineered if there is a need to.

Model Development involves the provision of a statistical algorithm with data to learn from. This process is known as Machine Learning. The learning algorithm finds patterns in the data used for training that maps the input features to the target variables; the output is a Machine Learning (ML) model that captures the discovered pattern.

Data Visualization

Data Visualization is the process that helps in the communication of the insights and patterns discovered or found in the data. This involves the direct interpretation of the data in a non-technical way, that the business can relate to. It also comes with actionable insights that were discovered through the Data Science process.

This step is where storytelling comes in. It is always advisable to let your data tell a story as it is one of the most effective ways of communicating your results.