3
$\begingroup$

I'm working on a project using an E-commerce dataset. I'm facing an issue in the data cleaning stage. I have the customers dataset, which has approximately. 1.6 million rows. One of the feature, "Job Title" in the dataset has around 600K Null values. Description of the Customer Dataframe

Should I Impute the Null values with values like "NA" or "Ignored", Or should i drop them? I can't afford to drop NA rows as they're connected to other datasets. Is there any way to fill these Null values in correct way for this situation? First 5 rows of the Customer Dataframe

This is the first 5 rows of my Customer DataFrame.

Improve this question
$\endgroup$
1
  • $\begingroup$ This question is similar to: Handling categorical missing values ML. If you believe it’s different, please edit the question, make it clear how it’s different and/or how the answers on that question are not helpful for your problem. $\endgroup$ Commented Apr 21 at 10:31

2 Answers 2

Reset to default
4
$\begingroup$

If dropping rows isn’t an option due to connections with other datasets, the best approach is to impute missing "Job Title" values with a neutral placeholder like "Unknown" or "Not Provided". This maintains relational integrity and avoids introducing bias from incorrect assumptions.

Additionally, consider creating a binary flag column (e.g., "Job Title Missing") to indicate which entries were originally null. This can be useful for downstream analysis or modeling, as the missingness itself might carry predictive value.

If "Job Title" plays a significant role in your analysis, and you have strong correlated features like age, income, industry, or transaction history, you could explore more advanced imputation using classification models or clustering techniques. However, simple placeholders are often sufficient if the field is not central to your objectives.

Improve this answer
$\endgroup$
0
0
$\begingroup$

And to add on to what @Philipp said, if you need the code to fill the null values it would be:

df['Job Title'].fillna('N/A', inplace=True)
Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.