Click here to Skip to main content
15,888,461 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
I have a column with mixed values as below:
Mixed Data - column name
100 socks
3467867327 - remove
94746723892 - remove
400 order
oils
essentials
b01232435 - remove
b0054543543543 - remove
garments
787 sales

I want to filter the column and remove the rows which are marked as remove above. I can achieve removing the string starting from "b0" but I am unable to apply filter to only numeric values. dtype is object of the columns. Below code does not work for me nor do I want to create a separate column to convert string to numeric.

pd.to_numeric(df['Mixed Data'],errors='coerce').min()

Note - the actual data is of huge count hence I need to a generic code and not specific code. Please help me.


What I have tried:

import pandas as pd
df = pd.DataFrame.from_dict(
    {
        "Mixed Data": [
            "100 socks",
            "3467867327",
            "94746723892",
            "400 order",
            "oils",
            "essentials",
            "b01232435",
            "b0054543543543",
            "garments",
            "787 sales",
        ],
        "Some other column": [
            "a",
            "b",
            "c",
            "d",
            "e",
            "f",
            "g",
            "h",
            "i",
            "j",
        ],
    }
)
df = df[df["Mixed Data"].str.startswith("b0") == False][
    df["Mixed Data"].str.isnumeric() == False
]
print(df)


the above ends up deleting all the records from the table. hence it does not work for me
Posted
Comments
Richard MacCutchan 8-Aug-22 8:31am    
I just ran your code and it does not delete all the records. Here is the output:
python cp.py
cp.py:30: UserWarning: Boolean Series key will be reindexed to match DataFrame index.
  df = df[df["Mixed Data"].str.startswith("b0") == False][
   Mixed Data Some other column
0   100 socks                 a
3   400 order                 d
4        oils                 e
5  essentials                 f
8    garments                 i
9   787 sales                 j

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS


CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900