logo
down
shadow

Pandas dataframe from nested dictionary to melted data frame


Pandas dataframe from nested dictionary to melted data frame

By : michaelvv
Date : November 22 2020, 10:56 AM
This might help you if i understand properly your question, i think you can simply do the following :
code :
mdf = pandas.melt(df)
mdf['rowvalue'] = df.index
mdf
  variable value rowvalue
0   93  465  A
1   93  0  C
2   93  1  D
3   93  4  E
4   93  1  F
5   93  2  G
6   93  0  H
7   93  1  I
8   93  0  K


Share : facebook icon twitter icon
Nested dictionary from pandas data frame

Nested dictionary from pandas data frame


By : Kyaw Mann
Date : March 29 2020, 07:55 AM
This might help you You can first create column zipped by zip lat and lng and then groupby with double to_dict:
code :
#python 3 need convert to list
df['zipped'] = list(zip(df.lat, df.lng))
print (df)
  iso   city  lat  lng      zipped
0  AB  city1  0.0  0.0  (0.0, 0.0)
1  AB  city2  0.1  0.2  (0.1, 0.2)
2  AB  city3  0.2  0.4  (0.2, 0.4)
3  BC  city4  0.3  0.6  (0.3, 0.6)
4  BC  city5  0.4  0.8  (0.4, 0.8)

d = df.groupby('iso').apply(lambda x: x.set_index('city')['zipped'].to_dict()).to_dict()
print (d)

{'AB': {'city3': (0.20000000000000001, 0.40000000000000002), 
        'city1': (0.0, 0.0), 
        'city2': (0.10000000000000001, 0.20000000000000001)}, 
'BC': {'city4': (0.29999999999999999, 0.59999999999999998), 
       'city5': (0.40000000000000002, 0.80000000000000004)}}
Pandas data frame from nested dictionary in a single column

Pandas data frame from nested dictionary in a single column


By : Alex Sah
Date : March 29 2020, 07:55 AM
seems to work fine When I look into your data I can see that the complication came from groups, so I decided to isolate it and work separately on it :
I decided to create a single data frame for every single group:
code :
data_df = {}
for category in data.get('Groups'):
    #print(category)
    data_df[category.get('Name')] = pd.DataFrame.from_records(category.get('Items'))
data_df['Faculty']
Eligible    IsOtherItem Name    NonVoters   RelativeTurnout Turnout Voters
0   7249    False   Faculty of Business, Law and Social Sciences    5880    4.779694    18.885363   1369
1   6226    False   Faculty of Arts, Design and Media   5187    3.627540    16.688082   1039
2   6156    False   Faculty of Computing, Engineering and the Buil...   5482    2.353188    10.948668   674
3   8943    False   Faculty of Health, Education and Life Sciences  7958    3.439006    11.014201   985
4   71  True    Other   56  0.052371    21.126761   15
Eligible    IsOtherItem Name    NonVoters   RelativeTurnout Turnout Voters
0   13246   False   18 - 21 10657   9.039173    19.545523   2589
1   6785    False   22 - 25 5939    2.953704    12.468681   846
2   3133    False   26 - 30 2862    0.946163    8.649856    271
3   5392    False   Over 30 5024    1.284826    6.824926    368
del data['Groups']
Convert deeply nested Dictionary into pandas data-frame?

Convert deeply nested Dictionary into pandas data-frame?


By : user2887372
Date : March 29 2020, 07:55 AM
will be helpful for those in need A nested dictionary has to be converted to pandas df: , You can check with from_dict
code :
pd.DataFrame.from_dict(d,'index').rename_axis('city').reset_index()
Out[119]: 
         city Column_20 Column_2  Column_1
0      Dallas      9876    23456       876
1     NewYork       500      695  12146144
2  Washington       734      864      5648
Pandas - create data frame from nested key values and nested list in the dictionary

Pandas - create data frame from nested key values and nested list in the dictionary


By : user3073504
Date : March 29 2020, 07:55 AM
I hope this helps . Update:
With the release of pandas 0.25 and the addition of explode this is now a lot easier:
code :
frame = pd.DataFrame(test).T
frame = frame.explode('names').set_index(
    ['company_id', 'names'],
    append=True).explode(
    'education').reset_index(
    ['company_id', 'names']
)
import pandas as pd

test = {
    'abc': {
        'company_id': '123c',
        'names': ['Oscar', 'John Smith', 'Smith, John'],
        'education': ['MS', 'BS']
    },
    'DEF': {
        'company_id': '124b',
        'names': ['Matt B.'],
        'education': ['']
    }
}

frame = pd.DataFrame(test).T

names = frame.names.apply(pd.Series)
frame = frame.merge(
    names, left_index=True, right_index=True).drop('names', axis=1)
frame = frame.reset_index().melt(
    id_vars=['index', 'company_id', 'education'],
    value_name='names').drop('variable', axis=1).dropna()

education = frame.education.apply(pd.Series)
frame = frame.merge(
    education, left_index=True, right_index=True).drop('education', axis=1)
frame = frame.melt(
    id_vars=['index', 'company_id', 'names'],
    value_name='education').drop(
    'variable', axis=1).dropna().sort_values(by=['company_id', 'names'])

frame.columns = ['set_name', 'company_id', 'names', 'education']

print(frame)
  set_name company_id        names education
2      abc       123c   John Smith        MS
6      abc       123c   John Smith        BS
0      abc       123c        Oscar        MS
4      abc       123c        Oscar        BS
3      abc       123c  Smith, John        MS
7      abc       123c  Smith, John        BS
1      DEF       124b      Matt B.          
4 dimensional nested dictionary to pandas data frame

4 dimensional nested dictionary to pandas data frame


By : Tao
Date : March 29 2020, 07:55 AM
To fix the issue you can do I don't really understand how json_normalize() fails this so hard, I have similar data with twice the nesting depth and json_normalize() still manages to give me a much better result.
I wrote this recursive function to replace the lists in your example with dictionaries:
Related Posts Related Posts :
  • Read data with NAs into python and calculate mean row-wise
  • How to print telnet response line by line?
  • Pylint: Avoid checking INSIDE DOCSTRINGS (global directive / rcfile)
  • Sending MIDI messages using Python (on Ubuntu)
  • Generate Dictionary in Python at run time
  • code is cluttered by try-except in Python
  • Python class inheritance - spooky action
  • Why is a Python multiprocessing daemon process not printing to standard output?
  • How to feed numeric data into a classifier?
  • How to unambiguously identify a Model as a lowercase string in Django
  • How to get only one specific line from subprocess output
  • Python network communication with encryption and password protection
  • with urllib urlopen read function but get none
  • django could not find database in ubuntu
  • How to replace a character in a string with a non ascii character in python?
  • Self learning data evaluation in Python
  • Django: UnicodeDecodeError while trying to read template 500.html
  • how can you Import an os variable into PYTHON and have it update?
  • Browse Folders to open a file in python
  • sql select group by a having count(1) > 1 equivalent in python pandas?
  • Why Insert command when button clicked OpenERP
  • Which is more efficient in Python: list.index() or dict.get()
  • Xor bits in python
  • A simple python confusion about format string
  • Finding index of a list containing an item, also in a list
  • How to test Domain is it 'www' domain or subdomain or name server domain or mail domain in python?
  • Keydown event for Pygame
  • Lazy loading csv with pandas
  • Use webdriver,python,beautifulsoup to retrieve dynamic website
  • Scrapy is Visiting same Url despite dont_filter=False
  • How to add support for SNI in python 2.7
  • Pandas: np.where with multiple conditions on dataframes
  • How to get meaningful words by splitting a continuous string?
  • TypeError: 'numpy.ndarray' object is not callable while extracting index and elements are stored in different array in p
  • Python: comparing list to a string
  • Is there any way to write '\r' into a file on linux with python?
  • matplotlib retrieve color from plt.plot command
  • Scrapy JSON export issues
  • beautifulsoup to retrieve the date
  • Django Rest Framework 3.0: Saving Nested, Many-To-One Relationship
  • Lost connection to MySQL server during Python connection
  • uploadede file path django
  • How to reduce a data with the longest string under pandas framework?
  • Single line for-loop to build a dictionary?
  • How to read in lines until a certain line?
  • Beautifulsoup to retrieve the href list
  • Python - Vincenty's inverse formula not converging (Finding distance between points on Earth)
  • Saving django model instance into another model
  • Same URL request fails with python->urllib but not with curl
  • Python dictionary: set value as the key string
  • With setuptools, when does namespace packages __init__.py files disappears?
  • Paraview: NameError: name 'inputs' is not defined
  • Too many values to unpack during template rendering
  • A different type of iterative function
  • Test if value exists in several lists
  • Go and Python HMAC libraries give different results
  • Python Nested Loop Working Weirdly
  • Why is this not assigning the correct value?
  • 'numpy.ndarray' object is not callable
  • How to deal with globals in modules?
  • shadow
    Privacy Policy - Terms - Contact Us © ourworld-yourmove.org