Write to csv: columns are shifted when item in row is empty (Python)

» python » Write to csv: columns are shifted when item in row is empty (Python)

By : Angelina Tornetto

Date : November 24 2020, 05:47 AM

will help you I think you don't understand how CSV module works in Python. Let's say that kres.csv file has this data in it:

code :

Share :

python - how to write empty tree node as empty string to xml file

By : Shubham Agrawal

Date : March 29 2020, 07:55 AM

seems to work fine I want to remove elements of a certain tag value and then write out the .xml file WITHOUT any tags for those deleted elements; is my only option to create a new tree?

code :

import lxml.etree as et

xml  = et.parse("test.xml")

for node in xml.xpath("//neighbor"):
    node.getparent().remove(node)


xml.write("out.xml",encoding="utf-8",xml_declaration=True)

from xml.etree import ElementTree as et

xml  = et.parse("test.xml")


for parent in xml.getroot().findall(".//neighbor/.."):
      for child in parent.findall("./neighbor"):
          parent.remove(child)


xml.write("out.xml",encoding="utf-8",xml_declaration=True)

<?xml version='1.0' encoding='utf-8'?>
<data>
    <country name="Liechtenstein">
        <rank>1</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        </country>
    <country name="Singapore">
        <rank>4</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        </country>
    <country name="Panama">
        <rank>68</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        </country>
</data>

x = """<?xml version="1.0"?>
<data>
    <country name="Liechtenstein">
        <rank>1</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    </country>
    <country name="Singapore">
        <rank>4</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
           <neighbor name="Costa Rica" direction="W" make="foo" build="bar" job="blah"/>
        <neighbor name="Malaysia" direction="N"/>
    </country>
    <country name="Panama">
        <rank>68</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        <neighbor name="Costa Rica" direction="W" make="foo" build="bar" job="blah"/>
        <neighbor name="Colombia" direction="E"/>
    </country>
</data>"""

import lxml.etree as et

xml = et.fromstring(x)

for node in xml.xpath("//neighbor[not(@make) and not(@job) and not(@make)]"):
    node.getparent().remove(node)
print(et.tostring(xml))

 <data>
    <country name="Liechtenstein">
        <rank>1</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        </country>
    <country name="Singapore">
        <rank>4</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        <neighbor name="Costa Rica" direction="W" make="foo" build="bar" job="blah"/>
        </country>
    <country name="Panama">
        <rank>68</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        <neighbor name="Costa Rica" direction="W" make="foo" build="bar" job="blah"/>
        </country>
</data>

from xml.etree import ElementTree as et

xml = et.parse("test.xml").getroot()

atts = {"build", "job", "make"}

for parent in xml.findall(".//neighbor/.."):
    for child in parent.findall(".//neighbor")[:]:
        if not atts.issubset(child.attrib):
            parent.remove(child)

from xml.etree import ElementTree as et

xml = et.parse("test.xml")

for parent in xml.getroot().iter("*"):
    parent[:] = (child for child in parent if child.tag != "neighbor")

In [30]: !cat /home/padraic/untitled6/test.xml
<?xml version="1.0"?>
<data>
    <country name="Liechtenstein">#
      <neighbor name="Austria" direction="E"/>
        <rank>1</rank>
        <neighbor name="Austria" direction="E"/>
        <year>2008</year>
      <neighbor name="Austria" direction="E"/>
        <gdppc>141100</gdppc>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    </country>
    <country name="Singapore">
        <rank>4</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        <neighbor name="Malaysia" direction="N"/>
    </country>
    <country name="Panama">
        <rank>68</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        <neighbor name="Costa Rica" direction="W"/>
        <neighbor name="Colombia" direction="E"/>
    </country>
</data>
In [31]: paste
def test():
    import lxml.etree as et
    xml = et.parse("/home/padraic/untitled6/test.xml")
    for node in xml.xpath("//neighbor"):
        node.getparent().remove(node)
    a = et.tostring(xml)
    from xml.etree import ElementTree as et
    xml = et.parse("/home/padraic/untitled6/test.xml")
    for parent in xml.getroot().iter("*"):
        parent[:] = (child for child in parent if child.tag != "neighbor")
    b = et.tostring(xml.getroot())
    assert  a == b

## -- End pasted text --

In [32]: test()

Pandas Shift columns but add new rows where non-shifted columns are null

By : ZenSurfer

Date : March 29 2020, 07:55 AM

I wish did fix the issue. Say I have a dataframe: , You can reindex first, and then shift only one of the columns:

code :

df = df.reindex(range(8))
df["y"] = df.y.shift(3)

     x    y
0  1.0  NaN
1  2.0  NaN
2  3.0  NaN
3  4.0  5.0
4  5.0  4.0
5  NaN  3.0
6  NaN  2.0
7  NaN  1.0

How to calculate shifted columns over Groups in Python Pandas

By : user32994

Date : March 29 2020, 07:55 AM

I think the issue was by ths following , I have the following pandas dataframe: , Use groupby and shift, then join it back:

code :

df.join(df.groupby('Circuit-ID').shift().add_suffix('-1'))

   Circuit-ID          DATETIME  LATE?        DATETIME-1  LATE?-1
0       78899  07/06/2018 15:30      1               NaN      NaN
1       78899  08/06/2018 17:30      0  07/06/2018 15:30      1.0
2       78899  09/06/2018 20:30      1  08/06/2018 17:30      0.0
3       23544  12/07/2017 23:30      1               NaN      NaN
4       23544  13/07/2017 19:30      0  12/07/2017 23:30      1.0
5       23544  14/07/2017 20:30      1  13/07/2017 19:30      0.0

pd.concat([df, df.groupby('Circuit-ID').shift().add_suffix('-1')], axis=1)

   Circuit-ID          DATETIME  LATE?        DATETIME-1  LATE?-1
0       78899  07/06/2018 15:30      1               NaN      NaN
1       78899  08/06/2018 17:30      0  07/06/2018 15:30      1.0
2       78899  09/06/2018 20:30      1  08/06/2018 17:30      0.0
3       23544  12/07/2017 23:30      1               NaN      NaN
4       23544  13/07/2017 19:30      0  12/07/2017 23:30      1.0
5       23544  14/07/2017 20:30      1  13/07/2017 19:30      0.0

write on csv but getting empty file with no row and columns

By : user3433577

Date : March 29 2020, 07:55 AM

I hope this helps . It is a bit unclear from your example, but I think you are closing the CSV early.
When you do with open('keras_character_W.csv', 'w') as csvFile:, you start a context in which:

Write without quotes in empty columns in a CSV?

By : JhonFow1212

Date : March 29 2020, 07:55 AM

To fix the issue you can do It seems that you either have quotes everywhere (QUOTE_ALL) or no quotes (QUOTE_MINIMAL) (and other exotic options useless here).
I first posted a solution which wrote in a file buffer, then replaced the double quotes by nothing, but it was really a hack and could not manage strings containing quotes properly.