Repository navigation
Append Datafarmes #6806
Description
Activity
Hi @pi-curst , I would say that this is expected behavior. Append method gets IEnumerable rows as an argument. Internaly for each row in the provided sequence this method calls Append(IEnumerable row). So for each DataFrameRow object IEnumerable is called and it works based on column position in DataFrame.Columns collection (column index), column names are just ignored:
public class DataFrameRow : IEnumerable<object> { ... /// <summary> /// Returns an enumerator of the values in this row. /// </summary> public IEnumerator<object> GetEnumerator() { foreach (DataFrameColumn column in _dataFrame.Columns) { yield return column[_rowIndex]; } } ... }So actualy your code looks similar to this:
... dataFrame1.Append(new object[] { "f", 100, 6 }, true); dataFrame1.Append(new object[] { "g", 200, 7 }, true); dataFrame1.Append(new object[] { "c", 300, 3 }, true); ... var resultDataFrame = dataFrame1.Append(new object[] { "e", 500, 5 }, true);However, I checked the same behaviour in pandas dataframe and it works as you expected:
import pandas as pd d1 = {'ColumnA': ['a', 'b', 'c'], 'ColumnB': [1, 2, 3], 'ColumnC': [10, 20, 30]} df1 = pd.DataFrame(data=d1) d2 = {'ColumnA': ['d', 'e', 'f'], 'ColumnC': [40, 50, 60], 'ColumnB': [4, 5, 6]} df2 = pd.DataFrame(data=d2) df3=df1.append(df2) print(df3.to_string())Result:
ColumnA ColumnB ColumnC 0 a 1 10 1 b 2 20 2 c 3 30 0 d 4 40 1 e 5 50 2 f 6 60@JakeRadMSFT, @luisquintanilla what do you think? Should it be changed inline with Pandas behavior?
I am ready to work on a fix if requiredReacted by Luis QuintanillaHi @asmirnov82,
Thanks for the replies and investigations. I can see why it works the way it does now, but I too would expect that if the column names are the same, appending two DataFrames would require that their column names match up and by extension would append the values based on the column names.
@JakeRadMSFT thoughts?
- ghost locked as resolved and limited conversation to collaborators
on Oct 2, 2023
System Information (please complete the following information):
Describe the bug
Hi, Appending Dataframes (DataFrame.Append Method) doesn't seem to append based on column names, but appends based on the position. Is this normal?
To Reproduce
`var data = new DataFrame();
var col1 = new StringDataFrameColumn("ColumnA", new string[] { "a", "b", "c", "d", "e" });
var col2 = new Int32DataFrameColumn("ColumnB", new int[] { 1, 2, 3, 4, 5 });
var col3 = new Int32DataFrameColumn("ColumnC", new int[] { 10, 20, 30, 40, 50 });
var col4 = new StringDataFrameColumn("ColumnA", new string[] { "f", "g", "c", "d", "e" });
var col5 = new Int32DataFrameColumn("ColumnB", new int[] { 6, 7, 3, 4, 5 });
var col6 = new Int32DataFrameColumn("ColumnC", new int[] { 100, 200, 300, 400, 500 });
var dataFrame1 = new DataFrame(col1, col2, col3);
var dataFrame2 = new DataFrame(col4, col6, col5);
var dataFrames = new List { dataFrame1, dataFrame2 };
var resultDataFrame = dataFrame1.Append(dataFrame2.Rows);`
Exhibited behavior
Expected behavior