ENH: Added autofilter parameter to Excel output for both xlsxwriter and openpyxl engines

[Upabjojr]

Added autofilter parameter to Excel output for both xlsxwriter and openpyxl engines

• closes ENH: Add AutoFilter option to to_excel #15307
• tests added / passed
• Ensure all linting tests pass, see here for how to run them
• whatsnew entry

[Upabjojr]

Hi @jreback , I've seen you have followed the freeze_pane discussion for the Excel output some years ago. Could you provide some feedback on this PR?

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

[Upabjojr]

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

It would be nice to have this PR merged, it's a fairly simple addition that adds a useful feature to the excel exporter.

[jreback]

since this is only a feature of openpyxl, let's just do this via the engine_kwargs option. a test with docs would be sufficient. -1 on adding additional kwargs to the already long list.

[Upabjojr]

since this is only a feature of openpyxl, let's just do this via the engine_kwargs option. a test with docs would be sufficient.

Well, this PR adds a backend for both openpyxl and xlsxwriter. It's not limited to openpyxl only, and could potentially be extended...

engine_kwargs will not work because there is no parameter in both libraries to add autofilters. In fact, we have to use multiple operations in both xlsxwriter and openpyxl in order to add autofilters (they need to be added to every column independently).

-1 on adding additional kwargs to the already long list.

Consider also the simplicity of looking up the documentation with df.to_excel? in Jupyter/IPython. This feature is analogous to freeze_panes and perfectly fits together.

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

[Upabjojr]

I think this PR is fine the way it is. Don't agree with the previous review at all.

[jreback]

@rhshadrach thoughts here

[rhshadrach]

This feature is analogous to freeze_panes and perfectly fits together.

I find this reasoning compelling. But it is also compelling for the numerous other features of excel, all with slightly different APIs that might or might not be implemented in different engines. Going down this road seems to me like a maintenance nightmare. Instead, I find your reasoning above to make a compelling case to remove freeze_panes.

It is easy for the user to add an autofilter today:

df = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]})

with pd.ExcelWriter('test.xlsx', engine='openpyxl') as writer:
    df.to_excel(writer, sheet_name='test')
    writer.sheets['test'].auto_filter.ref = "B1:C3"
    

with pd.ExcelWriter('test2.xlsx', engine='xlsxwriter') as writer:
    df.to_excel(writer, sheet_name='test')
    writer.sheets['test'].autofilter(0, 1, 2, 2)

This is why I think we should implement #43088 - make all protected attributes prefixed with _ and make attributes such as sheets public. Numerous third party libraries have already documented using writer.sheets.

[jreback]

agree with @rhshadrach here, let' privatize things more.

closing this in light.

[Upabjojr]

It is easy for the user to add an autofilter today:

I wouldn't call those examples "easy".

[Upabjojr]

Instead, I find your reasoning above to make a compelling case to remove freeze_panes.

I just know that an easier API for excel output formatting would make life easier to lots of pandas users.

[rhshadrach]

I wouldn't call those examples "easy".

Can you expand on this? What do you find difficult?

[Upabjojr]

Can you expand on this? What do you find difficult?

• XlsxWriter and OpenPyXL have two different APIs requiring the range to be specified for adding autofilter. In Microsoft Excel and LibreOffice that's just a single operation (no need to specify the range). That is a very simple operation vs a complicated one.
• End-users of Pandas may not be familiar with programming, let alone backends such as XlsxWriter or OpenPyXL. At this point it's way faster to open the file in Microsoft Excel and add the autofilters there rather than by using python. I see this operation all too often.
• It's easy to make mistakes when specifying the range of the autofilter and potentially leaving some rows out.

I understand you are not happy with having too many parameters in the .to_excel( ) method and the consequent maintainence problems, but I believe this parameter would ease the life to many people.

I would even go as far as suggesting that the autofilter should be on by default when producing excel outputs.

[rhshadrach]

Thanks for the response @Upabjojr. I don't believe autofilter should be considered in isolation, so I've opened #45572. Please feel free to share any thoughts there.