read_fwf raises BytesWarning when running with -bb

[fsouza]

Code Sample, a copy-pastable example if possible

from io import StringIO

import pandas

input = StringIO("some\tcontent\n")
pandas.read_fwf(input)

Problem description

If you run the above code with -bb, it crashes with BytesWarning:

％ python -bb repro.py
Traceback (most recent call last):
  File "repro.py", line 6, in <module>
    pandas.read_fwf(input)
  File "/Users/fsouza/.venvs/pandas-debug/lib/python3.7/site-packages/pandas/io/parsers.py", line 791, in read_fwf
    return _read(filepath_or_buffer, kwds)
  File "/Users/fsouza/.venvs/pandas-debug/lib/python3.7/site-packages/pandas/io/parsers.py", line 457, in _read
    parser = TextFileReader(fp_or_buf, **kwds)
  File "/Users/fsouza/.venvs/pandas-debug/lib/python3.7/site-packages/pandas/io/parsers.py", line 895, in __init__
    self._make_engine(self.engine)
  File "/Users/fsouza/.venvs/pandas-debug/lib/python3.7/site-packages/pandas/io/parsers.py", line 1147, in _make_engine
    self._engine = klass(self.f, **self.options)
  File "/Users/fsouza/.venvs/pandas-debug/lib/python3.7/site-packages/pandas/io/parsers.py", line 3709, in __init__
    PythonParser.__init__(self, f, **kwds)
  File "/Users/fsouza/.venvs/pandas-debug/lib/python3.7/site-packages/pandas/io/parsers.py", line 2348, in __init__
    r"[^-^0-9^{decimal}]+".format(decimal=self.decimal)
BytesWarning: str() on a bytes instance

It doesn't crash without that flag, but we do end-up with the incorrect regexp:

>>> decimal = b'.'
>>> r"[^-^0-9^{decimal}]+".format(decimal=decimal)
[^-^0-9^b'.']+

Notice how you end-up with an unexpected regexp.

Expected Output

No crash.

Output of pd.show_versions()

>>> pandas.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.5.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 18.7.0
machine          : x86_64
processor        : i386
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 0.25.3
numpy            : 1.17.4
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 19.3.1
setuptools       : 41.6.0
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None

[lwgray]

Maybe I’m missing something but shouldn’t you receive a warning or error when using -bb option (see below from DOCS )

``
1.1.3. Miscellaneous options
-b
Issue a warning when comparing bytes or bytearray with str or bytes with int. Issue an error when the option is given twice (-bb).

Changed in version 3.5: Affects comparisons of bytes with int.
``

[fsouza]

@lwgray -b is warning, -bb is error. But the reason it warms/errors is because the code doesn't do what's intended when running on python 3. I can send a PR that coalesces decimal to str before using it in format, if that's ok/acceptable.

[lwgray]

@fsouza I'm not a maintainer.