Windows Conda package

[mguijarr]

Hi,

There is no Conda package for Windows on tango-controls Conda channel.

Would be great to have one.

Thanks in advance !

[andygotz]

Which version are you looking for? I thought there was a Conda package for 9.2.5.
For 9.3.3 (the latest version) I let the PyTango experts answer you. AFAIK this is either aavailable (somewhere) or only needs a small effort.

[ajoubertza]

There are Conda packages for 9.2.5 and 9.3.0, but these are only for Linux. I've no idea if it would be easy to make for Windows. We don't have cppTango Conda packages for Windows either, which may be required...
Agree that it would be useful though. There's still a long-open issue (#176) for making Windows builds of PyTango (outside of Conda).

https://anaconda.org/tango-controls/pytango/files

[andygotz]

@NexeyaSGara and @izadworny can you two comment on this as PyTango Windows experts?

[tiagocoutinho]

Hi,
So far there as never been a PyTango for Windows conda package.
I have made a preliminary PyTango 9.3.1dev0 for windows available on pypi (it is missing python 2.7 64bits because of VS9 availability). Don't know if it helps.

I also agree it could be useful. You are not the first to ask :-). I can have a try and how it goes.
I keep you informed.

[ajoubertza]

@tiagocoutinho What is in the exe files you uploaded to pypi - are they like wheels inside an exe? How exactly did you make them? Could they be installed in a Conda environment on Windows?

[tiagocoutinho]

What is in the exe files you uploaded to pypi - are they like wheels inside an exe?

There is no wheel. After installation you end up with a tango directory inside site-packages which contains exactly the same as a linux installation. The only difference is that instead of having a _tango.so you have _tango.pxd

Could they be installed in a Conda environment on Windows?

I haven't tried. My guess is, since it is not a wheel, pip does not support it.

I can have a look and see how hard it is to make a wheel. If I succeed there might be a better chance of being able to make a conda package out of it.

[tiagocoutinho]

I have uploaded wheels to Pypi.

On windows, pip install --pre pytango should now install PyTango correctly (inside or outside a conda environment).

Conda packages will require a little more love :-)

When #277 is accepted we can think of an official 9.3.1

[sdebionne]

I made a pytango package for Windows some time ago (available on our esrf-bcu channel). It's basically a repackaging of the binaries in the.msi files generated by the pipeline but that worked fine so far. The recipe is as simple as this:

msiexec /a pytango-9.2.3.win-amd64-py2.7.msi /qb TARGETDIR=%PREFIX%
del %PREFIX%\pytango-9.2.3.win-amd64-py2.7.msi
rmdir /Q /S %PREFIX%\Lib\site-packages\pytango-9.2.3-py2.7.egg-info

Recompiling might not be actually necessary to build the conda package.

[tiagocoutinho]

Thanks for the hint @sdebionne. I think since we have a wheel it should be even simpler.

[vxgmichel]

@sdebionne @tiagocoutinho
Hi guys! Aren't the pytango wheel binaries statically linked? A windows conda package for pytango should link dynamically to the other installed packages (tango, zeromq, omniorb, etc.). Avoiding the building of pytango within the conda build environment doesn't seem very doable to me :)

[sdebionne]

A windows conda package for pytango should link dynamically to the other installed packages (tango, zeromq, omniorb, etc.).

@vxgmichel Hi Vincent! Is this a requirement for "symmetry" with the linux package or a guideline for Conda packaging? AFAIK, the Windows C++ tango package is also statically linked so this would need to be modified as well.

Does anyone know the rationale for statically linking on Windows in the first place?

[tiagocoutinho]

To install pytango outside a conda environment, pytango needs to be statically linked because:

• boost provides no dll for different visual studio compilers, 32/64bits and python versions
• tango c++ is very picky about the omniorb and zeromq versions so it provides its own versions of DLLs for omniorb and zeromq
• tango c++ installation uses arbitrary installation directory so the C++ DLLs are not found by pytango
• we would have to be sure pytango links with the precise architecture and compiler version of the tango dlls (this was in the past a great headache which gave origin to many issues being reported (back when Pytango was in SourceForge svn)
• we would require users that install pytango to install tango and boost since this cannot be automatized

Having conda makes life a little easier to create dynamically linked PyTango. But it would require a lot of effort to prepare a conda build environment which would create a dynamically linked version of PyTango. Would it be worth the effort? What downside do you see in providing a statically linked PyTango on conda (besides +RAM and +disk space) @vxgmichel?

[vxgmichel]

Hi @tiagocoutinho :)

But it would require a lot of effort to prepare a conda build environment which would create a dynamically linked version of PyTango. Would it be worth the effort?

Is it really that bad? I had the intuition that conda-build would help with most of the issues you described. There are probably windows recipes for boost python projects on conda-forge, maybe getting inspiration from them would help?

What downside do you see in providing a statically linked PyTango on conda (besides +RAM and +disk space) @vxgmichel?

I can see of few points:

• It's confusing to have a different linking policy on linux and windows (for conda packages)
• It's nice to be able to bump one of the dependency (say after a bug fix) without rebuilding all the packages
• It's nice to be able to fiddle around with the dependencies when investigating a bug
• It's nice to have a windows package for tango c++ anyway
• It should be easier to automate conda builds than wheels or windows installer, so that could allow for development releases for windows.

But since I'm not the person doing all the work, it's easy for me to say :)
Or maybe call the conda package pytango-static and leave the way open to dynamic linking, waiting for someone who's willing to invest those resources.

[tiagocoutinho]

Hi @vxgmichel ,

This is the typical thing which starts as a simple task and ends up eating a lot of your time :-).
Conda & cmake it might make things a little easier.

I agree with all your points.
I also agree that if someone has availability and interest it would be a nice thing to have.
For me, the one condition would be that both conda and non-conda packages on windows come from the same kind of binary. Otherwise we would be inconsistent within the same platform.

In the meantime, ok for me to name the conda package pytango-static

[andygotz]

+1 for naming the static version pytango-static

@tiagocoutinho thanks for working on this!

[sdebionne]

Or maybe call the conda package pytango-static

That might be a good use case for Conda Features?

[tiagocoutinho]

That might be a good use case for Conda Features?

Indeed. So we should:

• create a pytango package, with a feature pytango-static
• A metapackage called pytango-static that tracks the pytango-static feature

Does everyone agree?

The windows build seems to be working fine so a new package could come out soon.

[sdebionne]

@tiagocoutinho I have never used this Features feature before but AFAIU the feature and metapackage could be just static and be reused for the tango package is needed, no?

[sdebionne]

Thanks for the Windows package for v9.3.1dev1. Is there any chance to get one for python 3.6 and 3.7 as well? I could not find the source of the recipe in the repo, is it in a external repo? Is there any plan to release pre-compiled wheels on PyPi as well?

[ajoubertza]

Thanks for the Windows package for v9.3.1dev1. Is there any chance to get one for python 3.6 and 3.7 as well?

The pre-release on PyPI already includes wheels for Python 3.6 and 3.7:
https://pypi.org/project/pytango/9.3.1.dev1/#files - is that not what you are looking for?

I could not find the source of the recipe in the repo, is it in a external repo?

AppVeyor + cmake is used to generate the wheels. https://github.com/tango-controls/pytango/blob/develop/appveyor.yml. The repo's readme has a badge linking to the latest AppVeyor builds: https://ci.appveyor.com/project/tiagocoutinho/pytango

Or are you talking about Conda?

Is there any plan to release pre-compiled wheels on PyPi as well?

As mentioned above, there are wheels on PyPi. Are these useful? Future releases of pytango should include wheels like these.

[sdebionne]

Sorry, I was talking about conda. The tango-controls channel has only support for win/py27. I did not check Tiago's channel before and there is indeed everything I need there, awesome!

[laurent-claustre]

Hi,
Is there some progress to get a conda package for windows and python 3?
Thanks for your help.

[ajoubertza]

@laurent-claustre and @sdebionne As of v9.3.1, there are conda packages for Windows (64-bit). We'd appreciated some feedback - do they work for you?

They are built from the wheels which used cppTango 9.3.3, so, if I understand correctly, having a different version of cppTango installed in conda won't make a difference.

[sdebionne]

Thank you @tiagocoutinho for the conda package for Windows. I did a quick test with Lima and it works for me. In the near future, I am going to remove my hand-made pyTango package for Windows from the esrf-bcu channel to avoid confusion.

You are right, since cppTango is currently linked statically on Windows, a cppTango package can be installed along pyTango without interfering.

[ajoubertza]

Thanks for testing, @sdebionne.

You are right, since cppTango is currently linked statically on Windows, a cppTango package can be installed along pyTango without interfering.

This might also lead to confusion if someone expects PyTango to use the separately installed cppTango package. Anyway, glad it works.

[tiagocoutinho]

Anyway, glad it works.

Me too :-) Thanks for testing

[laurent-claustre]

I did test with Lima simulator and basler cameras, it works fine.
Thanks a lot for conda users.