speed up Maxima computations by using more of maxima_calculus.eval()

[dimpase]

Empirically, maxima_calculus.eval() is 5-7 times faster than maxima.eval(), cf. e.g.

sage: timeit("L = [(i/100.0, maxima_calculus.eval('jacobi_sn (%s/100.0,2.0)'%i)) for i in range(-300,300)]")
5 loops, best of 3: 110 ms per loop
sage: timeit("L = [(i/100.0, maxima.eval('jacobi_sn (%s/100.0,2.0)'%i)) for i in range(-300,300)]")
5 loops, best of 3: 556 ms per loop

For some reason, there are dozens of maxima.eval calls in sage/[src,doc] - and none of maxima_calculus.eval. Just going and changing these will be an improvement (in particular, on Cygwin, where pexpect is very flaky, cf #22191).

maxima.jacobi_sn() is also much slower than
maxima_calculus.jacobi_sn()

This should be a part of #17753.

CC: @nbruin @rwst @kcrisman

Component: interfaces

Issue created by migration from https://trac.sagemath.org/ticket/30071

[dimpase]

comment:2

I really want to understand why maxima_calculus.foo() is as slow as maxima.foo() - this look more like a bug than anything else.

I can't seem to find the code responsible for this, though.

[mkoeppe]

comment:3

Testing on macOS, I get

%timeit maxima_calculus.jacobi_sn(0.1,2.0)
1000 loops, best of 5: 484 µs per loop
sage: %timeit maxima.jacobi_sn(0.1,2.0)
100 loops, best of 5: 1.98 ms per loop

[dimpase]

comment:4

Replying to @mkoeppe:

Testing on macOS, I get

%timeit maxima_calculus.jacobi_sn(0.1,2.0)
1000 loops, best of 5: 484 µs per loop
sage: %timeit maxima.jacobi_sn(0.1,2.0)
100 loops, best of 5: 1.98 ms per loop

right, I probably did something wrong here. Indeed, on Linux I get similar timings.

[dimpase]

comment:5

still there is something funny going on

sage: timeit("L = [(i/100.0, maxima_calculus.jacobi_sn(i/100.0,2.0)) for i in range(-300,300)]")
5 loops, best of 3: 612 ms per loop
sage: timeit("L = [(i/100.0, maxima_calculus.eval('jacobi_sn (%s/100.0,2.0)'%i)) for i in range(-300,300)]")
5 loops, best of 3: 89.3 ms per loop

If I remove _calculus above, the first line errors out with TIMEOUT: Timeout exceeded..

[nbruin]

comment:6

Replying to @dimpase:

still there is something funny going on

sage: timeit("L = [(i/100.0, maxima_calculus.jacobi_sn(i/100.0,2.0)) for i in range(-300,300)]")
5 loops, best of 3: 612 ms per loop
sage: timeit("L = [(i/100.0, maxima_calculus.eval('jacobi_sn (%s/100.0,2.0)'%i)) for i in range(-300,300)]")
5 loops, best of 3: 89.3 ms per loop

I think this example is showing that maxima's string parser is faster than our to/from binary data conversion for ecllib. I can believe that: there's still a lot of work and memory allocation to do to translate python objects to lisp objects and back. I can imagine that parsing such a small string is easier (the construction of the lisp float objects etc. needs to happen in both cases, so it's really just parsing against the python object constructions). You could try and profile the code to see if anything bad is happening.

For the results via the expect interface: since the python-to-maxima translation now needs to happen through parsing anyway, it's bound to be slower: there's no upside compared to the "eval" at all. The fact that it actually times out: I'd suspect there's an io buffer that gets flooded or an expect interface that gets out of sync due to the high data flow. Note that the timeout happens in the expect interface.

[dimpase]

comment:7

In src/sage/calculus/calculus.py one reads

The symbolic calculus package uses its own copy of Maxima for
simplification, etc., which is separate from the default
system-wide version::

    sage: maxima.eval('[x,y]: [1,2]')
    '[1,2]'
    sage: maxima.eval('expand((x+y)^3)')
    '27'

If the copy of maxima used by the symbolic calculus package were
the same as the default one, then the following would return 27,
which would be very confusing indeed!

::

    sage: x, y = var('x,y')
    sage: expand((x+y)^3)
    x^3 + 3*x^2*y + 3*x*y^2 + y^3

Is this the only issue (variable binding in SR done via maxima, not maxima_lib) that
prevents Sage from fully switching to maxima_lib backend?

[mkoeppe]

comment:8

For some reason, there are dozens of maxima.eval calls in sage/[src,doc]

Note that at least in some modules, actually lazy_import('sage.interfaces.maxima_lib','maxima') is done.

[dimpase]

comment:9

Replying to @mkoeppe:

For some reason, there are dozens of maxima.eval calls in sage/[src,doc]

Note that at least in some modules, actually lazy_import('sage.interfaces.maxima_lib','maxima') is done.

in just one, in src/sage/calculus/calculus.py (which I quote in comment:7)

[mkoeppe]

comment:10

Also the code of sage/symbolic/expression.pyx (but not doctests) use from sage.calculus.calculus import maxima. Likewise other modules in sage/symbolic

Let's make a list... Outside of doctests, I see:

combinat/combinat.py: from sage.interfaces.all import maxima

Anything else?

[dimpase]

comment:11

in src/doc - the example in the ticket description is from there

[mkoeppe]

comment:13

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.