fj! functions?

[ChrisRackauckas]

It would be nice to have simultaneous f and Jacobian evaluations from this to match the JuliaNLSolvers API

[pkofod]

Would make our job a little bit easier :) Though it's obviously not a deal breaker if it isn't in when we change.

[dextorious]

Since, apart from some more tests and refactoring, I'm largely done with complex-valued Jacobians, I can now start looking at various convenience wrappers. For clarity and since I'm not too familiar with the JuliaNLSolvers API, could you give me a simple example for the feature you're looking for here?

[ChrisRackauckas]

https://github.com/JuliaNLSolvers/NLsolve.jl/blob/master/src/autodiff.jl#L12

fj! is a function which computes the Jacobian simultaneously with f, since the evaluation of f is required to get the Jacobian anyways. fg! is the same for the gradient. Of course, they could do one after the other, but this can be slightly more efficient and maybe would just need a tweak to the API to have this done by default?

[dextorious]

The equivalent of fj! is now available (f788176). The syntax is (ignoring the usual optional arguments): DiffEqDiffTools.finite_difference_jacobian!(J, df, f, x, Val{:forward}) (as opposed to DiffEqDiffTools.finite_difference_jacobian!(J, f, x, Val{:forward}) if you don't need the derivative), the result is stored in the variable df.

[ChrisRackauckas]

Finished as a part of #17

[pkofod]

The syntax is (ignoring the usual optional arguments): DiffEqDiffTools.finite_difference_jacobian!(J, df, f, x, Val{:forward}) (as opposed to DiffEqDiffTools.finite_difference_jacobian!(J, f, x, Val{:forward}) if you don't need the derivative), the result is stored in the variable df.

Sorry, but I don't really understand the syntax here. What is df again? What fj! is supposed to do is to take in an x and two cache arrays F, F(x), and J, J(x) and update the "residuals" and Jacobian is-place. Is df equal to F(x)?

[dextorious]

I'm not sure what you mean by F(x) here. In DiffEqDiffTools df denotes the derivative of the function f at the points given in x. So if you pass in two caches to finite_difference_jacobian! - J and df, it will mutate both, yielding both the Jacobian and the derivative.

[pkofod]

Then I think OP was misunderstood. We have F(x)=0 that we want to solve. We use the Jacobian J(x) as part of this. To evaluate F(x) and J(x) we have

function fj!(x, F, J)
... update F inplace store in F
... update J inplace store in J
end

Is this what was implemented in #117?

[dextorious]

So, if I understand you correctly, rather than evaluating the Jacobian and the derivatives at the same time, you want to evaluate the Jacobian and the function values f(x) at the same time? Something like

function fj!(f, x, fx, J, fdtype, funtype)
  @. fx = f(x)
  DiffEqDiffTools.finite_difference_jacobian!(J, f, x, fdtype, funtype, Val{:Default}, fx)
end

could be easily added as a helper function. Since finite_difference_jacobian! receives the values fx, it will not perform extraneous function evaluations in this case.

[ChrisRackauckas]

Yeah, that's my bad that this slipped by the review. It should be an inplace update into fx of just the function evaluation, not the function's derivative. Thanks for catching this @pkofod

[dextorious]

I'm not sure it makes sense to make this a part of the general finite_difference_derivative! / finite_difference_jacobian! implementations, because only in the case of forward differences do they even need the values of f(x) directly. Central and complex-step difference formulas require function values at different points. That might be why I misunderstood the feature request originally, since there's much more overlap with the derivative computations.

For this, I think adding a helper function along the lines of what I posted previously, only with appropriate parameter constraints and default values might be the way to go.

[ChrisRackauckas]

Well every numerical differentiation method needs to calculate f(x), the idea here is to save out that call so it doesn't need to be repeated in the next step of the rootfinder or optimizer. Though I'm not sure how much this really effects the total number of f calls though.

[dextorious]

Well every numerical differentiation method needs to calculate f(x)

Not really, no. Forward differences need f(x) and f(x+epsilon). Central differences need f(x+epsilon) and f(x-epsilon). Complex-step differences require f(x+im*epsilon).

[ChrisRackauckas]

Good point. I was just thinking forward differences.

[pkofod]

Anyway, it doesn't really matter, I can just evaluate f and pass fx. I'll make the pr for nlsolve