native Apple Silicon version

[rdzman]

I'd like to compile a version with a native Apple Silicon version of MATLAB.

I have successfully built Ipopt (using coinbrew) and an Octave MEX interface, but I am afraid I don't know how to get mexIPOPT to recognize the version of Ipopt I built from source when building for MATLAB.

Can you give me some pointers here? Or do you have a precompiled version for a native Apple Silicon version of MATLAB?

Thanks.

[ebertolazzi]

Sorry, I do not have an ARM, however on sept. I will ask a friend to give me an ARM to try a compilation and see where the problems are.

[rdzman]

I'm happy to give it a try on my new M2 MacBook Pro if you are able to help walk me through it.

[rdzman]

Any update on this? I'm very eager to get this working. Let me know if I can help test or anything.

[rdzman]

I got it working!

1. Install Ipopt and its dependencies via Homebrew.

   brew install ipopt
   

2. Clone this repo.

   git clone https://github.com/ebertolazzi/mexIPOPT.git
   

3. Apply this patch (homebrew_ipopt.patch) to toolbox/CompileIpoptMexLib.m.

   cd mexIPOPT/toolbox
   patch -u -b CompileIpoptMexLib.m -i homebrew_ipopt.patch
   

4. Run CompileIpoptMexLib at the Matlab prompt from the mexIPOPT/toolbox directory.

5. Copy toolbox/bin/osx/ipopt_osx.mexmaca64 to ipopt.mexmaca64 somewhere in your Matlab path (along with toolbox/lib/ipopt_auxdata.m).

6. Run an example ...

   >> cd examples
   >> examplehs071
   This is Ipopt version 3.14.13, running with linear solver MUMPS 5.6.2.
   
   Number of nonzeros in equality constraint Jacobian...:        4
   Number of nonzeros in inequality constraint Jacobian.:        4
   Number of nonzeros in Lagrangian Hessian.............:       10
   
   Total number of variables............................:        4
                        variables with only lower bounds:        0
                   variables with lower and upper bounds:        4
                        variables with only upper bounds:        0
   Total number of equality constraints.................:        1
   Total number of inequality constraints...............:        1
           inequality constraints with only lower bounds:        1
      inequality constraints with lower and upper bounds:        0
           inequality constraints with only upper bounds:        0
   
   iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
      0  1.6109693e+01 1.12e+01 3.89e+01   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
      1  2.3652710e+01 1.12e+00 1.22e+01  -0.5 1.22e+01    -  9.80e-02 1.00e+00h  1
      2  1.7171955e+01 9.56e-01 1.14e+01  -0.0 4.59e+01    -  9.92e-01 2.70e-01f  1
      3  1.7506115e+01 3.03e-02 3.00e-01  -0.7 3.06e-01    -  1.00e+00 1.00e+00h  1
      4  1.7002639e+01 1.22e-01 3.20e-02  -1.3 3.46e-01    -  9.59e-01 1.00e+00f  1
      5  1.6970896e+01 8.58e-02 2.37e-02  -2.9 8.74e-02    -  9.33e-01 1.00e+00h  1
      6  1.7013026e+01 2.34e-03 5.93e-04  -4.2 1.77e-02    -  1.00e+00 1.00e+00h  1
      7  1.7014017e+01 4.01e-06 1.49e-06  -5.9 8.07e-04    -  1.00e+00 1.00e+00h  1
      8  1.7014017e+01 1.42e-10 1.30e-10 -11.0 3.76e-06    -  1.00e+00 1.00e+00h  1
   
   Number of Iterations....: 8
   
                                      (scaled)                 (unscaled)
   Objective...............:   1.7014017140162622e+01    1.7014017140162622e+01
   Dual infeasibility......:   1.2966117155257268e-10    1.2966117155257268e-10
   Constraint violation....:   1.4200551845533482e-10    1.4200551845533482e-10
   Variable bound violation:   9.9621045857034574e-09    9.9621045857034574e-09
   Complementarity.........:   4.1225384883754112e-11    4.1225384883754112e-11
   Overall NLP error.......:   1.4200551845533482e-10    1.4200551845533482e-10
   
   
   Number of objective function evaluations             = 9
   Number of objective gradient evaluations             = 9
   Number of equality constraint evaluations            = 9
   Number of inequality constraint evaluations          = 10
   Number of equality constraint Jacobian evaluations   = 9
   Number of inequality constraint Jacobian evaluations = 9
   Number of Lagrangian Hessian evaluations             = 8
   Total seconds in IPOPT                               = 0.015
   
   EXIT: Optimal Solution Found.
   
   ans =
   
       1.0000    4.7430    3.8211    1.3794
   

[anilvrao]

Did I do something wrong? I get the following error:

Operands to the logical AND (&&) and OR (||) operators must be convertible to logical scalar values. Use the ANY or ALL functions to reduce operands to
logical scalar values.

Error in ipopt (line 264)
elseif strcmp( cmp, 'MACI64') == 1 || regexp( cmp, 'x86_64-apple-darwin') == 1

Error in examplehs038 (line 57)
[x info] = ipopt(x0,funcs,options);

[anilvrao]

Here is what I get when I build the Mex file after installing the patch you provided:

---

mex -largeArrayDims -Isrc ./src/ipopt.cc ./src/IpoptInterfaceCommon.cc -Isrc_ipopt_osx/coin-or -DOS_MAC -output bin/osx/ipopt_osx -L/opt/homebrew/lib -lipopt /opt/homebrew/lib/gcc/current/libquadmath.dylibLDFLAGS='$LDFLAGS -Wl,-rpath,. -framework Accelerate -ldl' CXXFLAGS='$CXXFLAGS -Wall -O2 -g'
Building with 'Xcode Clang++'.
/Users/anilvrao/Desktop/mexIPOPT/toolbox/./src/IpoptInterfaceCommon.cc:1890:15: warning: 'vsprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use vsnprintf(3) instead. [-Wdeprecated-declarations]
nchar = vsprintf(s,pformat,ap);
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/usr/include/stdio.h:199:1: note: 'vsprintf' has been explicitly marked deprecated here
__deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use vsnprintf(3) instead.")
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'
#define __deprecated_msg(_msg) attribute((deprecated(_msg)))
^
1 warning generated.

ld: warning: -undefined error is deprecated
ld: warning: -undefined error is deprecated

MEX completed successfully.
----------------------- DONE ----------------------------

[anilvrao]

Did you complete your build with clang and clang++ or with gcc and g++?

[anilvrao]

Can you send me your Mex file?

[rdzman]

Did I do something wrong? I get the following error:

Operands to the logical AND (&&) and OR (||) operators must be convertible to logical scalar values. Use the ANY or ALL functions to reduce operands to logical scalar values.

Error in ipopt (line 264) elseif strcmp( cmp, 'MACI64') == 1 || regexp( cmp, 'x86_64-apple-darwin') == 1

Error in examplehs038 (line 57) [x info] = ipopt(x0,funcs,options);

I think the issue here is that you have ebertolazzi's ipopt.m in your path. I find that it does a lot of unnecessary stuff to figure out which mex file to call. If you just rename your MEX file to ipopt.mexmaca64 and get rid of ipopt.m I think it should work.

[rdzman]

Did you complete your build with clang and clang++ or with gcc and g++?

Not actually sure what it uses. I just built it with that patched M-file and it worked for me.

[anilvrao]

I do not have the ipopt.m file used by ebertolazzi in my path. I have only the Mex file in my MATLAB path. I am not sure if I need to reinstall homebrew from scratch.

[rdzman]

If you type which ipopt at the MATLAB prompt what does it return?

[anilvrao]

It returns the location of the Mex file that I compiled using your instructions. —— Anil V. Rao, PhD E-mail: ***@***.*** Tel: 352-672-1529 Web:  http://anilvrao.com

…

On Nov 16, 2023 at 9:03 PM -0500, Ray Zimmerman ***@***.***>, wrote: If you type which ipopt what does it return? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[rdzman]

Ah ... it looks like the examplehs038.m file temporarily adds the ipopt.m to the path. Try one of the other examples.

[anilvrao]

I tried both examplehs038 and examplehs071.  Neither one works and they give the error I posted in my earlier comment. —— Anil V. Rao, PhD E-mail: ***@***.*** Tel: 352-672-1529 Web:  http://anilvrao.com

…

On Nov 16, 2023 at 9:32 PM -0500, Ray Zimmerman ***@***.***>, wrote: Ah ... it looks like the examplehs038.m file temporarily adds the ipopt.m to the path. Try one of the other examples. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[rdzman]

Comment out the addpath() command in those examples. The error you posted earlier is happening in ipopt.m.

[ebertolazzi]

I read the various discussions.
I decide to change the way to compile IPOPT based on cmake.
Now in the branch develop I have reorganized the procedure:

Read README-OSX.md to setup an up-and-running IPOPT on OSX (Linux and windows for the moment I have not changed).

For MA57 there is no way to use MATLAB internal lib. To use libhsl.dylib it is necessary the lilbrary is in the same directory as the running script or in a directory MATLAB is searching for dynamic library. For example
/Applications/MATLAB_RXXXX.app/bin.

[rdzman]

Thanks @ebertolazzi. I gave it a try on my M2 Mac with the native Apple Silicon version of MATLAB and I'm afraid it didn't work. It compiled fine, but I get the following runtime error ...

Invalid MEX-file
'/usr/local/ipopt/matlab-build/ebertolazzi/mexIPOPT/toolbox/lib/ipopt.mexmaca64':
dlopen(/usr/local/ipopt/matlab-build/ebertolazzi/mexIPOPT/toolbox/lib/ipopt.mexmaca64,
0x0006): Library not loaded: @rpath/libMatlabEngine.dylib
  Referenced from: <03DCC345-5CDF-3C01-9167-13B4217B2A96>
  /usr/local/ipopt/matlab-build/ebertolazzi/mexIPOPT/toolbox/lib/ipopt.mexmaca64
    Reason: tried: '$ORIGIN/libMatlabEngine.dylib' (no such file),
  '$ORIGIN/libMatlabEngine.dylib' (no such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/libMatlabEngine.dylib' (no
  such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/./libMatlabEngine.dylib'
  (no such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/../../sys/os/maca64/libMatlabEngine.dylib'
  (no such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/libMatlabEngine.dylib' (no
  such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/./libMatlabEngine.dylib'
  (no such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/../../sys/os/maca64/libMatlabEngine.dylib'
  (no such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/../../runtime/maca64/libMatlabEngine.dylib'
  (no such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/../../sys/java/jre/maca64/jre/lib/server/libMatlabEngine.dylib'
  (no such file),
  '/Applications/-math/MATLAB_R2023b.app/bin/maca64/../../sys/java/jre/maca64/jre/lib/libMatlabEngine.dylib'
  (no such file)

FWIW, the build.make file in build/CMakeFiles/ipopt.dir includes the line:

ipopt.mexmaca64: /Applications/-math/MATLAB_R2023b.app/extern/bin/maca64/libMatlabEngine.dylib

And that file does exist.

Also, In case it's relevant, as you can see, I do have my MATLAB installed in a subdirectory of /Applications rather than in the default location (i.e. /Applications itself).

[anilvrao]

I commented out the addpath command in the examples and the Mex file works! I also tried it on several of the examples in our optimal control software GPOPS-II and it works great! Actually, I have never seen performance this fast with IPOPT. The improvement must be a combination of the Silicon version of MATLAB together with the Silicon processor itself! Quite remarkable how much faster GPOPS-II runs with this new Mex file!

Thank you for your help!

[anilvrao]

Using your same instructions above, I was not able to compile the Mex file for Apple Silicon on my M1 laptop. I am getting the following error when I run "CompileIpoptMexLib":

Error using mex
Supported compiler not detected. For options, visit https://www.mathworks.com/support/compilers.

Error in CompileIpoptMexLib (line 128)
eval(CMD);

I have so many C and C++ compilers installed on my computer, and they are the same ones that are installed on my Apple Studio. I am not sure why MATLAB does not find a supported compiler.

I tried "mex -setup" and "mex -setup -v". When I run the latter I obtain the following output. Not sure what is going on.

Output of "mex -setup -v":

Verbose mode is on.
... Looking for compiler 'Xcode with Clang' ...
... Looking for environment variable 'DEVELOPER_DIR' ...No.
... Executing command 'xcode-select -print-path' ...Yes ('/Library/Developer/CommandLineTools').
... Looking for folder '/Library/Developer/CommandLineTools' ...Yes.
... Executing command 'which xcrun' ...Yes ('/usr/bin/xcrun').
... Looking for folder '/usr/bin' ...Yes.
... Executing command 'defaults read com.apple.dt.Xcode IDEXcodeVersionForAgreedToGMLicense' ...No.
... Executing command 'defaults read /Library/Preferences/com.apple.dt.Xcode IDEXcodeVersionForAgreedToGMLicense' ...Yes ('15.0.1').
... Executing command '
agreed=15.0.1
if echo $agreed | grep -E '[."]' >/dev/null; then
lhs=expr "$agreed" : '\([0-9]*\)[\.].*'
rhs=expr "$agreed" : '[0-9]*[\.]\(.*\)$'
if echo $rhs | grep -E '[."]' >/dev/null; then
rhs=expr "$rhs" : '\([0-9]*\)[\.].*'
fi
if [ $lhs -gt 4 ] || ( [ $lhs -eq 4 ] && [ $rhs -ge 3 ] ); then
echo $agreed
else
exit 1
fi
fi' ...Yes ('15.0.1').
... Executing command 'xcrun -sdk macosx --show-sdk-path' ...No.
Did not find installed compiler 'Xcode with Clang'.
Error using mex
Supported compiler not detected. For options, visit https://www.mathworks.com/support/compilers.

[rdzman]

I got that error too even though I had accepted the Xcode license, so I just commented out the lines in CompileIpoptMexLib that do that check.

[anilvrao]

I followed the above instructions for compilation of the IPOPT Mex file on a new Apple Silicon M3 Max computer. I receive the following errors upon running CompileIpoptMexLib. Apparently the version of IPOPT is for Intel (X86_64) and not for Apple Silicon, but I ran "brew install ipopt" to install IPOPT.

Any help is appreciated.

Thanks!

Error using mex
ld: warning: ignoring file '/usr/local/Cellar/ipopt/3.14.14/lib/libipopt.3.dylib':
found architecture 'x86_64', required architecture 'arm64'
ld: Undefined symbols:
  Ipopt::IpoptApplication::Initialize(bool), referenced from:
      _mexFunction in ipopt.o
  Ipopt::IpoptApplication::Statistics(), referenced from:
      _mexFunction in ipopt.o
      _mexFunction in ipopt.o
  Ipopt::IpoptApplication::OptimizeTNLP(Ipopt::SmartPtrIpopt::TNLP const&),
  referenced from:
      _mexFunction in ipopt.o
  Ipopt::IpoptApplication::IpoptApplication(bool, bool), referenced from:
      _mexFunction in ipopt.o
  Ipopt::IpoptApplication::~IpoptApplication(), referenced from:
      _mexFunction in ipopt.o
      _mexFunction in ipopt.o
  Ipopt::Journal::SetPrintLevel(Ipopt::EJournalCategory, Ipopt::EJournalLevel),
  referenced from:
      vtable for Ipopt::MatlabJournal in IpoptInterfaceCommon.o
  Ipopt::Journal::SetAllPrintLevels(Ipopt::EJournalLevel), referenced from:
      vtable for Ipopt::MatlabJournal in IpoptInterfaceCommon.o
  Ipopt::Journal::Journal(std::__1::basic_string<char, std::__1::char_traits,
  std::__1::allocator> const&, Ipopt::EJournalLevel), referenced from:
      Ipopt::MatlabJournal::MatlabJournal(Ipopt::EJournalLevel) in
      IpoptInterfaceCommon.o
      Ipopt::MatlabJournal::MatlabJournal(Ipopt::EJournalLevel) in
      IpoptInterfaceCommon.o
  Ipopt::Journal::~Journal(), referenced from:
      Ipopt::MatlabJournal::~MatlabJournal() in IpoptInterfaceCommon.o
      Ipopt::MatlabJournal::~MatlabJournal() in IpoptInterfaceCommon.o
      Ipopt::MatlabJournal::~MatlabJournal() in IpoptInterfaceCommon.o
  Ipopt::Journal::IsAccepted(Ipopt::EJournalCategory, Ipopt::EJournalLevel) const,
  referenced from:
      vtable for Ipopt::MatlabJournal in IpoptInterfaceCommon.o
  typeinfo for Ipopt::Journal, referenced from:
      typeinfo for Ipopt::MatlabJournal in IpoptInterfaceCommon.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Error in CompileIpoptMexLib (line 128)
eval(CMD);

[rdzman]

Is it possible you are pointing it to an old Homebrew install under /usr/local, rather than a current one in the default location which is now /opt/homebrew?

[anilvrao]

I am now able to compile to get a Mex file, but I get the following error when I try to execute the Mex file. Looks like something is still looking for an Intel version.

Thanks again for any help.

Operands to the logical AND (&&) and OR (||) operators must be convertible to
logical scalar values. Use the ANY or ALL functions to reduce operands to logical
scalar values.

Error in ipopt (line 264)
    elseif strcmp( cmp, 'MACI64') == 1 || regexp( cmp, 'x86_64-apple-darwin') == 1

Error in examplehs038 (line 57)
  [x info] = ipopt(x0,funcs,options);

[rdzman]

This looks like the same issue you had back in Nov. See my previous reply.

[anilvrao]

Ok. I got everything working. Thanks for your help.

I have another question. Which linear solver is better, MUMPS or MA57? I know from a few colleagues that the group in France has made some amazing improvements to MUMPS over the past few years. From what I understand, MUMPS now outperforms MA57, but I am not sure if my understanding is correct.

Also, is the MUMPS library that is included with the mexIPOPT toolbox up to date, or should an updated version of this library be used when compiling the Mex file? In addition, is the MUMPS library dynamically loaded, or is it statically compiled for use with the Mex file?

Finally, I know that Mathworks has included a dynamic MA57 library for years. I see, however, that for the Apple Silicon version of MATLAB, the library libmwma57.dylib is found in the directory /Applications/MATLAB_R2023a.app/bin/maci64 for MATLAB 2023a and in the directory /Applications/MATLAB_R2023b.app/bin/maca64 for MATLAB 2023b. Given the different locations of this library in the two different Apple Silicon versions of MATLAB, is this library compatible with the Apple Silicon processor, or is it an "Intel" version that was copied from the Intel version of MATLAB? I have tried linking to this MA57 library when compiling the Mex file but it does not work as I receive an error stating that the library cannot be loaded. Wondering if Mathworks forgot to update the MA57 library when creating the Apple Silicon version of MATLAB.

Sorry for so many questions. Just trying to see what configuration will provide the best performance.

[anilvrao]

Does anyone have any knowledge as to which linear solver is better when choosing between MUMPS and MA57? I have been told that MUMPS has been improved immensely over the past several years and now outperforms MA57, but I would like to know if anyone has any direct experience comparing the performance of these linear solvers.

Another question. Does a publicly available (and freely distributable) version of MA57 exist? I ask because nobody seems to have been able to link IPOPT to the MATLAB MA57 dynamic library when compiling the IPOPT mex file.

Thanks!

[wahln]

Another question. Does a publicly available (and freely distributable) version of MA57 exist? I ask because nobody seems to have been able to link IPOPT to the MATLAB MA57 dynamic library when compiling the IPOPT mex file.

I once managed to do that on Windows. I got inspired by a zip-package within the OPTI toolbox here: https://github.com/jonathancurrie/OPTI/blob/c9636bb2da69dc225d5791eac04fb36c7c0e71c8/Solvers/Source/opti/libmwma57.zip
It provides a def file and mock interfaces, which allow you to link the lib providing the respective symbols while making the mex-file. When calling the mex-interface, the ma57-dll will be found on matlab's library path.
Ipopt version was older though, and just now I couldn't find my pipeline anymore to give the exact details. If you are interested, I should be able to find the mex file.
Don't know how easily this could be transfered to mac, though, as I don't have much experience with the Silicon architecture.

[jadechantrell]

Hello @rdzman thanks for the steps above. I'm still having issues with the patch provided
I've followed the steps. Since I originally had @ebertolazzi add-on installed I renamed the compile~.m and ipopt.m files to ~_original.m and checked the ipopt.m it was calling. Copied the ipopt_osx.mexmaci64 file into the MEX osx file on the path.

Any suggestions? The mac I'm working on is M1. I do have an M2 available to me as well.

When I run the examples I comment out the addpath commands.
I'm still getting error:

examplehs038
Operands to the logical AND (&&) and OR (||) operators must be convertible to
logical scalar values. Use the ANY or ALL functions to reduce operands to logical
scalar values.

Error in ipopt (line 264)
elseif strcmp( cmp, 'MACI64') == 1 || regexp( cmp, 'x86_64-apple-darwin') == 1

Error in examplehs038 (line 57)
[x info] = ipopt(x0,funcs,options);

[rdzman]

Sorry, @jadechantrell, just seeing this now. I'm pretty sure this error is because you have the ipopt.m file in your path. In my view, that file unnecessarily complicates things. I prefer to simply rename the MEX file to ipopt.mex..., then you don't need the M-file at all.