Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TW#14952] How to use CMake compile library for esp32 (Project: open62541) #921

Closed
Gfast2 opened this issue Aug 19, 2017 · 16 comments
Closed

Comments

@Gfast2
Copy link

Gfast2 commented Aug 19, 2017

Hi esp-idf,

In the past two days I've spent many hours trying to port open62541 project to esp32 platform with no luck. 🦄


About this protocol: Besides MQTT as communication protocol for IoT applications, many companys choose OPC UA as their preferred choice. Because OPC Classic is well adopted since late 90's as M2M protocol in industry. Till now this standard is going on developed, specified and well standardized as IEC TR 62541 by opc foundation and meanwhile very well documented on its online doc. Big Players like ABB, Bosch, Festo, KUKA, SAP, Siemens, Cisco, Microsoft ... are their paied members.

The "Problem" of OPC UA is that its not easy to getting started with and not real open-source. For example, as far as I know/understand, even this in public domain hosted UA-AnsiC Stack from opc foundation will only cost free for their paied members. The Project open62541 on the opposite side is the single open source implementation of OPC UA in C. (List of Open Source OPC UA Implementations).


Right now this project deploy CMake to build the library from its source. I've tested it on my linux boxes. I can build/use this library in Linux enviroment without any issue. Because esp-idf uses FreeRTOS+LWIP for xtensa cores. I started from this PR which is for ST32 (also FreeRTOS+LWIP based). After 2 day long tweaking around, I've learned pretty much from it. ("CMake Cross Compiling", "How does CMake works"...). But I still can not get this job done...


I've to say, it's beyond my right now skill range. I'll appreciate to any help from here. 🚀 🌝

P.s:
Because I really don't know how far what I've done before it makes open62541 works on esp32, So I don't post my error logs here. If it did make sense, I will.

@Spritetm
Copy link
Member

FWIW, that STM32 branch is a mess when it comes to the Makefile... toolchain-specific hacks (like the nano.spec, Windows-specific and install-specific paths hardcoded, ... I'm not surprised you couldn't get it to compile.

Actually, the vanilla project seems to almost compile given the correct makefile... I've whipped one up here: https://gist.github.com/Spritetm/3f0c30af13b92a06a91850996694997d . The only issue seems to be that the lwip bsd-socket implementation isn't entirely compatible. The library may need some mangling in the tcp connector, and you may be fine.

@Gfast2
Copy link
Author

Gfast2 commented Aug 20, 2017

Hi @Spritetm ,

thanks for you explanation and this Gist first! It enlight me to trying the other possible way to build up this library - Using esp-idf's build system directly 💡
I'll checkout your component.mk today later.


P.s:
This let me come to another question: Though Uncle Google can tell me endless info about kconfig. But I still didn't get the core idea... Is it something similar to cmake, but very Linux kernel (and esp-idf) specilized?

@Spritetm
Copy link
Member

Actually, esp-idfs build system normally handles compiling and linking itself, but it can be persuaded to relegate those tasks to other build systems - like cmake. The component.mk file in the gist does just that: it sets up the environment for cmake to understand esp-idfs compiler and include paths, then just calls on cmake to do the hard work of compiling and linking.

Kconfig is the config file that is used in 'make menuconfig', most famously known from the Linux kernel. I don't know cmake too well, but I infer it also has a configuration mechanism that is somewhat alike to it, yes.

@Gfast2
Copy link
Author

Gfast2 commented Aug 20, 2017

Hi @Spritetm. Cool thanks for your answer firstly. I'll update my working status here later! (Hopfully do even a pull request for this.) 😎
(Sorry for the Issue close / repoen, my 9 month baby loves my mouse 🍼 )

@Gfast2 Gfast2 closed this as completed Aug 20, 2017
@Gfast2 Gfast2 reopened this Aug 20, 2017
@FayeY FayeY changed the title How to use CMake compile library for esp32 (Project: open62541) [TW#14952] How to use CMake compile library for esp32 (Project: open62541) Aug 25, 2017
@Gfast2
Copy link
Author

Gfast2 commented Sep 13, 2017

Hi ESP, I'm back. 🖐

I've tried the component.mk file with a clean open62541 source. I stucked here:

[ 50%] Building C object CMakeFiles/open62541-object.dir/open62541.c.obj

Because of the lack of library porting experience. Though I've invest another couple of hours, I still can't let cmake happy with esp_idf ecosystem. I got two type of errors.

  • First type:
/home/gfast2/esp/esp-idf/components/esp32/include/esp_intr_alloc.h:267:1: error: function declaration isn't a prototype [-Werror=strict-prototypes]
 void esp_intr_noniram_disable();

In my opinion it's the compiler restriction setting which controlled by cmake is somehow more servere then the implementations of esp-idf. There are two discussions about this: This and This. But I think these shouldn't be big deal for me. 🍈

  • Second type:
/home/gfast2/workspace/open62541_compile/build/open62541/open62541.c:26684:25: fatal error: sys/select.h: No such file or directory

This is the point where I spend most of my time. According to This Question. I've tried to fake these files, but still with no luck. Is there some suggestions for me ❔


The only issue seems to be that the lwip bsd-socket implementation isn't entirely compatible. The library may need some mangling in the tcp connector, and you may be fine.

Is this means, even though I got succeed to port this library to esp_idf, I still need to fighting some incompatibilitys here. ♿️


If the full error log is needed, I'll post here. 😸

@igrr
Copy link
Member

igrr commented Sep 13, 2017

error: function declaration isn't a prototype [-Werror=strict-prototypes]

We don't use -Wstrict-prototypes when compiling IDF source code, so header files may contain constructs which are not compatible with this option.

fatal error: sys/select.h: No such file or directory

At the moment, there is no sys/select.h on this platform. There is LwIP select, but it can only be used with LwIP sockets. So you need to port that part of the library (i.e. modify it to use LwIP's select) if you want to use it in ESP-IDF.

@Gfast2
Copy link
Author

Gfast2 commented Oct 9, 2017

Thanks @igrr ! 🥇
Right now when I compile this library with the marvelous component.mk by @Spritetm the first kind of error is through tweaking CMakeLists.txt beautifully solved.

I'm putting some effort to inlcude the right header files for plugins/ua_network_tcp.c from this library. Till now cmake can nearly finish the job (82% at the time). But get stucked by including sys/socket.h which I manually add into this file. Because esp-idf is none of Linux, QNX, BSD... If I don't include sys/socket.h, ESP-IDF won't throw fatal error: xx/xx.h: No such file or director, but only a pile of error: implicit declaration of function 'xxxx' (most of the missing functions can be found in LWIP's sys/socket.h)

No matter what I've tried - #include <sys/socket.h>, #include <socket.h>, or #include <posix/sys/socket.h>, it seems like the cmake won't include the header from esp-idf. Is that means in order to let cmake know which folder should be included, I should specify more detailed information in its component.mk or any tweaking in CMakeLists.txt for cmake?

Any suggestions are highly appreciated! 👼

@Gfast2
Copy link
Author

Gfast2 commented Oct 10, 2017

Hi ESP-IDF,

The test ist going on here. 🌞
I tried to add the include directory where the #include <sys/socket> can be found by cmake in its library´s component.mk. (And I made this file as a Gist. And I defined INCLUDE_FIXUP_DIR as the path where this header file can be found directly.


My Question is somehow simplified:
How can I known which included folders are being defined vendored by ESP-IDF to cmake?

@Gfast2
Copy link
Author

Gfast2 commented Oct 10, 2017

In case some one wanna see the compiling log of cmake:

(I made a small sorting for the compiling log)

Part1:

-- Build files have been written to: /home/gfast2/workspace/ioc-110-ethernet/build/open62541
[  2%] Generating src_generated/ua_nodeids.h
[  5%] Generating src_generated/ua_types_generated.c, src_generated/ua_types_generated.h, src_generated/ua_types_generated_handling.h, src_generated/ua_types_generated_encoding_binary.h
[  7%] Generating src_generated/ua_transport_generated.c, src_generated/ua_transport_generated.h, src_generated/ua_transport_generated_handling.h, src_generated/ua_transport_generated_encoding_binary.h
[ 10%] Generating src_generated/ua_statuscode_descriptions.c
Scanning dependencies of target open62541-object
[ 12%] Building C object CMakeFiles/open62541-object.dir/src/ua_types.c.o
[ 15%] Building C object CMakeFiles/open62541-object.dir/src/ua_types_encoding_binary.c.o
[ 17%] Building C object CMakeFiles/open62541-object.dir/src_generated/ua_types_generated.c.o
[ 20%] Building C object CMakeFiles/open62541-object.dir/src_generated/ua_transport_generated.c.o
[ 22%] Building C object CMakeFiles/open62541-object.dir/src/ua_connection.c.o
[ 27%] Building C object CMakeFiles/open62541-object.dir/src/ua_session.c.o
[ 30%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_server.c.o
[ 32%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_server_binary.c.o
[ 35%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_server_utils.c.o
[ 37%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_server_worker.c.o
[ 40%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_securechannel_manager.c.o
[ 42%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_session_manager.c.o
[ 45%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_nodes.c.o
[ 47%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_nodestore.c.o
[ 50%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_nodestore_concurrent.c.o
[ 52%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_discovery.c.o
[ 55%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_securechannel.c.o
[ 57%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_session.c.o
[ 60%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_attribute.c.o
[ 62%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_nodemanagement.c.o
[ 65%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_view.c.o
[ 67%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_call.c.o
[ 70%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_subscription.c.o
[ 72%] Building C object CMakeFiles/open62541-object.dir/src/server/ua_services_subscription.c.o
[ 75%] Building C object CMakeFiles/open62541-object.dir/src/client/ua_client.c.o
[ 77%] Building C object CMakeFiles/open62541-object.dir/src/client/ua_client_highlevel.c.o
[ 80%] Building C object CMakeFiles/open62541-object.dir/src/client/ua_client_highlevel_subscriptions.c.o
[ 82%] Building C object CMakeFiles/open62541-object.dir/plugins/ua_network_tcp.c.o
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:21:25: fatal error: sys/socket.h: No such file or directory
compilation terminated.
CMakeFiles/open62541-object.dir/build.make:775: recipe for target 'CMakeFiles/open62541-object.dir/plugins/ua_network_tcp.c.o' failed
make[4]: *** [CMakeFiles/open62541-object.dir/plugins/ua_network_tcp.c.o] Error 1
CMakeFiles/Makefile2:168: recipe for target 'CMakeFiles/open62541-object.dir/all' failed
make[3]: *** [CMakeFiles/open62541-object.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make[2]: *** [all] Error 2
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/component.mk:42: recipe for target 'build' failed
make[1]: *** [build] Error 2
/home/gfast2/esp/esp-idf/make/project.mk:421: recipe for target 'component-open62541-build' failed
make: *** [component-open62541-build] Error 2
gfast2@gfast2-VirtualBox:~/workspace/ioc-110-ethernet$ 

Part2:

CMakeFiles/Makefile2:33: warning: undefined variable 'VERBOSE'
CMakeFiles/Makefile2:168: warning: undefined variable 'COLOR'

@Gfast2
Copy link
Author

Gfast2 commented Oct 10, 2017

Hi ESP-IDF,

It's far more complecated then I expected.


The problem that cmake can't find sys/socket.h is caused by make clean can't clean the build folder. After I delete this folder manually, The result of compiling the same code looks different.


Now I believe I enter the next level of debugging. And I leave the result here:
(This is only for bug shooting of file compiling: ./plugins/ua_network_tcp.c)

gfast2@gfast2-VirtualBox:~/workspace/ioc-110-ethernet$ /home/gfast2/esp/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc  -DUA_DYNAMIC_LINKING_EXPORT -DUA_NO_AMALGAMATION -I/home/gfast2/esp/esp-idf/components/app_trace/include -I/home/gfast2/esp/esp-idf/components/app_update/include -I/home/gfast2/esp/esp-idf/components/aws_iot/include -I/home/gfast2/esp/esp-idf/components/aws_iot/aws-iot-device-sdk-embedded-C/include -I/home/gfast2/esp/esp-idf/components/bootloader_support/include -I/home/gfast2/esp/esp-idf/components/bt/include -I/home/gfast2/esp/esp-idf/components/coap/port/include -I/home/gfast2/esp/esp-idf/components/coap/port/include/coap -I/home/gfast2/esp/esp-idf/components/coap/libcoap/include -I/home/gfast2/esp/esp-idf/components/coap/libcoap/include/coap -I/home/gfast2/esp/esp-idf/components/console/. -I/home/gfast2/esp/esp-idf/components/cxx/include -I/home/gfast2/esp/esp-idf/components/driver/include -I/home/gfast2/esp/esp-idf/components/esp32/include -I/home/gfast2/esp/esp-idf/components/esp_adc_cal/include -I/home/gfast2/esp/esp-idf/components/ethernet/include -I/home/gfast2/esp/esp-idf/components/expat/port/include -I/home/gfast2/esp/esp-idf/components/expat/include/expat -I/home/gfast2/esp/esp-idf/components/fatfs/src -I/home/gfast2/esp/esp-idf/components/freertos/include -I/home/gfast2/esp/esp-idf/components/heap/include -I/home/gfast2/esp/esp-idf/components/jsmn/include -I/home/gfast2/esp/esp-idf/components/json/include -I/home/gfast2/esp/esp-idf/components/json/port/include -I/home/gfast2/esp/esp-idf/components/libsodium/port_include -I/home/gfast2/esp/esp-idf/components/libsodium/libsodium/src/libsodium/include -I/home/gfast2/esp/esp-idf/components/log/include -I/home/gfast2/esp/esp-idf/components/lwip/include/lwip -I/home/gfast2/esp/esp-idf/components/lwip/include/lwip/port -I/home/gfast2/esp/esp-idf/components/lwip/include/lwip/posix -I/home/gfast2/esp/esp-idf/components/lwip/apps/ping -I/home/gfast2/workspace/ioc-110-ethernet/main/. -I/home/gfast2/workspace/ioc-110-ethernet/main/minmea -I/home/gfast2/esp/esp-idf/components/mbedtls/port/include -I/home/gfast2/esp/esp-idf/components/mbedtls/include -I/home/gfast2/esp/esp-idf/components/mdns/include -I/home/gfast2/esp/esp-idf/components/micro-ecc/micro-ecc -I/home/gfast2/workspace/ioc-110-ethernet/components/mongoose/. -I/home/gfast2/esp/esp-idf/components/newlib/platform_include -I/home/gfast2/esp/esp-idf/components/newlib/include -I/home/gfast2/esp/esp-idf/components/nghttp/port/include -I/home/gfast2/esp/esp-idf/components/nghttp/nghttp2/lib/includes -I/home/gfast2/esp/esp-idf/components/nvs_flash/include -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/include -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/src_generated -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins -I/home/gfast2/esp/esp-idf/components/openssl/include -I/home/gfast2/esp/esp-idf/components/pthread/include -I/home/gfast2/esp/esp-idf/components/sdmmc/include -I/home/gfast2/esp/esp-idf/components/soc/esp32/include -I/home/gfast2/esp/esp-idf/components/soc/include -I/home/gfast2/esp/esp-idf/components/spi_flash/include -I/home/gfast2/esp/esp-idf/components/spiffs/include -I/home/gfast2/esp/esp-idf/components/tcpip_adapter/include -I/home/gfast2/esp/esp-idf/components/ulp/include -I/home/gfast2/esp/esp-idf/components/vfs/include -I/home/gfast2/esp/esp-idf/components/wear_levelling/include -I/home/gfast2/esp/esp-idf/components/wpa_supplicant/include -I/home/gfast2/esp/esp-idf/components/wpa_supplicant/port/include -I/home/gfast2/esp/esp-idf/components/wpa_supplicant/../esp32/include -I/home/gfast2/esp/esp-idf/components/xtensa-debug-module/include -I/home/gfast2/workspace/ioc-110-ethernet/build/include -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/include -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/src_generated -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins -I/home/gfast2/workspace/ioc-110-ethernet/build/open62541/src_generated -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/deps -I/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/src  -mlongcalls -nostdlib  -DWITH_POSIX -DMBEDTLS_CONFIG_FILE='"mbedtls/esp_config.h"' -DHAVE_CONFIG_H  -Wall -fPIC -o CMakeFiles/open62541-object.dir/plugins/ua_network_tcp.c.obj   -c /home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c: In function 'socket_write':
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:117:27: error: 'UA_Connection {aka struct UA_Connection}' has no member named 'lwip_close_r'
                 connection->close(connection);
                           ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c: In function 'socket_set_nonblocking':
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:214:37: error: macro "fcntl" requires 3 arguments, but only 2 given
     int opts = fcntl(sockfd, F_GETFL);
                                     ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:214:16: warning: initialization makes integer from pointer without a cast [-Wint-conversion]
     int opts = fcntl(sockfd, F_GETFL);
                ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c: In function 'ServerNetworkLayerTCP_start':
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:389:8: warning: implicit declaration of function 'gethostname' [-Wimplicit-function-declaration]
     if(gethostname(hostname, 255) == 0) {
        ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c: In function 'UA_ClientConnectionTCP':
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:693:21: error: storage size of 'hints' isn't known
     struct addrinfo hints, *server;
                     ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:703:17: warning: implicit declaration of function 'getaddrinfo' [-Wimplicit-function-declaration]
     int error = getaddrinfo(hostname, portStr, &hints, &server);
                 ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:707:34: warning: implicit declaration of function 'gai_strerror' [-Wimplicit-function-declaration]
                        hostname, gai_strerror(error));
                                  ^
In file included from /home/gfast2/esp/esp-idf/components/lwip/include/lwip/posix/sys/socket.h:33:0,
                 from /home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:49:
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:712:40: error: dereferencing pointer to incomplete type 'struct addrinfo'
     SOCKET clientsockfd = socket(server->ai_family, server->ai_socktype,
                                        ^
/home/gfast2/esp/esp-idf/components/lwip/include/lwip/lwip/sockets.h:555:63: note: in definition of macro 'socket'
 #define socket(domain,type,protocol)              lwip_socket(domain,type,protocol)
                                                               ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:721:9: warning: implicit declaration of function 'freeaddrinfo' [-Wimplicit-function-declaration]
         freeaddrinfo(server);
         ^
/home/gfast2/workspace/ioc-110-ethernet/components/open62541/open62541/plugins/ua_network_tcp.c:693:21: warning: unused variable 'hints' [-Wunused-variable]
     struct addrinfo hints, *server;
                     ^

@Gfast2
Copy link
Author

Gfast2 commented Oct 11, 2017

Hi ESP-IDF,

yesterday, I forget to write down some extra problem/discovery here:
the cmake settings of open62541 (CMakeLists.txt) are trying to use -std=c99 as its compiling standard, which will triger the error that the xtensa compiler won't understand asm().

@Gfast2
Copy link
Author

Gfast2 commented Nov 20, 2017

Hi ESP-IDF,

What is the Macro of ESP32 / ESP-IDF for #ifdef inclusion?, I saw _WIN32, __OpenBSD__ ...

@Gfast2
Copy link
Author

Gfast2 commented Dec 13, 2017

Got the library up and running!

#1452

@Gfast2 Gfast2 closed this as completed Dec 13, 2017
@Pro
Copy link
Contributor

Pro commented Feb 23, 2018

Note that support for ESP32 will be added through the PR #1595 (tested and working with my Adafruit ESP32 Board)

@Pro
Copy link
Contributor

Pro commented Apr 17, 2018

My example project for OPC UA on an ESP32 using Arduino is now available here:
https://github.com/pro/open62541-arduino

@xenotrax
Copy link

I try to compile your example project with Arduino .. but I got stuck with following errors..

sketch/open62541.c.o:(.literal.ServerNetworkLayerTCP_listen+0x34): undefined reference to getnameinfo' sketch/open62541.c.o:(.literal.ServerNetworkLayerTCP_start+0x30): undefined reference to gethostname'

There is something wrong but I can't figure out what.
I use arduino-1.8.7-linux64.tar.xz , espressif/arduino-esp32 , and the actual open62541.c/h files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants