How do create a language server with Surelog/UHDM

[alaindargelas]

From @hs-apotell:

A little context here would have helped. My bad.

This issue came up because one of team member is attempting to get a language server working and he has a very basic functional prototype to showcase. It was just by accident that he was using a rather large design for test case and the response time was very slow. So, he decided to fallback to load just the cache file, which worked well (for the minimum feature set he was focusing on) until he pulled and refreshed his local repository.

Long story short, if we are working with rather large design and we know for sure that only a specific file has been modified and need to be refreshed in the cache, can this be done safely. This thread is the result of that discussion.

So, coming back to the need for fast response time that is required for a language server like utility to work, what can be done to avoid/prevent having to re-parse the entire design over and over again for every keystroke?

[alaindargelas]

1. Let's say you have a large set of RTL IPs that rarely change and you want to precompile a UHDM model for each one of them that you want to import into an existing project (And refresh the precompiled model when the RTL source changes).
   You need to create a system that will load all the UHDM DBs for the IPs you are using in the project. UHDM needs to offer a complete model that is unelaborated.
   Today to get the complete model of a given set of modules/packages it requires both the nonelaborated and the elaborated views. That is what the Yosys and Verilator plugins use to build the design.
   This implies we need to come up with a non-elaborated view that completely models the modules/packages (generate stmts) that can be elaborated later in the context of the complete design under editing.

2. You want interactive speed when doing code completion in an IDE. For the part of the code that is under heavy editing, you need to stay away from saving/loading UHDM files from disk, you need inter process communication in between the IDE and the server (Using libsurelog.a). Surelog is capable of incremental parsing (look at the unit tests in Surelog we recently created), and with a big of work will even be able to invoke the Antlr parser on a snippet of code (The line that just got edited), not a complete file. The UHDM model can be updated in memory but should not saved/reloaded from disk. The serialization/deserialization of UHDM is a slower that it should be because it is transforming all the pointers into IDs and is updating all the relations for the purpose of serialization and is doing the counterpart for deserialization. So try to stay in memory and avoid this.

[alaindargelas]

@hs-apotell for comments.

[hs-apotell]

You need to create a system that will load all the UHDM DBs for the IPs you are using in the project.

We are planning on doing this. We are still in a prototype phase focused on very basic features. I am bringing up this issue now so we have time to plan for a how to use surelog for a full featured language server.

This implies we need to come up with a non-elaborated view that completely models the modules/packages (generate stmts) that can be elaborated later in the context of the complete design under editing.

Completely out of scope for us.

look at the unit tests in Surelog we recently created

Please provide some test names. So far, we have been invoking the entire process thru' the command line args way and that isn't optimal for use with language server. All the saving/loading is happening as part of call to start/shutdown compiler.

[alaindargelas]

I changed the meaning of -nouhdm option (You can also use CommandLinePrser::writeUhdm(false) ) .
When turned off, UHDM is still produced in-memory but it is not dumped on disk.
So in an interactive session, you save the write/reload time that is by default done in the start_compiler function.
If you use the example given in hellouhdm you are fine, no need to switch to the mechanism used in the unit tests like Elaboration_test.cpp.

[hs-apotell]

We will give this a try and revert back with findings.

P.S. This might take a little longer to implement since the developer who was working on it is on personal leave for next week.

[alaindargelas]

Is this still relevant or you have moved on some other solution?

[hs-apotell]

Yes. There is work in progress on this just not a very high priority at the moment for us. Unfortunately, the core developer working on this had to move on to something more pressing. The core issue is however still relevant.

[hzeller]

For other implementations: I've implemented a SystemVerilog language server as one of the tools in Verible, which at this point does syntax checking, linting and formatting https://github.com/chipsalliance/verible/tree/master/verilog/tools/ls

[hs-apotell]

BTW, are you going to contribute back the Linter or the LS? Have you noted the UhdmLinter and SynthSubset optional classes in UHDM?

Yes & No. Yes it will be open source once I have it working with demonstrable features. However, It may be part of Surelog. The reason being that I am employing two different parsers - Surelog & Verible. Surelog for static parsing i.e. all the files that aren't currently changing and Verible for dynamic parsing i.e. the ones that are actively being edited. I think this approach gets us best of both worlds. All the semantic information that Surelog can provide and the speed that is needed for dynamically changing code.

[alaindargelas]

Excellent!