Improve extract speed

[vvmruder]

Improve municipality handling, this might break existing adapted non standard sources!!!

This is related to #823

[peterschaer]

Thanks! This is waaaay faster than before (on a small real estate from 3secs to 1secs for a JSON extract!). So it looks good to me.

[vvmruder]

@peterschaer Nice that it works. Do you know if you use some adapted elements in your project which you need to change? Or was it working out of the box?

[vvmruder]

And in addition: Did you tried the capabilities request? The fix is interfering with this. So I would be interested if output is still like you would expect it.

[peterschaer]

It was working out of the box, I didn't have to change anything. however the capabilities request has different output. before it had 346 municipalities (the same number as in the municipality table), now it has 702 municipalities. there are a lot of duplicates.

[vvmruder]

Ups... I will have a look at it.

[vvmruder]

@peterschaer Could you please try again? I hope my fix solve this issue.

[vvmruder]

And maybe can you try several times to see if the duplicates are increasing if the error still exists?

[pfirpfel]

Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- pyramid_oereb/standard/sources/municipality.py  1
         

See the complete overview on Codacy

[peterschaer]

very good, the duplicates are gone even if i try multiple times.

[vvmruder]

Juhu. I will wait for review of the others. But in my opinion we can merge this. It is quite an improvement. Even if it only really bothers cantons with really many municipalities 😃

[vvmruder]

@peterschaer Thanks for catching this in the code!

[kdeininger]

Very nice performance improvement!

This change should be even allow backward compatibility as fosnr is an optional argument. Calling the read methods without argument, they will act the same way as they did before and even if someone has extended one of the classes, the optional argument should not cause any problems.

[vvmruder]

@kdeininger I'am glad you see it like this. I think this too. I will wait for comment from @jwkaltz and then we can merge.

[jwkaltz]

Thanks @vvmruder it looks good to me, feel free to merge.

Question: should we increase the version number of pyramid_oereb to 1.6, since this is a potentially breaking change? (Although I am currently not aware of anybody using non-standard sources)

[voisardf]

Thanks a lot, just one question not having the data flow present right now.

[voisardf]

Is it still a list at this point? If not, we should adapt the comment

[kdeininger]

Yes, it still is a list. But this is a good point. The comment should be adapted anyway as the list may be filtered now.