Error in execution

[nitinnew]

Hello!

I am encountering the following error while executing. Will be grateful if you may please check:-

Traceback (most recent call last):

File "/usr/local/lib/python3.10/dist-packages/IPython/core/interactiveshell.py", line 3553, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)

File "", line 6, in <cell line: 6>
import dbcv

File "/usr/local/lib/python3.10/dist-packages/dbcv/init.py", line 1, in
from .core import dbcv

File "/usr/local/lib/python3.10/dist-packages/dbcv/core.py", line 35
return arr[*np.meshgrid(inds_a, inds_b)]
^
SyntaxError: invalid syntax

[FelSiq]

Hi,

I was able to reproduce your error using Python 3.10. Weirdly enough, it hadn't happened when I tested using Python 3.11.

I'll check the exact cause of this behavior, and I'll upload a fix for this issue as soon as possible.

Thanks for letting me know about this issue,
Felipe.

[FelSiq]

I've done a quickfix to the main branch regarding this issue. Please install the updated version, and you should be good to go this time.

If the problem persists, or you encounter any other issue, please let me know.

Felipe.

[nitinnew]

Thanks for the prompt update. I was able to get it running.

I wanted to check if you have had a look at the implementation at https://github.com/christopherjenness/DBCV.
While your implementation is much faster, i am getting different DBCV score with both.
Would it be possible for you to have a look please?

[FelSiq]

Thanks for your response.

I indeed had looked into Christopher's implementation. In fact, it was that implementation that motivated me writing this package. In my opinion, that implementation was far too complex when compared to the original's paper description [1] (admittedly, mine was far more simpler before I implemented support for multiprocessing and variable precision bits for density computation), isn't vectorized at all (hence why it is so slow), and I didn't quite understand how his implementation handles noise (or even if it handles noise at all).

I'm aware that both implementation handle different results but, up to my personal interpretation of [1], I believe I followed correctly the description provided by the authors. Sadly, they don't provide an official implementation, so it's hard to tell apart who's right and who's wrong in this case.

Also note that I use the metric "sqeuclidean" (squared Euclidean distance) by default, not the Euclidean distance. That metric is mentioned in [1] as one valid possibility, and as far as I know there's no clear way to decide upfront which option is best for your analysis.

I suggest you to take a look into [1], analyze both implementations (Christopher's and mine), and draw your own conclusions. If you find any mistake in either implementation, please let us know.

EDIT (29/04/24): My current implementation is correct (tested against the original MATLAB implementation). See further discussion in #3.

Best regards,
Felipe.

---

[1] Moulavi, Davoud & Andretta Jaskowiak, Pablo & Campello, Ricardo & Zimek, Arthur & Sander, Joerg. (2014). Density-Based Clustering Validation. 10.1137/1.9781611973440.96.

[nitinnew]

Thank you so much for your detailed inputs! And thanks for this implementation. I will examine the paper as well and reach out in case I need some more assistance on this.

Warm regards,

[FelSiq]

Hello David. Thanks for your feedback. You are correct. My current implementation has a space complexity of O(N**2). Perhaps a way to mitigate this issue would be implementing a fall-back computation method for larger datasets with lower space complexity, although I'm not exactly sure how to do it efficiently (iterating over data points in Python is unacceptable imo). I'm open to suggestions. Best regards, Felipe. Em seg., 29 de abr. de 2024 às 01:38, David Waterworth < ***@***.***> escreveu:

…

@FelSiq <https://github.com/FelSiq> one difference between yours and Christopher's implementation is you seem to compute the full distance matrix upfront (using cdist) requiring O(n**2) memory, whilst Christopher's implementation appears to compute the distance from a point to (all?) neighbors - so I think it's probably O(n) memory but trades that off against speed. I rapidly ran into out-of-memory on large datasets using your version - on the other hand Christopher's is quite slow. — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEOOFNACQZVOAW6XEEWWI4DY7XFFHAVCNFSM6AAAAAA7KM72JWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRHA3TMOJWHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>