Hashlink access violation/segfault, when abstract is used as inerface

[neimanpinchas]

Posted on forum here https://community.haxe.org/t/hashlink-access-violation-when-abstract-is-used-as-inerface/4250
On thx fponticelli/thx.core#289
haxe try https://try.haxe.org/#33718600
GIthub repo to reproduce https://github.com/neimanpinchas/hashlink_fix/tree/main

Possibly related: #685

Altough I hope to find a temporary workaround, I believe that it is in hashlinks interest to become compatiable with the other targets so I am making an issue here to track it, sorry if I have been annoying anyone.

I am suffeing from a Access Violation error in hashlink.
In thx.core

Big.hx

public function multiply(that:BigIntImpl):BigIntImpl {
		if (that.isZero()) //<--- Excpetion here
			return Small.zero;
		return that.isSmall ? multiplySmall(cast that) : multiplyBig(cast that);
	}

BigIntImpl is a interface, and that is a BigInt abstract backed by the BigIntImpl, the method is being invoked by the * operator torough the abstract.

The base class is Big which extens BigIntImpl.

When I check it in the hashlink debugger it seems like that is now a simple int array hashlink debugger says Unknown value isZero.

To my wonder, while trying to make a reroducable example, I bumped into another bug, to be sure I’ve tested it with neko and cpp both are working fine, hashlink is the outlier in both cases.

[m0rkeulv]

#685 was solved with changes in haxe (not hashlink), i tested with the latest nightly build (5.0) and my segfault issues where gone.
i would recommend you test a nightly build and see if these changes also solved your issue.
HaxeFoundation/haxe#11723

[neimanpinchas]

Hi, this is great news.

Will 4.3.5 of 5 days ago work, or only the latest one?

I am looking for a reference how to get or compile the nightly.

[kLabz]

#685 was solved with changes in haxe (not hashlink), i tested with the latest nightly build (5.0) and my segfault issues where gone. i would recommend you test a nightly build and see if these changes also solved your issue. HaxeFoundation/haxe#11723

The provided try haxe still segfaults with Haxe nightlies, though. Maybe the hashlink version does matter?

[m0rkeulv]

The fix was merged 4 days ago (one day after 4.3.5 as far as i can tell) and is not listed in the 4.3.5 release so i think its only in the 5.0 alpha version
(https://build.haxe.org/builds/haxe/)

[kLabz]

And I tested on try haxe with nightlies from yesterday.

[m0rkeulv]

i tested with my setup and looks like this issue is not fully resolved

Haxe 5.0.0-alpha.1+3b8f2e0

src/Main.hx:28: 1907696701
src/Main.hx:29: 493041289
src/Main.hx:31: 51673857090504934725631733395741161813008219008706195647077546500090717902960980748691861309194923468857691927160124603739310617254728238018259587470463767121423492821899131087599955437259767467327500957945679268851815026083280717870978291245507750296608996657996182149579459029355660194999922887
557939590841962345733098343871875643620986727028239980866254301951570939336165745702550378647582994158937633708775930979399081047325282207269491829395658906969991009853921036487709055685278783255145754443531206178796378912908493680110483645663000568104296693754095057336127067858061405090377135901993445246675944526026088702457569858555334738069017964572765368676202383777733663092554016662528453673506859039308722997266774215703882910189572515669950219228625268285247127093247506842298199735962820503464375846283255612837293385957365465972163410569651966430738972935921830353168056361568647805569670516775979056080286424754864813998454770884145537892578230625657287521235062122138020631555593306104251904160638066851442486606842204194500736201728933970315667102641550328277036240586151652358881870758650214926441706430618666225486579444616221252821869621414058795555741449760448603876105266033206752662651304933312833221888315094413176105975172637756124411538336082185949707243238731543947544516256076567476832144209914689292296900861127830074091767763036035668613803846219570050898405390608448052718820372101386857669442764170560651220383707003184592198056163019700533629403976313106589674739166098678533379653336560892319891903709312801542024623151220222971996228894348643202799736058333099460735220595870005420861512756491316755756691974061561741089524451945278384745173473686593111326997237674371010720956053
Uncaught exception: Access violation
Called from thx.bigint.Big.multiply(thx/bigint/Big.hx:113)
Called from thx.bigint.Big.multiply(thx/bigint/Big.hx:113)
Called from thx.bigint.Big.multiply(thx/bigint/Big.hx:113)
Called from $Main.main(Main.hx:32)
Called from .init(?:1)

//BigInt.hx (isPrime)
while(t && less(d, nPrev)) {
        x = x.square().modulo(n); // <-- this  is the code that fails in both 4.3.5 and 5.0 alpha
        if(equals(x, nPrev))
          t = false;
        d = d.multiply(2);
      }

[m0rkeulv]

i played around with the code, and surprisingly removing the inline keyword on the function that does the multiply made the code work

fails:

   @:op(A*B) @:commutative
  inline public function multiply(that : BigInt) : BigInt
    return this.multiply(that);

result:

src/Main.hx:28: 1907696701
src/Main.hx:29: 493041289
src/Main.hx:32: 51673857090504934725631733395741161813008219008706195647077546500090717902960980748691861309194923468857691927160124603739310617254728238018259587470463767121423492821899131087599955437259767467327500957945679268851815026083280717870978291245507750296608996657996182149579459029355660194999922887
557939590841962345733098343871875643620986727028239980866254301951570939336165745702550378647582994158937633708775930979399081047325282207269491829395658906969991009853921036487709055685278783255145754443531206178796378912908493680110483645663000568104296693754095057336127067858061405090377135901993445246675944
526026088702457569858555334738069017964572765368676202383777733663092554016662528453673506859039308722997266774215703882910189572515669950219228625268285247127093247506842298199735962820503464375846283255612837293385957365465972163410569651966430738972935921830353168056361568647805569670516775979056080286424754
864813998454770884145537892578230625657287521235062122138020631555593306104251904160638066851442486606842204194500736201728933970315667102641550328277036240586151652358881870758650214926441706430618666225486579444616221252821869621414058795555741449760448603876105266033206752662651304933312833221888315094413176
105975172637756124411538336082185949707243238731543947544516256076567476832144209914689292296900861127830074091767763036035668613803846219570050898405390608448052718820372101386857669442764170560651220383707003184592198056163019700533629403976313106589674739166098678533379653336560892319891903709312801542024623151220222971996228894348643202799736058333099460735220595870005420861512756491316755756691974061561741089524451945278384745173473686593111326997237674371010720956053
Uncaught exception: Access violation
Called from thx.bigint.Big.multiply(thx/bigint/Big.hx:113)
Called from thx.bigint.Big.multiply(thx/bigint/Big.hx:113)
Called from thx.bigint.Big.multiply(thx/bigint/Big.hx:113)
Called from $Main.main(Main.hx:34)
Called from .init(?:1)

works:

   @:op(A*B) @:commutative
  public function multiply(that : BigInt) : BigInt
    return this.multiply(that);

Result from working code:

src/Main.hx:28: 1907696701
src/Main.hx:29: 493041289
src/Main.hx:32: 51673857090504934725631733395741161813008219008706195647077546500090717902960980748691861309194923468857691927160124603739310617254728238018259587470463767121423492821899131087599955437259767467327500957945679268851815026083280717870978291245507750296608996657996182149579459029355660194999922887
557939590841962345733098343871875643620986727028239980866254301951570939336165745702550378647582994158937633708775930979399081047325282207269491829395658906969991009853921036487709055685278783255145754443531206178796378912908493680110483645663000568104296693754095057336127067858061405090377135901993445246675944
526026088702457569858555334738069017964572765368676202383777733663092554016662528453673506859039308722997266774215703882910189572515669950219228625268285247127093247506842298199735962820503464375846283255612837293385957365465972163410569651966430738972935921830353168056361568647805569670516775979056080286424754
864813998454770884145537892578230625657287521235062122138020631555593306104251904160638066851442486606842204194500736201728933970315667102641550328277036240586151652358881870758650214926441706430618666225486579444616221252821869621414058795555741449760448603876105266033206752662651304933312833221888315094413176
105975172637756124411538336082185949707243238731543947544516256076567476832144209914689292296900861127830074091767763036035668613803846219570050898405390608448052718820372101386857669442764170560651220383707003184592198056163019700533629403976313106589674739166098678533379653336560892319891903709312801542024623151220222971996228894348643202799736058333099460735220595870005420861512756491316755756691974061561741089524451945278384745173473686593111326997237674371010720956053
src/Main.hx:34: 26701875066099271178044000619981672141942068706083691116674334945443500831647290539569481220837118222534940833492508408521659216191147352032944620139420809036430863861397822506624977900575443791856484144546070632678722764867327132643902247256832064516370494354327936450297983891989175639683082030
368639976721506505305448421269129252355448469099402550574272950366809134141262731718042577866781547936790271318367286543560145334524635332985047323680681186032993275037305097766250981135673634459840384798084993620224986175066508097954591099144657756823378860012478293380990834367690707977072481186175188388845021
877440056546943292958592059841276184454152800144287534206533924029777175544730257737031252617298448472405584559772197941198485662128686185583733260222640936267712480678963131382575674184445721981037523077040530262681514987003076850440198230588401626164510451835201764435338879505153298921078355125750713123388392
528860348019231029575343065793377970796537515635057412429468065450717848282338236683634318353311930509252411787013908109716965362038476942915119334031936586221582659489973799095406188251928242979763338302967070866445308531104380637864599927685844898371549243900608338189534677326908401577164390018082496653198075
204775156899743038968997089043058904925370051554505716433086888675283178304182254343099973998286096610962566813812520354679098238183939853926891169206824392751881871890816735262115820478781573134947668849508122627291002993843586047237779575765883051619648615471247587431713342507690649994425836523345473488513680
551495777790251183145680146522902947681468903988165553275550737079908159447045246312067564003458576968633962193136959644461506814885708941306449419543863763516692988472205465562351733166816830177719989540476933579459215598880099552143506332234461441031540554978015951808681118800144696380187796662604099171375875
987469421252026612604166266362681125888077241273486731449539563358178331265782507833814599365985354743776686395360823265113529118429386004884004765130944945249620564439638713054030972822891054529014735382604287302210367095736248546474401260884275256538001126715308475207057936563374189552150105606436600495686417
347682670655644672013865091803736742847318667752336928651612574528785748372424297139207132167803406057394383641126528943122428325580492625873388336213754004759017937026552319723053275205304285128151509100437432427877282055469529761523860960092281297837697556519546576839960562586236720621011565486530712927683460
054572198906156939021920636467893481346106429486815530377561646247080446987076201432230689490396229155218521101340903538792627811122379828109968802663350743892376412694555415758663905000379726723049880279374728119026070182259645268086072138174276351850721558988379589501225335878225749778677019049743092830910841
839348211260377446828886638344301224849120223682072790724466270542909644470983718596468814678621964448241909404472106088164625430232624852237267587522301333530803084842685821519466943199887922416718163144832697788212649436638629266131318438281115959455032200998240542194211096678267850715664444509468914886477607
09817321888088989547592104761715948995934946752315920004317376929683546984752163759009525722637018852620918800510114663459649909083134266377789816108116450083008213384795351066953977840625893344710376733967855810139706886891315021610553665611509379091181529232242059603673664360539178040759654945774224690357338809

@yuxiaomao do you know if in-lined methods are handled differently than the ones you solved here ?
HaxeFoundation/haxe#11723

[neimanpinchas]

this is not properly adjusted when inlined?

[m0rkeulv]

it could be something like that, the reason why i asked if there's something different done for interfaces when in-linening is because the method called is from an interface.

BigIntImpl is an interface and is used as the underlying type for BigInt.
the 2 types that implements the BigIntImpl interface are Big and Small and these classes have different implementations om the method, so i was wondering if things got mixed up and the wrong implementation was used.

[yuxiaomao]

I confirm that after fixing #685 I have tested your bug and it is not fixed. Interface and abstract both have some special handing as far as I know, will check when I have time.

[yuxiaomao]

The difference is maybe in the bytecode and how hl/jit (jit.c) deal with it:
Bugged: callmethod 6, 0[13](0) (same register for obj and arg, which is rarely generated)
and
Not bugged: callmethod 6, 0[13](10) (different registers for obj and arg, can be produced if remove inline / multiply with another bigInt / use -D hl_no_opt etc)

I'm not very familiar with jit :/ Didn't find a valid fix yet.